Why does Google hate this site?

Something happened the weekend before thanksgiving, Nov 17; a Google algorithm change of some sort that once again took away about a third of our search traffic (it was reported here). This comes on the heels of a first ‘Panda update’ at the end of June which took a third of our search traffic then, and which I wrote about here at the time. Needless to say, this has been a blow to the site on many levels, including its very financial viability. For the past six months I have been taking steps to ‘recover’ from Panda; I see now that this is not going to happen.

The question I really want to know now is: why does Google hate this site? Why did their data refresh or update have a negative, rather than a positive effect on us?. I like this site. I really do, and I would like it even if I were not the one putting it out. I enjoy publishing this site; it is my full time job, it takes a long time to write and to research, and the content is thoughtful and well written. So why does this giant corporation keep punishing us?

It is now almost three weeks since this happened, and it is obviously not a mere thanksgiving-related lull in traffic and shows no signs of letting up. But what is certain is that we are having our lowest levels of traffic (and search traffic) since early 2010. You can see this in the chart below (note that the weekly chart contains a full week’s worth of data for the last data point).

Death - illustrated - p2

In fact, since the first Panda event on June 25th my traffic was steadily increasing, mostly on the strength of newly published content rather than any ‘recovery’. I was hopeful that we will get back to where we where and move on, but all of that was again wiped out last week, by Google’s ‘data refresh’ or whatever it was. And I again find myself seething with anger that this seems to happen for no reason, and angry at my complete inability to affect or influence (or anticipate) what happens to my site and livelihood. I am angry that Google does what it does in purposeful, opaque secrecy, for no apparent reason, and that they deliberately withhold information that affects the lives of thousands of publishers.

Let the guessing game begin:

Here’s my speculation on why this may have happened. This is all shooting in the dark, of course:

(1) Some sort of technical SEO reason

I really wish the reason were as simple as this. About a week before thanksgiving, and based on the recommendation of someone who was advising me on my site experience and SEO, I removed noindex from blog, category, and tag pages, in effect adding hundreds of pages to be indexed that weren’t previously. My taxonomy pages display the titles and thumbnails for posts only (see example here), but the way I implemented this was to hide the excerpts using CSS. What may have happened, therefore, was that I added hundreds of pages of duplicate, invisible content, which in hindsight I figure Google did not like.

Of course, I fixed this eventually, first by reintroducing noindex to these pages, and then by re-instituting it and preventing the hidden duplicate text from being served at all. Its been more than a week though, and my search traffic has not changed or inched back up. I am starting to think that this issue, the only explanation I could think of, is a mere distraction and has nothing to do with what happened.  To my mind, it would be strange to think that Google would punish a blog for having hidden excerpts of it’s own content on its taxonomy pages, but you never know I suppose; they may have thought that I am engaging in some sort of stunt to gain SEO advantage. And how long does it take for Google to respond to changes on a site anyway? I fear its going to be months and months, if at all.

When it comes to Google and search, it seems like the simple act of publishing a blog is sometimes akin to walking through a minefield. Google says ‘oh, don’t overthink it, just focus on creating great content that people want to read’, and that’s pretty much what I was doing. It’s a lie.

(2) Adsense issues, and Google’s end of year results

If you read some of the SEO sites, some users suggest that Google is A/B testing different sites in its SERPs, in order to determine what would best benefit its bottom line in terms of Adsense revenue (with the implication that more profitable sites would be given higher rankings). I am highly skeptical of this, which seems like a conspiracy theory.

But I am very worried that Google doesn’t like my Adsense ad layout (which I discussed previously). So I scheduled an appointment with an optimization guy at Adsense to get their feedback. Adsense are presumably completely separate from Google Search, and the person I spoke with had no information to offer; in fact he was not even aware of any algorithm update (they are never informed, apparently), and had absolutely no input on my ad layout or whether Google search had changed the way they weigh this.

I asked him if Google Search has access to clients’ Adsense data at all, and predictably he said they did not, that search and Adsense were completely separate.

The irony is that Adsense sent out 2 ‘automated’ type emails (on Nov 3rd and 16th) suggesting I move more ad placements above the fold, which is precisely the kind of ad placement that I am worried about from an SEO standpoint.

(3) Some other factor which I am not aware of

I would love to hear ideas that you might have as your read this, because I am completely stumped. If you have any ideas, please share in the comments section below.

The way forward:

I feel like blogging is so fickle and unpredictable that the smart thing would be to get out of it. It really seems to me that the Google people do not know what they are doing, that they put out algorithms that cause uncertainty without promoting the best content, and that maybe this is a symptom that there’s trouble under the surface at Google.

It also seems to me that the model of advertising-supported content is dying. The obvious answer for me would be to find other sources of revenue: selling ebooks, mobile apps, ‘premium’ content, donation drives, etc. But let’s face it: there’s no revenue without eyeballs reading this stuff. For the first time I found myself considering selling my site. It is becoming increasingly clear that publishing a website is a dangerously unpredictable way of making a living, and I find myself wasting time and energy daydreaming about how a small guy like myself can get back at this gigantic corporation that is accountable to no one. I also am thinking about what career I could move to where, like blogging, I could be my own boss, but where I am able to make a semi decent income in the process.


 
 
 
Samer Kurdi

Samer Kurdi

Has been reviewing software since 2006 when he started Freewaregenius.com
flattr this!
  • http://romanianstampnews.blogspot.com Max Peter

    Maybe there are certain companies which really do not love freeware software.
    Just saying…

  • Hardik

    Hi Samer,

    It’s not just your site but many legitimate sites is receiving same kind of treatment. google is going full aggressive and each day finding new ways so you pay them. its like all they want now is money from adwords for all kind of publishers.

    go checkout seomoz and searchengineland there are articles each week describing google’s behaviour. They have reduced natrual SERPS to 7 from 10 in search page. 3 results you see comes from adwords without any warnings. and there penguin and panda update have really broken huge sites without any reasons. i believe google is getting more and more greedy as the day passes. also they are focusing on huge brands that pay them more and there results comes first now instead of small sites. so if cnet is giving them huge some of money they need to give them fever by making there sites all over in first page. in searchengineland you can find article about this.

    my suggetion is keep publishing updated content and find new ways aka bing, fb, twitter and lot other social networks find your users there as with google either you have to pay or they will eventually show the door to nonpaying sites.

  • http://n/a chad tunis

    You have an excellent site. Hope you keep it running. Anything I can do to help, let me know!

  • Toni

    Samer I am reallyreally sorry to hear the site is doing so bad. It is such a great site!

    Ok, I am going to make a really wild guess, but I have some serious advice too at the end:
    The wild guess is: what if the recent drop has not so much to do with Google, but with the sales of tablets and smartphones? PC’s and laptops are loosing terrain very fast.. Advertisements show up differently on smartphones, placing ads above the fold probably doesn’t mean much there. If this is the case, the recent drop in visitors might be explained by Black Friday. Many smartphones and tablets are sold, and this might cause a change. I know this might sound like a farfetched theory, but I thought: what if you are looking in the wrong place? What if it isn’t Google? Bottomline the internet is changing rapidly these days due to these new devices…

    Having said that, I tried a few search terms on Google, things like ‘free media player’ and so on, to see what happened. Whatever I search, I noticed Freewaregenius NEVER showed up on the first page… But I do see that your friends at Techsupportalert do regularly show up in the top ten of results. So, my advice is you talk to them!

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      Toni,
      The future is in smartphones and tablets without a doubt, and we are/will be writing up free apps for those. But whats killing us now is that whereas my site used to show up 1-4 for some terms, now it is showing up 7-10 or even second page for these.
      As for showing up on some terms and not on others, it seems to depend on whether we published a ‘big’ article on the subject. I don’t have an article on the best free media players, for example, and Techsupportalert is a much stronger domain.

  • Beecher Bowers

    It may be due to the number of ads above the fold. They’ve been making statements referring to this in the recent past and I’ve seen a traffic hit on one of my sites that I think is related to this.

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      @ Beecher: I changed my ads in July in order to greatly increase the ratio of ads to content above the fold. I thought it was an acceptable compromise, but maybe I was wrong. I also see many high ranking sites that have ads above the fold, and they seem to not be affected.
      Suffice it to say this constant guessing and second guessing is driving me crazy.

  • Krishna

    Hang in there Sameer. You are doing an excellent job. You will overcome this and others also. May the courage be with you.

    • http://www.reawakeningthepast.com kelvin

      Don’t give up he is right. There is always a way.

  • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

    Thanks guys for your kind sentiments, and for allowing me to vent!

  • Toni

    Samer keep us updated! I’m sorry we can’t come up with a real solution (yet!).

  • kell

    Sorry you are having such a hard time.

    I’m learning to hate Google, but I wonder if Toni is on to something. Because of all the hot gadgets, something newer and hotter every day, the internet is changing. Many of my favorite sites are gone or floundering, or their blogs are all about Android, and iSomethings, and bitty little gadget apps.

    I would be the first to admit that I don’t like change. But there is more to it than that. I work with a desktop so I can use two screens and see the many applications I have open at one time. Therefore, besides the fact that I couldn’t possibly afford a tablet, and a smarphone, and whatever other thingies are the tout of the day, I’m just not very interested in them or the cutsey little apps they use. Yet, that – and Windows8, of course – is all there is to read about now.

    The fun part of the internet – for me – has always been learning about new and interesting PC applications. Once I spent at least 1 and 1/2 hours every day visiting the sites that covered them. Now I’m through in 30 minutes – and that includes my email time.

    I’d hate to see your site turn into a bitty thingamabob blog, and I’d hate it more to see it go away. There are worse things though. Heartbreaking as it can be, sometimes the best thing to do, is walk away while you still have your sanity and your health. Whatever you do, I wish you the very best.

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      Kell,
      If PCs become extinct to be replaced by gadgets and handhelds, we will write about those. We have already started. But the problem is in our search rankings, where once we were 1-4 in the results, now we are bottom of page or second page (and sometimes even lower than that).
      I generally have no problem walking away. Things change, life takes unpredictable turns. But I can’t help feel like the content we put out here is more useful than Google makes it out to be. I can’t help but feel like we are being wronged.

  • Nick Marsh

    Your article is a fascinating example of the impact of Google on the creation of valuable content. I have been a regular – if infrequent – visitor to your site for some years and I am therefore not dependent on Google to find it. I value the site because of the content within it, and specifically the ability to find good software without getting waylaid by poor adware-crippled commercial substitutes. It troubles me greatly that good content like yours might be de-prioritised in Google’s search results. It is clear that Google should consider creating a visible, trustworthy governance structure to maintain impartiality and give transparency and accountability to their decision-making processes. These topics are discussed frequently on the TWiT network podcast, “This Week in Google”, with Leo Laporte, Jeff Jarvis and Gina Trapani, which I recommend highly to those interested in the topics raised in this blog post.

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      Nick, thanks for tip about the TWiT podcast. I will certainly look it up. As for governance and transparency in Google, I would not hold my breath. Its a private company, not an elected body.

  • http://www.financialsamurai.com Financial Samurai

    How is the traffic now compared to during the 1H Panda update?

    The only thing I was surprised about was your above the fold ads (100% ads) before in the 1st half of 2012. Other than that, your site looks good!

    I agree Google should warn us before crushing us. I’m still waiting to be crushed one day, but nothing yet. Traffic has gone from 100K/month to 200K/month visits Jan to now. I don’t know what’s up. I post long articles 3-4 times a month and do no link building. I also have WAY less RSS subscribers than you. Can you give us an idea of your traffic?

    Sam

    btw, your notify me of followup comments is what breaks the comments

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      @ Samurai: you can see the traffic trends in the chart above, as well as the extent of the first and second drops.
      Your site looks great btw. It is outstanding given that you are posting only 3/4 times a month. Hopefully you will not get hit.
      Looking into the notify me of follow up comments issue – thanks!

  • http://geniusgeeks.com Ricky Shah

    Hey Samer,
    Why don’t you just hire some guy to do technical SEO audit? It is worth the investment. At least, you would be able to find out (hopefully) the culprit behind it. Of course, Google Panda is unpredictable and not many can assure recovery. Looking at the quality of your website article, I am sure that it is more of a technical glitch.
    Try guys over at Dejanseo. I often seem them doing free SEO audits on Google+ hangout. There is nothing wrong in offering your website for free/ paid audit.

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      Ricky,
      I had just hired a couple of prominent bloggers to help with ideas to improve the site, a few weeks before this happened. They had plenty of SEO suggestions, but certainly nothing that could have anticipated what happened. I actually doubt that most SEO guys have much to offer, but am looking into Dejanseo as I write this — thanks for the tip!

  • iamnobody319

    Samer,
    I think you need to look into search hijacking… Rob Pegoraro does a nice job of summarizing his similar experience. http://bit.ly/UijNpT
    -iamnobody319

  • http://www.greggdeselms.com/ Gregg DesElms

    It would appear you’re far from alone, Samer. The “InfoPackets” website wrote, today (2 Dec 2012), what they’ve decided to do about it:

    http://www.infopackets.com/news/announcements/2012/20121202_more_content_fewer_ads_now_accepting_donations.htm

    I’m sorry things have not been going well lately. If I could think of a solution, I would; but I’m as befuddled as pretty much the entire rest of the Internet which is also reeling from the most recent Google changes (and the ones before).

    Keep-up, in any case, the good work.

  • Zandt88

    Well after all the great freeware you have provided over the years (not to mention the insightful reviews) I have attempted a form of payback by putting a link to your site on reddit.com in the r/freeware index subreddit. Hope that sends some traffic your way.

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      Thanks man!

  • bruno

    @ Samer,

    1/ I am so sorry to read that you could give up freewaregenius.com
    2/ If there is anything I can do to make freewaregenius.com back at the top of google seraches, let me know (I am ready to launch google searches repetitively to improve your ranking, or any other stupid idea).
    3/ Advertise your donation button. After all, you have a real customer basis, I am sure that they would be happy to contribute a little.
    4/ I will authorize google ads on your site (by deactivating addblock), and will click on links, hoping that this will bring revenue to you (fellow readers, please do the same).
    5/ your site comes up at rank 22 when searching “freeware ” on duckduckgo.com. I am sure that more and more people will to turn to duckduckgo in the future, to get non biased results (hope i am not too optimistic). Any way to improve the ranking here ?

    My 2 cents. Bruno.

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      @ Bruno. Thanks for your kind sentiment (and your support). But PLEASE PLEASE DO NOT click on ads ….. I do not condone it, and it will do nothing for me financially but can get me in lots of trouble. I took the liberty to strike out that part of your comment above.

      I am not advertising my donate button because there is some issue with Paypal, as you have seen they rejected your kind donation a few months ago. I don’t know what their deal is but frankly have been too lazy to fix it since donations are so few and far between.

      As for search rankings, I am not concerned about terms like ‘freeware’ but rather some of the keywords you see next to my logo on the top right. I am working hard on improving the rankings, although mostly it’s guesswork and shooting in the dark.

      But for some strange reason, I am very optimistic. I don’t know if it is the supportive comments from people like yourself or the realization that I must have hit rock bottom so the only way is up ;).

      Anyway, thank you very much, really!

      • bruno

        @Samer,

        Thanks for your kind answer. A couple of comments:
        1/ I was convinced my donation had gone through, I did not check. Too bad.
        2/ I thought that your revenues were coming from people clicking on ads on your pages. A little bit of an explanation would be welcome.
        3/ I feel frustrated that there is nothing I can do to help :-(

        • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

          @ Bruno
          1) they took their sweet time before rejecting the payment. Emailed me to ask what it was and what were the circumstances. They did not like the explanation, apparently, that I run a site and got a donation. After 2 weeks of this nonsense I didn’t have the desire to go back in there and solve the problem.

          2) My revenue does come from people clicking on ads. However, these clicks have to come as part of the normal browsing experience of readers, not by purposeful clicking. The only thing that can come out of purposeful clicking, which can more or less be detected by statistical methods, is trouble for me and undermining my credibility as a serious publisher, and I do not want that.

          3) That one is easy: you already help by leaving comments. If you want to do more, send an email to a few people that you think may be interested in an article or in the site saying “hey check this out”, or share via Facebook or Google plus.

          Thanks man ;)

  • http://www.paleografie.tk Cerberus

    This site, and especially your posts, Samer, are the best on the net. So please don’t give up. I sympathise with your plight. I certainly post links to your site on various other sites, like Stack Exchange’s chat, which is indexed by Google. Take care! I’m sure your traffic will eventually grow back.

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      Thanks, Cerberus. Your frequent comments alone do much to make this site better, and I appreciate them!

  • Prashant

    Hi Samer,

    Raymond from raymond.cc has a similar problem and he is currently, almost entirely, re-writing all the content.
    I hope you have read bout it here : http://www.raymond.cc/blog/updates-x-ray-2-0/

    In the article, he has mentioned about freewaregenius, as well.

    I hope you are taking the same measures that Raymond is to “recover” from Panda!

    Wishing you the best!

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      Thanks Prashant for the link. I read with interest.

      • http://www.raymond.cc Raymond

        Google Panda is a very complex algorithm and there is no silver bullet to fix it.
        Although my site haven’t recover but I know for fact that my site got penalized due to poor and shallow content. All I needed to do was go back to the first post and read from there.

        After reading many panda recovery cases, all of them involves deleting really low quality posts and “rewriting most of them”. Even one of the moderator in WebmasterWorld told me this privately via sticky mail.

        I assume that once you get hit by Panda, it’s not something you can get out of very easily. Probably it’s due to trust signal and also the manual refresh that only runs once a month.

        I wish you all the best.

  • lp009

    Say it ain’t so, Samer!
    You have the best software review site I’ve ever used and the fact that its all about freeware makes it that much better. Hang in there..

    The direction that Google is moving makes it less and less useful for many users. They have been the search of choice for me since they opened their doors but it is becoming more and more difficult to find what I’m looking for using their service.

    * Page after page of links pointing to the same site when I should be seeing multiple sources for the info I’m looking for….
    * Commercial site listings/links trumping sites that have the actual content I want…
    * Limiting their commercial results to paid advertisers instead of listing all available sources..

    etc. etc..

    In the past year, my Google search usage has dropped for 95%+ to about 70% and is steadily falling each month. A search engine that won’t give you what you want without having to wade through page after page of commercial ads and irrelevant content is not a useful tool.

    If they continue down this path they will end up being as useful as Yahoo! (re: ..of No use at all!)

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      Sadly, it is so :( . It doesn’t seem to be getting better.
      But I appreciate your very kind words.
      I too am finding Google’s search results to be of less quality lately. It’d be interesting to see if they do indeed fade into irrelevance.

  • Mountainking

    Hi Samer. While I have been reading, enjoying and learning from this site for 3+ years, it maybe the first time I am commenting on an article. I hope the problem you are in gets fixed real soon. This is a great website with tons of awesome, well written content. I really do hope it all works out well…for you and for us :)

  • Mountainking

    I forgot to add, I was thinking of writing an article and posting on another website but I don’t think I have the time. It was about specifics of searching on bing/g00gle/duckduckgo and maybe yahoo.
    If I do get time, I will give it a thought or share my idea with you.

  • jim

    Samir I sympathize. My business site took a HUGE dip from the algo updates this year. Fortunately I managed to claw my way back up, but it took time, and cost me a lot of money in lost income.

    To get the site ranking, you have to know (a) what keywords you ranked for before, and (b) which keywords you have retained rank for, and which have dropped since the algo updates.

    Once you have that information, you can start optimizing for the keywords that were bringing you the most clicks before the dump.

    BTW, I know a good (affordable) SEO guy for the offsite stuff, if you’re interested.

    But yeah, its all about the keywords. This is the basic information to start fighting for your rank again.

    Jim

    • https://plus.google.com/u/0/110765036334827012485 Samer Kurdi

      Jim: thanks for this, it is very useful stuff. I will email you seperately to get the contact info of the seo guy you mention.

  • Pingback: Top 10 404TS articles in 2012 | 404 Tech Support()

  • GreenHornet

    Samer: As long you use their services, Adsense, Google Analytics (let them know how many visitors your site have daily — and all this information for FREE) nothing will be change.

    The major issue with Google is not their business, it’s their bad relationship and arrogance with the ones providing their basis: webmasters and worldwide IM’ers.

    If all of us webmasters worldwide launches a campaign against google and all of us direct our websites visitors towards using different search engines such as Bing, with the power of banners, story lines, accusations, blocking Google bot wit Robots.txt and don’t use Adsense etc. Google will experience a devastating effect that has a huge chance of going viral trough the whole world.

    These NY Times articles are very helpful because they lend credibility to the situation. It’s not just conspiracy theories from webmasters.

    Google Casts a Big Shadow on Smaller Web Sites:
    http://www.nytimes.com/2012/11/04/technology/google-casts-a-big-shadow-on-smaller-web-sites.html

  • Rob

    Hi Samer, today the situation has improved?

    • Samer Kurdi

      Not at all. It is the worst it’s ever been, sadly.

  • Peter Wills

    Can I suggest that anyone reading this article and the comments post a recommendation for this site on their Facebook and Twitter accounts …. I have.

  • speakn2pabst

    Another regular but infrequent visitor chiming in here. I hope the ship stops taking in water very soon! I’ll throw out another possible unspoken rule that may have weight with Google and which is related to some previous comments; mobile optimized page formatting. I have no idea if this is already addressed with your site’s code (I don’t browse on a mobile device) but with Google’s investment into mobile devices (Android) they probably are up-ranking sites that (attempt to) play nice on those little unreadable screens those devices have. Perhaps you already have addressed this but in case you haven’t I thought I’d throw it out there. I’m sure this is the kind of decision Google feels comfortable making (“We’re acting democratically by up-ranking those blessed souls who make effort to support ALL devices..(especially these we have financial investment in)”) never mind the fact they make these type of decisions without transparency or discussion or forewarning of changes. Just another personal theory without any evidence, for what it’s worth. Best of luck to you and this blog!