UPDATE: Matt Cutts of Google joined in on the discussion over this article at Hacker News. Looks like a site that was sitting on a sub-domain was indeed hacked and that had triggered the warning. Matt stated in plain language that the split testing approach that I described does NOT trigger the warning.
I’m a big fan of testing and my tool of choice is Visual Website Optimizer. It can help you make more money and it can prevent you from making costly decisions. However, today I’d like to throw in a word of caution when doing conversion rate optimization.
There are a few different ways to do a/b split testing.
This is where you make changes to the page on the server, before it’s sent to the visitor’s browser.
The advantage of this approach is that the page that is sent back is what is displayed to the end user. In addition, you have a lot of control on what exactly gets sent back to the visitor.
The disadvantage of this is that it’s complicated to implement. There are helpful tools like A/Bingo available but it means that you’re going to need to get a web developer involved.
The advantage of this approach is that it’s a lot less work, a non-programmer can actually get started with testing in less than 10 minutes and it’ll work on pretty much all modern browsers.
The disadvantage is that the page sent back may be different than what is displayed and Google might not like that. So far, they’ve been okay with it from what I can tell.
Conversion Rate Optimization is critical to optimizing the ROI from Adwords campaigns and I believe it’s in Google’s best interest (both from a visitors perspective and an advertisers) to encourage testing.
Split Page Variations
Inspiration For My Test
A few days ago I watched an excellent webinar on MarketingExperiments.com called The 5 easiest changes to make to your landing pages right now. (For those interested, here’s a mindmap of what was covered.) After watching the video version of the webinar, I decided to do what I normally do when I have/gather an idea: test, test, test! This was at 4 in the morning so I created a design. This turned into a variation of the home page at is visible here. After looking at for a bit longer, I made some more changes and created this version (which is currently the home page.) Happy with how it looked and sounded, I went into Visual Website Optimizer and prepared the test.
Setting Up The Test
Since the pages were so different, I decided to use the split page approach. I setup the test to ignore everyone that are probably not prospects by excluding everyone who isn’t in the US, UK, Australia or New Zealand and excluding everyone who visited the site using a search phrase that didn’t demonstrate that they had the intention to buy. For example, we offer a coupe popular free WordPress themes called Motion and Pixel. I excluded everyone that visited the site by searching for “motion” or “pixel” or “wordpres” or theme, etc. When it was all setup, people who would participate in this test would be good potential clients. After making sure that everything was setup properly, I activated the test.
What Happened After Test Went Live
Visitors started visiting the site and the numbers started rolling in. All was good. I took off for the day and worked on client projects. On the way back home that afternoon, I decided to do a quick sanity check on Google for my rankings. We rank well for a few terms and I’ve developed a habit of verifying where we’re placed every morning and evening. Usually, I search on my phone, verify it’s there and close the browser but this time wasn’t as simple. When I did a search, I saw this:
“This site may be compromised” freaked me out and my gut reaction was “F***!” I raced to the nearest Starbucks so I could get on my laptop and see what was going on. I was also pissed at Sucuri as I had paid them specifically so they could tell me if my site ever got compromised and here I was finding out from Google instead.
When I got on my laptop, I logged into Webmaster Tools. This where Google communicates with site owners about their sites. I logged in and sure enough, there was a message in there. Here’s the message:
Dear owner or webmaster of http://www.webdesigncompany.net/,
We are writing to let you know that some pages from http://www.webdesigncompany.net/ will be labeled as potentially compromised in our search results. This is because some of your pages contain content which may harm the quality and relevance of our search results. It appears that these pages were created or modified by a third party, who may have hacked all or part of your site. Many times, they will upload files or modify existing ones, which then show up as spam in our index.
The following are some example URLs which exhibit this behavior:
These URLs are using practices which do not follow our quality guidelines, which can be found here:http://www.google.com/support/webmasters/bin/answer.py?answer=35769&hl=en.
We encourage you to investigate this matter in order to protect your visitors. If your site was compromised, it’s important to not only remove the malicious content from your pages, but also to identify and fix the vulnerability. Note that in many cases, this malicious content is hidden through a process known as cloaking. To learn what cloaking is, seehttp://www.google.com/support/webmasters/bin/answer.py?answer=66355&hl=en. You can confirm if your site is cloaking by using the Fetch as Googlebot tool:http://www.google.com/support/webmasters/bin/answer.py?answer=158587&hl=en. A good first step towards resolving the problem is to contact your web host’s technical support for assistance. It’s also important to make sure that your website’s software is up-to-date with the latest security updates and patches. More information about how to fix your site can be found at:
Once you’ve made sure your site is clean and secure, you can request reconsideration by going tohttps://www.google.com/webmasters/tools/reconsideration?hl=en.
Google Search Quality Team
Specifically, it told me that a core WordPress file was the culprit. I opened up the file but I saw no signs of compromise. I looked through the rest of the site and did a fine grain search through the code for all the popular type of WordPress hacks. Nothing showed up.
I decided to download a copy of the site and reinstall WordPress to make sure the site was 100% clean. After the reinstall, I downloaded a second copy of the site and compared it. Both copies were identical. I used the Google Fetchbot and the results looked clean. Hmm! I was stumped. I decided to take a break so I could get some distance from the problem.
When I got back an hour later, I remembered that I had started the testing campaign earlier today and that a lot of times, malware on websites use the same technique to redirect visitors. That’s when it hit me. Google was thinking that I was compromised because my testing code was automatically redirecting a certain percentage of users. Another reason I think this might be the case is that the site isn’t blacklisted and Google’s Webmaster Tools doesn’t say that there are any malware on the sites. I immediately stopped the test and submitted the site for reconsideration. I don’t have anything to back this up but it’s the only thing that makes sense. If this is true, than Sucuri is at no fault.
Anyone else see something that I don’t?