Split Testing May Cause Google to Accidentally Think You’ve Been Compromised

by Melvin Ram on May 6, 2012

UPDATE: Matt Cutts of Google joined in on the discussion over this article at Hacker News. Looks like a site that was sitting on a sub-domain was indeed hacked and that had triggered the warning. Matt stated in plain language that the split testing approach that I described does NOT trigger the warning.

tl;dr: I setup 2 different pages with different url’s that I wanted to test. The VWO javascript automatically redirected equal portions of the traffic to the test pages. Google marked my site as being compromised. I double checked everything and there wasn’t a sign of a compromise. Through process of elimination, I suspect it was redirecting of the page through javascript that triggered the warning sign.

I’m a big fan of testing and my tool of choice is Visual Website Optimizer. It can help you make more money and it can prevent you from making costly decisions. However, today I’d like to throw in a word of caution when doing conversion rate optimization.

There are a few different ways to do a/b split testing.

Server-Side Variations

This is where you make changes to the page on the server, before it’s sent to the visitor’s browser.

The advantage of this approach is that the page that is sent back is what is displayed to the end user. In addition, you have a lot of control on what exactly gets sent back to the visitor.

The disadvantage of this is that it’s complicated to implement. There are helpful tools like A/Bingo available but it means that you’re going to need to get a web developer involved.

Client-Side Variations

The alternative to server side variations is client side. Using the power of javascript, tools like Visual Website Optimizer & Optimizely will allow you to make fine grain changes to your page and present the different variations as needed.

The advantage of this approach is that it’s a lot less work, a non-programmer can actually get started with testing in less than 10 minutes and it’ll work on pretty much all modern browsers.

The disadvantage is that the page sent back may be different than what is displayed and Google might not like that. So far, they’ve been okay with it from what I can tell.

Conversion Rate Optimization is critical to optimizing the ROI from Adwords campaigns and I believe it’s in Google’s best interest (both from a visitors perspective and an advertisers) to encourage testing.

Split Page Variations

This approach is sort of a hybrid of Client-side and Server-side variations. Here’s how it works. Let’s say you want to test your landing page at yourdomain.com/landinag-page. You would create additional pages and using javascript, the visitor would be redirected to one of the pages. This allows you to keep things simple on the server side and tools like Google Website Optimizer, Visual Website Optimizer and others can help you equally split up your traffic between the different pages and track the conversion rates for each page. This is how split testing actually used to be done when it started getting popular and is continued in a number of different tools.

Inspiration For My Test

A few days ago I watched an excellent webinar on MarketingExperiments.com called The 5 easiest changes to make to your landing pages right now. (For those interested, here’s a mindmap of what was covered.) After watching the video version of the webinar, I decided to do what I normally do when I have/gather an idea: test, test, test! This was at 4 in the morning so I created a design. This turned into a variation of the home page at is visible here. After looking at for a bit longer, I made some more changes and created this version (which is currently the home page.) Happy with how it looked and sounded, I went into Visual Website Optimizer and prepared the test.

Our Company Columbus

Our Company Columbus

Setting Up The Test

Since the pages were so different, I decided to use the split page approach. I setup the test to ignore everyone that are probably not prospects by excluding everyone who isn’t in the US, UK, Australia or New Zealand and excluding everyone who visited the site using a search phrase that didn’t demonstrate that they had the intention to buy. For example, we offer a coupe popular free WordPress themes called Motion and Pixel. I excluded everyone that visited the site by searching for “motion” or “pixel” or “wordpres” or theme, etc. When it was all setup, people who would participate in this test would be good potential clients. After making sure that everything was setup properly, I activated the test.

What Happened After Test Went Live

Visitors started visiting the site and the numbers started rolling in. All was good. I took off for the day and worked on client projects. On the way back home that afternoon, I decided to do a quick sanity check on Google for my rankings. We rank well for a few terms and I’ve developed a habit of verifying where we’re placed every morning and evening. Usually, I search on my phone, verify it’s there and close the browser but this time wasn’t as simple. When I did a search, I saw this:

“This site may be compromised” freaked me out and my gut reaction was “F***!” I raced to the nearest Starbucks so I could get on my laptop and see what was going on. I was also pissed at Sucuri as I had paid them specifically so they could tell me if my site ever got compromised and here I was finding out from Google instead.

When I got on my laptop, I logged into Webmaster Tools. This where Google communicates with site owners about their sites. I logged in and sure enough, there was a message in there. Here’s the message:

Dear owner or webmaster of http://www.webdesigncompany.net/,
We are writing to let you know that some pages from http://www.webdesigncompany.net/ will be labeled as potentially compromised in our search results. This is because some of your pages contain content which may harm the quality and relevance of our search results. It appears that these pages were created or modified by a third party, who may have hacked all or part of your site. Many times, they will upload files or modify existing ones, which then show up as spam in our index.

The following are some example URLs which exhibit this behavior:

  • http://www.webdesigncompany.net/wp-includes/js/comment-repl

These URLs are using practices which do not follow our quality guidelines, which can be found here:http://www.google.com/support/webmasters/bin/answer.py?answer=35769&hl=en.

We encourage you to investigate this matter in order to protect your visitors. If your site was compromised, it’s important to not only remove the malicious content from your pages, but also to identify and fix the vulnerability. Note that in many cases, this malicious content is hidden through a process known as cloaking. To learn what cloaking is, seehttp://www.google.com/support/webmasters/bin/answer.py?answer=66355&hl=en. You can confirm if your site is cloaking by using the Fetch as Googlebot tool:http://www.google.com/support/webmasters/bin/answer.py?answer=158587&hl=en. A good first step towards resolving the problem is to contact your web host’s technical support for assistance. It’s also important to make sure that your website’s software is up-to-date with the latest security updates and patches. More information about how to fix your site can be found at:

http://www.google.com/support/webmasters/bin/answer.py?answer=163634&hl=en

Once you’ve made sure your site is clean and secure, you can request reconsideration by going tohttps://www.google.com/webmasters/tools/reconsideration?hl=en.

Sincerely,

Google Search Quality Team

Specifically, it told me that a core WordPress file was the culprit. I opened up the file but I saw no signs of compromise. I looked through the rest of the site and did a fine grain search through the code for all the popular type of WordPress hacks. Nothing showed up.

I decided to download a copy of the site and reinstall WordPress to make sure the site was 100% clean. After the reinstall, I downloaded a second copy of the site and compared it. Both copies were identical. I used the Google Fetchbot and the results looked clean. Hmm! I was stumped. I decided to take a break so I could get some distance from the problem.

When I got back an hour later, I remembered that I had started the testing campaign earlier today and that a lot of times, malware on websites use the same technique to redirect visitors. That’s when it hit me. Google was thinking that I was compromised because my testing code was automatically redirecting a certain percentage of users. Another reason I think this might be the case is that the site isn’t blacklisted and Google’s Webmaster Tools doesn’t say that there are any malware on the sites. I immediately stopped the test and submitted the site for reconsideration. I don’t have anything to back this up but it’s the only thing that makes sense. If this is true, than Sucuri is at no fault.

Anyone else see something that I don’t?

{ 4 comments… read them below or add one }

James Sandberg November 25, 2012 at 12:17 am

I’m going to sign up for Google’s webmaster tools right now to get in on this testing action!!!

Reply

Hilary January 1, 2013 at 4:56 am

Great article about starting a/b testing! Couldn’t agree more that a/b testing will save you tons of money in the long run. Test, test, and test some more!

Reply

Michelle January 12, 2013 at 6:14 pm

Wow! You seem to know a lot about this. Do you design websites? I am trying to figure Wordpress out. Can you point me in the right direction? How to start if I don’t know how to code but I know what I want done? I am from South Africa and have just started a new company and organization. Not sure if you can help, trying to get hold of the people at man and mouse but i can’t find their email address.
Shot!

Reply

Melvin Ram February 11, 2013 at 3:34 am

If you fill out the request price quote button, I might be able to help you understand your options better.

Reply

Leave a Comment

{ 1 trackback }

Previous post:

Next post: