This Is Not a Test!

It’s an article about testing.

In a recent article in BrandWeek, Karen Gedney was interviewed for some pointers on tricks of the trade in today’s online marketing environment. As the article points out, behind all the buzz about online marketing lies … principals of Direct Marketing 101. The legacy tactics may be in decline – the Age of Spray ‘n Pray is behind us – but testing, digging deep into results and incorporating them into future campaigns or efforts is still what it’s all about.

In my early experience working at Global Business Research, a conference company, we were constantly testing list segments and acting on results. However, I decided to test paper stock – we (and other events organizations – which operated in a “me too” environment) traditionally printed brochures on 80# coated stock. I decided to test uncoated vs. coated – obviously would not have shiny brochures, but would it impact attendance numbers? End of the day it didn’t – a simple A/B between the mail drop produced no variation in registrations; moving forward we saved $ by printing on uncoated stock.

Obviously testing can be more complex – while at Gartner I hired Karen for her creative services for several campaigns for us, including one in which we tested single vs. dual campaign communication for two events running a week apart which had a large audience overlap (another issue for another day).

Today, of course, the Internet has brought testing to a new level – both in timeliness and complexity. Results can pretty much be gleaned in real-time now, and action (if need be) taken much faster. To that end, a couple of quick suggestions on methodology:

• Keep email testing simple, utilizing A/B splits to test subject lines, messaging, offers, etc. independently of each other. Get a grip on one element before testing others. Given the challenges in generating clicks and views via the email channel, I’d suggest keeping it simple – as numbers will be relatively small, you want to have a good sample size from which to make decisions.
• Use your Web site for more complex multi-variable testing. Work with your internal Web team to ensure proper analytics are in place (and work with someone versed in statistics to ensure the test is viable), and use the Web to test headlines, copy placement, images, etc. – with the ultimate driver of course being registrations (although measure of leads generated vs. conversion is also a good metric.) Your site visit numbers will be more robust than email results, so take advantage of the larger audience to gain insight from larger datasets.

In short, use online marketing as a channel – but keep the legacy practice of testing in place as you develop and roll out your online initiatives!

Good luck!

John

P.S. Thanks for all the positive feedback I’ve gotten on http://www.johnsgibb.net. In the process of working on some additional efforts – stay tuned!