Beware of marketers bearing statistics. A recent press release claiming to reveal the happiest states in the union raised my ire for its sloppy use of statistics. It didn’t make the cut as being worthy of an article.
My inbox is flooded every day with some new survey sent by someone who wants me to use it in an article. Why? Because it might drive a potential customer to that marketer’s website. The world of advertising for print media runs on tear sheets, but the internet runs on hits.
Anything that can increase hits on a website is good. A press release from a marketer may be just another tool to drive hits to an e-commerce site even if the information on the release is bad. I have three examples of press releases that illustrate this phenomenon, including the study on the happiest states
In my very first month of working for a newspaper, my editor passed two releases to me. Both were from businesses with websites designed to generate revenue. The first press release claimed that Idaho drivers were some of the nicest and most polite drivers in the country. The second press release claimed that Idaho led the nation on road rage. Both companies asked for linkbacks to their e-commerce websites. Visions of drivers suffering from extremely polite road rage prevented me from using either of these two releases.
Welcome to the world of press releases used as marketing tools. The practice has been around for many years; however, the demand for a constant stream of internet content has exaggerated the problem, especially when any bozo with a blog site can claim to be a journalist — and blogs live or die by delivering content on a schedule.
It doesn’t seem to matter whether the content of a press release is based on good data or is merely lying with statistics. That’s because the objective is to convince a reporter to hand out free advertising in the form of an article — with a linkback, of course. That’s because even bad data will still generate hits. Good information is not a priority.
My second example involves a case where the truth took a back seat to driving website hits. I wrote an article trashing someone’s press release. The release in question contained a data analysis of public unemployment statistics. Its mistake was using the drop in initial unemployment claims as a proxy for job recovery despite being independent variables. You can’t measure jobs recovered by looking at jobs lost. They aren’t the same thing.
What happened next was illuminating: the e-commerce business that authored the press release contacted me to request a linkback to their website despite the fact that I trashed their study in print. In this case, correct information and even the business’s credibility were not a concern compared to the marketers’ purpose of driving those hits to the firm’s website.
Unfortunately not every news outlet has someone like me with a solid background in statistics and data analysis. This leads to many publications recycling the contents of press releases for filler on slow news days without vetting the information they contain.
My last example is the recent press release that claimed to identify the happiest states in the union. Beyond the shaky claim that happiness can be measured, Idaho made the top ten based on an analysis of three weighted measures, each of which was a composite of even more weighted measures.
The results surprised me, especially since Utah was the second-happiest state while Idaho was ninth. The reason for the surprise is that Utah is infamous for having high rates of clinical depression and high antidepressant usage. Depending on the survey and the year, it has often been the worst in the country. The Utah result alone led me to question this study.
From a purely subjective and personal viewpoint, I will concede that Idaho is a happier place than many other states and certainly happier than Utah. Idaho’s top-ten ranking did not surprise me. I base this on having lived in some very depressing places, like northern Maine where the economy outside of tourism has collapsed. Simply put, not being able to find a job creates a lot of unhappiness.
Well, that’s my opinion and opinions are like sphincters: everyone has got one, and some have two. I will concede that there could be ways to measure happiness on a per-state basis so I took a closer look at the methodology of this study. The study took 32 published measures and added them using its own weighting formula.
Like the unemployment study, the choice of several variables was flawed. For example, one of the weighted variables was the number of hours spent working. That’s not a good proxy for happiness because a person working 20 hours a week who wants to work full-time can be miserable, whereas someone working 50 hours a week at something he or she loves can be quite content. Adverse childhood experiences was another dubious measure because many people with horrible childhoods become happy adults. The choice of variables matters.
The biggest problem with the happiness study was inappropriate design. The study raided pre-compiled data, all of which someone else assembled. Raiding someone else’s data doesn’t cost much money, which is probably why this e-commerce company used this methodology; but you get what you pay for.
The best study design to measure a subjective quality like happiness would be a real survey of real people, thousands of them, to ask them if they are happy and why — but that costs real money. When the object is to create a press release to generate website hits or pad out a blog, it’s not cost-effective to run a correctly designed statistical campaign. Dressing up bad design and variable choice in the objective-appearance of a weighted study is not just bad analysis: It’s lying with statistics.
Catie Clark is an Idaho Business Review staff writer who covers mainly development and real estate.