With all the data coming at us like an out-of-control firehose, it’s amazing that so many critically important things are seemingly being dominated by one guy.
Take the second Presidential debate this past weekend between Hillary Clinton and Donald Trump at Washington University in St. Louis. It was supposed to be a town hall meeting, but as we continue to learn from this campaign, nothing ever goes according to plan. Only a handful of the “undecided voters” in the audience even had the opportunity to ask the candidates a question.
But strangely, one of the last questioners – a 34 year-old schlub named Ken Bone – asked a particularly good question about energy policy. Somehow, this guy has blown up all over the Internet, appearing on national talk shows and becoming the meme of this debate. Maybe it was his tacky red sweater or it was America’s collective frustration with the process. Assuming we still remember his name in two weeks, there are even Ken Bone Halloween costumes being made (less scary to many kids than a Hillary or Trump mask). Somehow, the need to make Bone a pop culture sensation sums up our obsession with explaining away complicated phenomena by using a single person as the poster boy.
But in fact, one person can make an awfully big difference in the way elections – and radio ratings – turn out. While large samples of people can provide a high degree of research stability, we continue to revert to staring at a single individual, trying to figure out what’s really going on out there and what the data are trying to tell us.
Take the USC Dornsife/Los Angeles Times Daybreak poll, for example. If you follow national presidential polls, it has long been an outlier. Trump has held the lead in this survey for months, despite being behind Hillary Clinton in most other polls.
Now it turns out that thanks to transparency, the unusual outcome of this poll may be due to a single panelist – a 19 year-old African American man who is a Trump 
supporter. In other words, an anomaly.
This week, “The Upshot” column in The New York Times offered statistical theories about how a single person – this one guy – can impact a poll consisting of approximately 3,000 panelists. In “How One 19 Year-Old Illinois Man Is Distorting National Polling Averages,” journalist Nate Cohn details how a single respondent can hijack an important political tracking survey. (It’s a bit wonky, but a good read.)
Here is some of the reasoning behind how these poll results ocurred, and yes, this will sound very familiar to radio people who are used to looking at ratings data:
- Weighting – Because of this one guy’s demographic pattern, he could be weighted as high as 30 times more than the average panelist in the survey. Of course, weighting has been a ratings constant from Arbitron to Nielsen. Old-timers will remember the old PPDV values – Persons Per Diary Value – that upweighted small numbers of respondents who represented groups hard to convince to participate in the survey. Weighting is a part of many research surveys, but when it reaches extremes as it obviously has with the USC/LA Times poll, erratic outcomes can occur.
- Panels – Many of the same people participate in this poll, so when something goes wrong with a respondent or two (see #1), that mistake can be amplified across multiple surveys. “The Upshot” notes the USC/LA Times poll ran into “terrible luck” because the highest-weighted respondent was atypical of his demographic group. That is what has produced an odd polling result in their survey, and it’s often what happens in local market ratings report as well.
Now there are other issues with the USC/LA Times poll, not the least of which is weighting a respondent’s past Presidential vote from the 2012 election. What that has to do with the current political climate is anybody’s guess. And it’s not their actual vote – it’s who they say they voted for four years ago.
But once again, we see the impact that single respondents can have on panel surveys. And that brings us to Nielsen, and their newest feature: Control Panel Reports for PPM subscribers. For several years now, Nielsen has been “helping” stations understand “panel shifts,” as well as the impact that a single meter holder can have on the outcome of weekly and monthly ratings.
Now this new report helps make it official, tracking a station’s top 10 listeners to illustrate how their listening shifts during a six month period.
As Nielsen’s John Snyder explained in Inside Radio earlier this week, these changes in listening often have nothing to do with anything the station’s doing.
“What we’re seeing is things absolutely change in our panelists’ lives and, for one reason or another, their listening will vary greatly from month to month. It could be they took a new job with a longer commute, began driving a son or daughter to school or that their car is in the repair shop; or it could be any number of other lifestyle changes that impact their listening.”
Lifestyle changes. Not your new music test, your direct mail campaign, your morning show’s new benchmark feature, or Labor Day. When one person can make that much difference in the outcome of a survey, external issues that have nothing to do with radio programming, promotions, or contests can significantly alter findings. And that sometimes leads to programming adjustments that simply aren’t necessary or even beneficial to the brand.
We see this in our annual Techsurveys when we examine people who indicate having listened to less broadcast radio during the past year. When we drill down deeper as Nielsen is doing, we learn that “lifestyle change” is always among the top reason that explains listening erosion:
When programmers fixate on individual listeners, it takes their focus away from the intrinsic value of their station brands. And it’s a trap that often influences a strategic change that might end up causing even more damage to the station.
It can be comforting to try to single out that one respondent who is creating havoc with the outcome of the ratings, but it’s never smart. Back in the ‘90s, there was a hit sitcom called “Frasier,” the story of Frasier Crane, a neurotic radio shrink who made everyone’s life miserable.
There was an episode called “The Focus Group” where 11 of 12 respondents loved Crane’s show. But there was “the one guy” who had a problem with Frasier that he couldn’t quite articulate. The rest of the episode was devoted to Frasier tracking him down to get to the bottom the problem.
In polling – whether it’s Presidential politics or in radio – we can be guilty of the same obsession, and in the process, sometimes causing irreparable harm to our brand and our sleep patterns.
It does make you wonder now that both the Clinton and Trump campaigns know about this “one guy” in Illinois, if they’re not sending out vehicles to Illinois to locate him, and shower him with swag, prizes, and other incentives.
That’s the way we’d handle it in radio.
Jacobs Media has consistently walked the walk in the digital space, providing insights and guidance through its well-read national Techsurveys.
In 2008, jacapps was launched - a mobile apps company that has designed and built more than 1,300 apps for both the Apple and Android platforms. In 2013, the DASH Conference was created - a mashup of radio and automotive, designed to foster better understanding of the "connected car" and its impact.
Along with providing the creative and intellectual direction for the company, Fred consults many of Jacobs Media's commercial and public radio clients, in addition to media brands looking to thrive in the rapidly changing tech environment.
Fred was inducted into the Radio Hall of Fame in 2018.
- Hey Twitter, Gotta Go - November 12, 2024
- Kicking Ryan Seacrest’s Butt - November 11, 2024
- Get Short(y) - November 8, 2024
HUGE!
Excellent- should be required reading !
Appreciate that, Tom. Hope all’s well.