If there’s one phrase that sums up the state of the media industry these days, it might be: Out with the old, and in with the new. For the first time ever, the industry is speaking in unison and saying we need a new direction in measurement. The antiquated monopoly built for the needs of the 1950s is finally being set aside for a modern approach.
In a recent post on this site, Nielsen took the curious position that, in a world where concepts like machine learning and blockchain are redefining entire sectors of the global economy, big data and new approaches are something they fear. Perhaps this false agenda is because Nielsen has not been able to incorporate large-scale data into their own methodology in a credible way. They continue to rely on small panels with single-digit response rates that ignore most of the population as the foundation of their television measurements. It’s natural that they would try to refute modern approaches when they are so tied to the past and legacy methods. Simply proclaiming that their panels are “sources of truth” doesn’t make it so. It is good to hear Nielsen acknowledge Comscore as the alternative to this old measurement approach. Our more than 3,000 worldwide customers would agree with them. With Nielsen’s limited experience in this new world, it is misguided for them to discount Comscore’s more than a decade of experience and knowledge in creating stable, reliable, and predictable local and national measurement currencies using census-scale data.
Given the extent of the fallacies and mischaracterizations in Nielsen’s commentary, it would be tedious to do a point-by-point rebuttal. Instead, let’s take a step back and look at the big picture, in a particularly illuminating example. When the COVID-19 pandemic changed our world in 2020, the effect of it on television viewing was evaluated by Nielsen using small panels, and by Comscore using big-data assets. Nielsen concluded that TV viewing declined during the pandemic, whereas Comscore found the opposite: a substantial increase in TV and connected TV consumption. One was right and one was wrong. Given that history, why would anyone believe Nielsen’s self-proclaimed “source of truth”? The biggest event in generations was measured incorrectly by small panels.
What is really important when it comes to understanding the future of media measurement is transparency and collaboration. The shift in distribution platforms and advertisers’ interest in audiences — going beyond age and gender demos — requires modern thinking, modern data assets, and modern technology. Advertisers want to use actual, observed consumer information to evaluate their media choices. This approach, based on census-scale data, is the norm in digital, OTT, addressable, and connected TV (CTV) measurement. Nielsen’s commentary challenged the representativeness of return-path data (RPD), but in fact, the data comes directly from subscribers of certain TV services, without a need for high-touch recruitment. Nielsen ought to understand this since they too have begun to use RPD in their measurements. The problem is they are still relying on their unstable small panels as their starting point and not on the tens of millions of household actual observed viewing as Comscore does. Indeed, the RPD data that Comscore uses, comprising more than 35 million households and 70 million devices across all 210 local markets, aligns quite well both geographically and demographically with the country as a whole. These are the facts.
Contrast these facts with the reality of small panels today: most people who are invited to participate refuse to do so, and a few tens of thousands of respondents are inadequate to measure the fragmented media space of the world today. Small panels were a great idea 70 years ago. Today, not so much.
Can Nielsen really refer to their panel as “robust” when their national and local ratings accreditation was suspended after months of scrutiny in large part due to the unreliability of said panel?
There are fundamental challenges when it comes to being dependent on small panels, something I covered more extensively in a recent blog post. The most obvious problem is the low response rates. This means you are only measuring the rare people who are willing to participate; nothing is known about the refusers (and we know today that most consumers don’t want to be hassled with). Second, ongoing media fragmentation means it is harder to measure with a small panel. For example, a 1 rating is increasingly hard to achieve, and with a panel of, say, 50,000 households, that rating is based on only 500 of them. And it only goes down from there. The resulting estimates are unstable, fluctuating, and unreliable – especially at the local level – with a plethora of “zero cells” where the content audience is too small to be seen on the panel. With billions of cumulative dollars at stake, can advertisers afford to have such a limited and flawed audience measurement? More and more, the industry is saying no.
The Nielsen commentary suggests that big data and modern methodology cannot measure who is watching what as well as a small set of personal people meter (PPM) panelists. If that were true, then a comparison of Comscore’s estimates to PPM estimates would demonstrate it. In fact, for content that has a large enough audience that it can be measured by a small panel, the two approaches line up well, as seen for example in the figure below. My colleague Kumar Rao and I have described our methodology in more detail here.
Preparing for the Future of Media
It’s clear we are in a new era of media. To meet the needs of what the industry needs now, not years from now, Comscore has taken a measurement approach with the imperatives of reliability, inclusion, and stability in mind. We are proud to have built what we believe is the best methodology in the market. We have enormous scale – there is one Comscore-measured home in every three U.S. TV homes – and a data-driven/census-like approach that’s impossible to achieve with small samples.
Simply put, we believe that we provide the best insights into how today’s modern, cross-platform audiences consume media. We are proud that more than 1,000 local stations, 150 national networks, 50 station groups, and the world’s 10 largest advertising and media agencies trust us to provide them with actionable audience measurement that drives better business outcomes.
Nielsen can continue down its lonely path of being a company unwilling to embrace new technologies. Comscore is focused on the future, not the past; after all, that’s what the industry is demanding. As we enter the next generation of media, where agencies and advertisers are taking an audience-centric approach to planning and buying across myriad platforms and requiring impressions to evaluate their success, Comscore believes the time has come to redefine "currency." The future of media has arrived.
