Some of the 14,000 tourists a day who pass through The District, the sleek shopping mall inside the MGM Grand in Las Vegas, do not come here to shop. Instead, in groups of two dozen at a time, they arrive in one of the rooms tucked next to the food court and an attraction called CSI: The Experience. Inside these battleship-grey, fluorescent-lit rooms, rows of classroom-style tables are packed with Dell computers turned upright. The tourists sit at the computers, look at a 50-inch screen mounted to a wall, and receive a set of instructions. Their task is to do nothing less than determine the fate of TV programming. They are the engines of a little-understood factory of popular culture known as CBS Television City, open for business 12 hours a day, 365 days a year.
The facility, a closely guarded secret at the network, houses the latest in technology that tracks human emotion and impulse: sensors to track the facial expressions, eye movement, blood pressure and skin texture of viewers. Manipulating dials and answering researchers’ questions in focus groups, the recruits help shape the newest batch of CBS pilots as the network finalizes its fall schedule in time for the annual upfront pitch to ad buyers. Weeks ago, new CBS shows including Supergirl and Code Black were under the microscope in these very rooms.
Just a few hundred feet away, in an office outfitted with a one-way mirror and monitors displaying the rooms’ activity, stands the group’s keenest observer and the chief architect of this vast hall of research: David Poltrack, chief research officer of CBS. For him, it’s another day at the office. “We get a range of reactions here,” Poltrack shrugged.
Poltrack has been a CBS executive since 1969, coming aboard not long after abandoning a lawyer’s path out of Notre Dame and deciding to try the TV ad game. In conversation, his expression has impassivity and wisdom to it, and he teaches at NYU and Columbia, but he didn’t get to be CBS chief Leslie Moonves’ trusted lieutenant by merely being a sage. Poltrack’s bulldog side made him an Internet meme and industry hero in 2010, when he literally called bullshit on a Forrester Research exec arguing on a panel that TV ad revenue was disappearing forever. Like jazz musicians with notes or sculptors with blocks of clay, Poltrack senses numbers. A smile plays at the corner of his mouth when he speaks of the 44 measurable muscles in the human face, and the correlations between focus groups and Nielsen ratings. “Before there was Big Data, there was Dave,” said Stephanie Gaines, who worked for Poltrack as a marketer at CBS and now is VP of corporate marketing for the ad-tech firm YuMe.
“In our business, if you have one hit show, you have to go and make 21 more episodes after the pilot,” Poltrack said deliberately. “So you’d better understand everything there is to understand about what made it appealing to the audience.”
Ad-supported, primetime broadcast programming has never been more threatened—by online or cable competitors as well as by the Pandora’s boxes of time-shifting and mobile options. A recent MoffattNathanson report found traditional TV viewing has dipped 9% year-to-date as viewers seek out alternatives on SVOD or elsewhere. Pilot season is perhaps the ultimate expression of this vulnerability, with long odds confronting any new hopeful. Last week, from May 11-14 in New York, networks officially rolled out their 2015-16 schedules and pitched ad buyers, but their program batting average remains low. In 2013-14, they announced 48 new shows and only 15 returned, or 31.2%.
This year was closer to 50%, a possible sign the factory is gaining efficiency. But seldom is ratings gold spun out of a great script and an executive’s hunch. Networks now use every technological means possible to track the facial expressions, eye movement, blood pressure and skin texture of viewers and to understand the DNA of a hit. They throw budget constraints to the wind, spending tens of millions per cycle in pursuit of the next franchise.
‘Your Voice Is Going to Count’
CBS opened Television City in 2001, perhaps not coincidentally at the same time it was about to ride shows such as CSI and Survivor to the total-viewership ratings lead it has virtually never relinquished. The company frequently boasts about the facility, but it keeps an ultra-tight lid on the operation lest any rival in the hyper-competitive broadcast race gain an edge. One PR exec volunteered that on his first day at the network years ago, Moonves called him into his office and, after a few pleasantries, said pointedly, “Your No. 1 job is to protect our process.” Poltrack even demurred when asked for success stories, hit shows forged in the fires of Television City. Even so, the network agreed to give B&C a tour guided by Poltrack in April, as pilot season reached its apex. The visit came with caveats about what could be quoted or described in detail, but it was about as deep into the process as any reporter has gotten.
Poltrack, smartly attired in his customary dark suit and a cornflower-blue tie, affected a patrician air as he surveyed the scene at Television City. “We’re here because it’s the crossroads of America,” he explained, gesturing to passersby. “They’re from all around the country, all different groups. It gives us a great pool from which to draw.”
Participants receive token gifts, like a Starbucks gift card, though some net up to $100 if they are selected for extensive focus group sessions. Given the popularity of scripted TV, many would do it for free—and in fact, the facility is widely marketed as a free destination by Vegas tourism officials and the MGM Grand itself.
“We really promote the concept of pilots,” Poltrack said. “We tell people, ‘You’re going to see shows that are being considered and your vote is going to count.’ That gets them really excited.”
The focus group rooms could be call centers in Rapid City, except for the little dials that look like disembodied combination locks—viewers seated at the stations, which run on souped-up broadband connections, watch programs on the big screen on one wall, twisting the dials based on their sentiment and hitting the “tune-out” button if it comes to that. The rooms have one-way mirrors, enabling producers, CMOs or whoever happens to be “the client” to watch audiences react in real time. Videoconferencing technology also keeps New York and L.A. connected. Staff both onsite and at CBS HQ sift through the numbers, forming focus groups based on attitudes toward a pilot. Too many positive or negative responses shouldn’t be clustered together. The goal is to have 225 cumulative viewers evaluate a pilot at each stage.
In the late 2000s, with capacity becoming an issue, Poltrack oversaw a Television City expansion that saw a new suite of smaller rooms and offices built, including an earth-toned office where he spends a few days each month. His favorite room in the place, though, is down the hall. Decorated like a living room, it has a couch, easy chair, entertainment center and a computer desk on a far wall. Tiny cameras are embedded everywhere—that long bar under the TV set that looks like a sub-woofer? It’s an eye-tracking device to see where on the screen the viewer looks, invaluable insight for program and marketing research alike.
Seeing research subjects in an environment like their own living room gave Poltrack the idea to test how new devices affect the viewing experience—before smart watches or virtual-reality gear were available for purchase, they were scrutinized in Television City. The well-trained staff and well-established facility has also drawn clients from a range of cable networks who rent the site out for their own tests.
Cracks in the Foundation
Here is the thing about program research, however: It’s far from universally accepted that the current model for it works.
“I think it’s very broken, on a lot of levels,” said Kevin Goetz, CEO of Screen Engine, a previously film-centric research firm which acquired longtime TV power ASI in 2014. “The ridiculously short window in which TV studios are required to perfect their pilots is often not conducive to producing the best results.” In the film world, Goetz said, “You could have three to five screenings during a two-month period and truly evolve and move the numbers upwards. You could edit, or reshoot. In television, it’s a very, very unfair process. If you don’t get it right immediately, you will most likely not get a second chance.”
Julie Piepenkotter, executive VP of research at FX Networks, who before joining the network spent 20 years at the Disney-ABC TV Group, believes testing “is alive and well” and more crucial than ever because of the crowded marketplace. But there is intense pressure to deliver approval ratings in the “top two boxes” (very good and excellent), something that can be a challenge given the increased sophistication of focus group members. “‘Good’ used to get you two or three seasons,” she said. “Now, it doesn’t even get you a season.”
Henry Shapiro, CEO of another major research player, MarketCast, said his company has deliberately focused on TV research that is distinct from the pilot process, focusing instead on positioning/marketing studies or returning-show research.
“Our work is embraced more at the outset of a new property when it comes to guiding positioning and marketing rather than storytelling,” he said. “Once a show has secured a second season, the opportunity to look more closely at story and character has a better chance of being embraced by the creative community.”
One showrunner, who has created and run shows on CBS and Fox, did not want to go on the record about research. But he spoke for many creatives in saying, “Seeing something you’ve poured your creative soul into for a year reduced to 20 blue and red triangles on a graph isn’t what you dream about as a kid just wanting to get into showbiz.” At the same time, those little triangles can prove crucial wedges internally at a network to getting a show picked up and added to the fall schedule. “As much as showrunners bitch about testing,” this showrunner said, “when something tests well, we’re the first to say, ‘But it tested great!’”
Defusing Bombs
The showrunner also made one de rigueur reference to a lukewarm test of Seinfeld, which Jerry Seinfeld is said to have framed and put in a bathroom in his home.
Alan Wurtzel, president of research and media development at NBC, waves off the story as an “urban myth,” arguing, “What was tested had almost no relation to the show that wound up on the air.”
Wurtzel, who along with Poltrack is one of the mandarins of TV research, believes in the viability of the process—so much so that he has spearheaded the development of a new facility for NBC within its Universal Studios theme park in Orlando, Fla. It is under construction, with a tentative opening set for July.
Despite his bullishness on research, Wurtzel is a realist. “Pilot testing can tell you what’s not going to work—when you get below a certain score, it just won’t,” he said. “What it doesn’t do is predict a hit because you never know how good the second and third episodes will be, and you don’t know when it’s going to get scheduled.” One of his favorite occurrences is when a show becomes an upside surprise based on decisions made about the pilot based on testing—Scrubs, for example, was recut to emphasize certain supporting characters who proved popular in tests.
Compared with a generation ago, showrunners and producers are more open to what research can tell them—a shift that may also correlate with an attention being paid to fan engagement on social media. “There’s been more of an appreciation than there was 20 years ago. At that time, they wanted the research sent to them, but in a brown paper bag,” Wurtzel said.
“If you do a play, you stand in the back of the theatre and listen to the audience. When you work in a medium like television, there’s absolutely no way to know the audience’s response in real time. So research is one tool in helping understand the reaction.” The challenge comes with the interpretation, he adds. “People get so close to a program that they assume people understand the narrative.”
Agrees FX’s Piepenkotter, “It’s important to step back from the baby you’ve just birthed and get a civilian’s opinion.” While FX is in a different business than the broadcasters, they face similar crossroads with pilots. When Justified (then called Lawman) was being tested, audiences rebelled when Boyd Crowder, a character played by Walton Goggins, was shot and killed. He lived and thrived; so did the show.
