杏吧原创

The flawed way Netflix measures views leaves TV stuck in the past

Netflix claims its audience "views" reflect the desires of the masses, but this unreliable data means some bad shows get green-lit, while potentially good stuff doesn't, says Annalee Newitz

The Netflix Inc. television series 'Squid Game' on a laptop computer arranged in the Brooklyn Borough of New York, U.S., on Saturday, Oct. 16, 2021. Netflix Inc. is scheduled to release earnings figures on October 19. Photographer: Gabby Jones/Bloomberg via Getty Images

AUSTRALIAN comedian Hannah Gadsby recently came up with a : an 鈥渁moral algorithm cult鈥. She would know. It was Gadsby鈥檚 Netflix special Nanette that launched her to international fame. She chose to describe the streaming giant in those terms after an anonymous leak revealed details about the company鈥檚 system for rewarding creators based on algorithmic predictions of audience numbers.

When Netflix makes decisions about what shows to buy or cancel, their executives often cite 鈥渢he algorithm鈥, an internal tool that crunches data about viewership 鈥 and is implicitly more objective than a person. Spoiler alert: the system isn鈥檛 objective, and its algorithms are making predictions based on skewed data. This isn鈥檛 just about Netflix though 鈥 it is one of many streaming video companies whose audience metrics are increasingly questionable.

The current controversy starts after the release of Dave Chappelle鈥檚 latest comedy special The Closer. Instead of his usual social satire, Chappelle delivers what sounds like cable news commentary about how gay and transgender people are ruining everything. Not only did many viewers find it needlessly offensive, it simply wasn鈥檛 funny.

Amid demands that Netflix remove the special, a leak revealed that Chappelle鈥檚 previous special, Sticks & Stones, had, at $23.6 million, cost more than the current hit drama Squid Game. At the same time, the comedy special scored much lower than this series on a key audience metric called 鈥渁djusted view share鈥 鈥 a measure of return on investment, based on whether audience engagement justified the cost.

Netflix executives say they use algorithms and data to decide which shows to commission and how much to pay for them. But these latest revelations make it look like Netflix paid even more for The Closer, even though it knew that Chappelle鈥檚 previous special had underperformed. What kind of ass-backwards algorithm would recommend paying more for a show whose predecessor hadn鈥檛 lived up to expectations?

This question has opened a larger debate about how popularity is measured in the secretive world of streaming video. Netflix is famous for making barely any of its viewership numbers public, despite prominently displaying a 鈥渢op 10鈥 shows on a carousel on its front page. It isn鈥檛 alone. It is impossible to get solid viewership numbers on Amazon-produced shows or other streaming services like Disney+ and YouTube.

鈥淲hen companies do reveal how they come up with audience metrics, the reasoning often sounds bonkers鈥

Even when companies do reveal how they come up with audience metrics, their reasoning often sounds bonkers. Netflix now releases viewership data for many popular shows, but includes users in that data even if they have only watched the first 2 minutes of a single episode.

I watched the first 2 minutes of Squid Game and I didn鈥檛 even make it through the opening credits, yet Netflix can now add my 鈥渧iew鈥 to its audience data. Meanwhile, Facebook claims that a video has been viewed if a person watches it for 3 seconds or more. YouTube is a bit more generous, calling it a 鈥渧iew鈥 if someone watches for 30 seconds after deliberately pushing play.

Though all these firms have data on how many people watch videos or series through to the end, they choose not to publicly measure success in those terms. And you can see why: in 2020, Netflix acknowledged that its 2-minute metric is 鈥渁bout 35 percent higher on average than the prior metric鈥, which was based on how many people 鈥渨atched 70 percent of a single episode of a series鈥.

Netflix has just announced it will begin reporting viewership in terms of the total number of hours watched, too. Still, it is deeply problematic that executives apparently use flawed audience data to help determine what gets green-lit for production and how much money it receives. The algorithm is a shield that executives can use to claim objectivity when in fact they are making decisions exactly like gatekeepers did in the bad old days of the 20th century. They are selecting shows based on their own personal preferences.

Netflix execs are free to make decisions based on whatever they want, of course. The trouble is that they are using their algorithm to make it appear that it isn鈥檛 entirely their decision. They claim to be reflecting the desires of the masses as expressed in audience views. That makes it somehow our fault that bad shows get green-lit, while potentially good stuff never sees the light of day.

In reality, certain people in the US entertainment industry are cutting deals and promoting the things that they liked 20 years ago when Chappelle was fresh. They may have sparkly new algorithms, but they are still making decisions like it is 1982.

Annalee鈥檚 week

What I鈥檓 reading

by David Graeber and David Wengrow, a terrific, evidence-based history of humankind that will turn everything you know on its head.

What I鈥檓 watching

Three minutes of every show on Netflix, just to mess with its algorithm.

What I鈥檓 working on

Trying to reduce my water consumption by 15 per cent. The California governor has declared a state-wide drought emergency.

  • This column appears monthly. Up next week: James Wong