News organizations—including Bloomberg, Reuters, and The Economist—have been using A.I. powered video services to meet growing audience demand for audio-visual material.

But what do audiences think of the results?

Together with Dr. Sally Stares (City, University of London) and Dr. Michael Koliska (Georgetown University), we evaluated reactions to human-made, highly-automated, and partly-automated videos that covered a variety of topics including Christiano Ronaldo, Donald Trump, and the Wimbledon tennis championships. The partly-automated videos were post-edited by humans after the initial automation process.

The results show that there were no significant differences in how much news audiences liked the human-made and partly-automated videos overall. By contrast, highly-automated videos were liked significantly less. In other words, the results show that news video automation is better with human supervision.

One key takeaway of the study is that video automation output may be best when it comes in a hybrid form, meaning a human-machine collaboration. Such hybridity involves more human supervision, ensuring that automated video production maintains quality standards while taking advantage of computers’ strengths, such as speed and scale.

The full article, entitled Audience Evaluations of News Videos Made with Various Levels of Automation: A Population-based Survey Experiment is published on the Social Science Research Network (SSRN).