Newsletter

How publishers are shifting from AI experimentation to execution

We review the tasks that publishers find AI is best-suited for and the change management needed to roll out AI.

19th April 2024

Publishers are finding AI excels at summarising and finding patterns in content, and they are increasingly confident about their ability to leverage AI in their content operations. Image by Scott Graham from Unsplash

Publishers have been feverishly working to understand and experiment with a new generation of artificial intelligence tools and services, even as they grapple with how to engage with AI companies who want to use their content to train their models. More than a year after ChatGPT 3 was released in November 2022, publishers are shifting from experiments to implementation.

It is first important to say that many journalistic applications of AI are “relatively mundane”, as Felix Simon wrote for Tow Center for Digital Journalism at Columbia University’s Graduate School of Journalism. News organisations’ top priority for using AI was to automate back-end tasks such as tagging, transcription and copyediting, according to a survey in Reuters Institute’s annual predictions at the start of 2024. Publishers are deploying AI-powered services in their revenue operations from data and paywall providers to deploy dynamic paywalls and manage content experiences that deliver return on investment while staying clear of thorny editorial operations issues.

Content creation that leverages AI but still is under “human oversight” came in third in the Reuters Institute survey. Notwithstanding, publishers are adopting AI for content creation and content, and their processes in adopting this technology demonstrate best practices for how to adapt to the rapid changes happening across the industry.

Institutionalising AI experimentation

Major publishers are rolling out AI experiments either through existing innovation teams or new AI-focused teams who have been empowered to experiment with the technology and scale up what works. Those editors include Phoebe Connelly at the Washington Post as their first senior editor for AI strategy and innovation and Zach Seward, editorial director for AI initiatives at the New York Times.

Connelly advocates setting up “short-term teams … to work fast and hard at a problem for two years and see what change we could accomplish”. She previously led the Next Gen team, which had a similar remit to define its mission for exploring, testing and delivering products that served new audiences. That team already had built prototypes using generative AI, and now they will expand on that work to “develop tools and processes” leveraging AI in a way that respects the editorial standards of the Post. While she and her team work on AI projects, the Post is working on building AI skills in their newsroom. They just held their first prompt training for the newsroom, Connelly said. Bringing the newsroom with you on this or any innovation journey is important.

Seward gave a great tour recently of the good and the bad of AI in journalism for Harvard’s Nieman Lab. After reviewing some of the recent, well-publicised mistakes publishers have made using AI, he outlined the principles to govern the use of AI he is developing at the New York Times. He said:

“So how might we think about AI journalism that works? Well, it’s got to be vetted, in a rigorous way. The idea should be motivated by what’s best for readers. And, above all, the first principles of journalism must apply: truth and transparency.”

Using AI to Summarise Content

When thinking about any new technology, a key result of experimentation should be to determine what it does well and what it doesn’t. The bad examples that Seward highlights involve some abuses of editorial standards, and they also highlight where AI is simply being used to do something it still isn’t very good at. What is AI good at? It is good at being given a set of documents or an article and summarising the content.

Machine learning has been used to do this for several years, and Seward highlights an example from Quartz, a business news site he helped start. In 2019, Quartz partnered with the International Consortium of Investigative Journalists to go through a large tranche of documents they had received from the tax haven of Mauritius. They had received more than 200,000 documents with some weighing in at hundreds of pages. They built a machine learning model to find similar documents in the tranche, which made analysing them much more efficient.

The text doesn’t need to be mammoth though. INMA has just published a report looking at 31 examples of how publishers are using GenAI, and a couple of examples showed how the technology is being used to summarise content. A common application of AI is now using it to generate summaries of articles either for live coverage or time-sensitive mobile readers. The summaries must be expanded to be read by users, and they see a 20 to 40% open rate plus higher engagement amongst younger audiences.

Schibsted started experimenting with AI-enabled summary tools last year. Its summary tool started as a hackday project at one of its titles, VG. Journalists generate the summary and are presented with several options, which they score. This not only adds the element of human editorial review, but by scoring the results, it helps train the model. Moreover, it follows the “humans in the loop” principle that guides Scibsted’s AI efforts.

More than the technology, the process helped the Scandinavian news group develop better cross-functional collaboration. The product team was concerned that the summaries would lead to audiences spending less time with the articles, but a few design iterations allayed those fears. The VG team joined colleagues at another Schibsted title, Aftonbladet, plus data scientists working on a similar initiative. This collaboration soon extended to the CMS team, and they were able to quickly integrate the tool into the CMS so that it benefitted hundreds of Schibsted journalists.

Using AI for visual pattern matching

Just as AI is excellent at finding patterns in text, it can also analyse volumes of images to support reporting on major issues. Seward highlighted excellent examples of using AI and machine learning to analyse images including an investigation by the Wall Street Journal about lead cabling still present in communities. The reporting team used a machine-learning model to identify the cables in images from Google Street View.

He also praised the work of his colleagues at the New York Times who used an AI tool to identify bomb craters in Gaza to determine the use of heavy bombs by Israel. The AI tool identified some 1600 possible craters, which the team then was able to review and determine that Israel had dropped more than 200 2,000-pound bombs on southern Gaza, an area that was meant to be a safe place for civilians to shelter.

Using AI to organise your content

With AI’s ability to identify patterns in a set of content, it excels in mundane but important digital publishing tasks. 10Up, which Pugpig has worked with as an app partner on several projects, demonstrated its ClassifAI plugin during a WordPress VIP webinar. The plugin adds a range of AI functionality to WordPress. It can suggest headlines, summaries and excerpts, allowing editorial staff to choose a suggested headline from a few it has offered.

The plugin can automatically tag and categorise content and images and add descriptions to images. Consistent categories and image metadata support SEO, and tags can be used to discover content they are interested in, supporting higher levels of engagement.

It currently supports text services from OpenAI and Microsoft as well as image creation using DALL-E. Moreover, when an image is created using DALL-E, it automatically attributes the image to the AI service and includes the prompt used to generate the image in its alt text.

Managing the risks and rewards of AI

Publishers are growing more confident in their use of AI. During a recent INMA webinar, 60% of participants said they were “reasonably confident” in their ability to build consumer-facing AI products. While a quarter felt that working with AI was still too risky, 17% said that they were “very confident” that they had the right standards in place to operate in line with their editorial guidelines.

However, they still had concerns, with more than half, 56%, of respondents worried that AI could provide inaccurate information, and a quarter felt that AI would cost too much to implement.

Ultimately, as Bonnier’s Pia Rehnquist told WAN-IFRA, “AI is a mean, not a goal.”

Our view of AI for publishers is:

Publishers are successfully applying their editorial standards to their AI products and projects in a way that allows them to maintain accuracy while transparently delivering experiences that deliver.
Apps with AI features are enjoying a halo effect from the current interest in the technology, according to data.ai.
AI summaries like what Schibsted developed are ideal for mobile audiences, giving them an easy entry point to get the level of information they want or need at that moment.
Publishers should use AI tools to automate and improve back-end editorial tasks to free up reporters and digital editors to add value in other ways.
Technology is part of the equation, but you also must manage the change. You need to bring all of your organisation with you. As the examples show, the transformation required to adopt AI or meet the challenges of this volatile age requires collaboration across editorial, technical and commercial teams.

Pugpig has experts across the business who can advise you on how to leverage AI or other technologies to achieve your editorial and commercial goals. If you’d like to speak with us, respond to this email or set up a meeting using this calendar link.

Industry News

Here are some of the most important headlines about the business of news and publishing as well as strategies and tactics in product management, analytics and audience engagement.