Newsletter

Pugpig Media Bulletin: Peak newsletter? Peak push? How to use data and A/B testing to get the right audience development balance

This week we look at whether publishers have gone overboard with newsletters and push notifications and how to use A/B testing to drive your success.

26th October 2022

A/B testing is driving success for publishers, and it can make sure that you're getting the most from audience development strategies. Photo by Jason Deny on Unsplash

In the Pugpig weekly media bulletin, Pugpig’s consulting services director Kevin Anderson distills some of the best strategies and tactics that are driving growth in audiences, revenue and innovation at media businesses around the world.

Peak newsletter? Or peak paid newsletter?

Two articles questioned popular audience engagement tools last week, one on newsletters from the New York Times and one on push notifications from the Press Gazette. On the topic of newsletters, the New York Times has the perfect media trend story, picking a couple of examples of major companies pulling back from the format, which they put down in part to a reversion to the norm after the pandemic, during which audiences had more time on their hands to read newsletters.

Sign up to get the Media Bulletin in your inbox.

The NY Times’ highlighted Facebook’s Meta ending its Bulletin newsletter service and also The Atlantic rethinking their six-figure compensation packages for newsletter writers who converted 14,5000 readers to subscriptions. However, Facebook has been pulling back from news for several years, and as for The Atlantic, their goals proved too aggressive.

With all of the time, attention and effort being put into newsletters, it is unsurprising that there are questions about whether our overflowing inboxes can take the strain of yet another media newsletter. But a fair bit of the scepticism in the analysis seems to relate to newsletters as a standalone subscription product rather than as part of a broader subscription strategy.

For many publishers, newsletters remain an incredibly effective way to engage readers and convert them to subscribers. The Local Media Association in the US highlighted a few newsletter success stories from an event that they held in August. For example, a local news site in Arizona had an eight-email evergreen campaign designed to provide information to newcomers to the city. The campaign built a relationship with new residents, and the site generated revenue with sponsorships and display ads. They also were able to convert some of the newsletter subscribers to paying members.

And as we discussed last week, the Toronto Star has found success with simple, automated newsletters that allow readers to follow topics or writers, providing the newspaper with valuable first-party data. It shows that for publishers, newsletters pay for themselves not with direct subscription revenue but rather by converting audiences from anonymous to known and being part of a larger subscription funnel.

Are publishers pushing too much with push notifications?

At Pugpig, we have seen how publishers using our Bolt app platform are effectively using push notifications to engage users, and the data proves that more engaged users are more likely to rate or rank an app. This helps increase in visibility of their apps on app stores.

But competition is fierce for users’ attention on the lock screen, and the Press Gazette put that competition into stark relief with data from push notification service Airship and their own analysis. Press Gazette found that between 16 September and 1 October that 27 international, national and local publishers sent 2,300 alerts, and Airship found that media companies sent 2 bn additional push notifications in their coverage of the Queen’s death.

While push notifications drive only a single-digit percentage of traffic, the real benefit is found in keeping a connection with audiences, which drives publishers’ subscription strategies. The Press Gazette quoted Patrick Mareuil, Airship’s managing director for EMEA, as saying that publishers who use push have six times the retention rate after 90 days than those who don’t, and app users who opt-in for push notifications generate five times the advertising revenue.

Publishers interviewed by the Press Gazette found that less is more to a certain extent, and publishers can take advantage of audience segmentation to make sure that users are getting notifications that are most relevant to them. Airship is one of the push notification services that is integrated with Pugpig’s Bolt app platform, and it allows sophisticated segmentation including whether a person has opened a push on that topic before. The key as with any digital strategy is data, and whether using Google’s Firebase or Airship, publishers have access to a wealth of data including being able to run A/B tests to find the topics and treatments that engage users the most.

The power and practice of A/B testing

Speaking of A/B testing, Piano and Digiday have released a report looking at how media companies are using A/B testing to iterate their products and drive success, and they found that this testing is “one of the biggest drivers of publisher growth”. As the authors of the report point out, ongoing refinement of your digital products is important to keep pace with the changes in consumer behaviour and market conditions. Publishers are testing subscription user flows, pricing, assets in their app stores and the overall user experience. As anyone who has made changes to a website or app knows, changes bring both opportunities and risks, but A/B testing allows you to reduce the risk of making a misstep and iteratively improve.

The results for publishers is impressive. Faisal Kalim of What’s New in Publishing highlighted just a couple of results from the report:

Indian tech media platform Inc42, used A/B testing to drive a 100% increase in their memberships and improve retention by 43%.
Spain’s El Confidencial increased their subscriptions by 60% by A/B testing their offers, calls-to-action and how they signposted premium content on their home page.

The report highlights a few things to consider when setting up A/B tests including setting the right KPIs, ones that lead to long-term sustainability. You also need to consider how long to run a test to capture an appropriate sample. And one key point is to keep A/B tests simple to make sure that you are isolating the variable you want to test whether that be a call-to-action, a subscription offering or pricing. Running a test with multiple variables means that you won’t be able to determine what change was responsible for the outcome.

If you want to know more about using A/B tests on your website or app, get in touch with Pugpig Consulting Services at info@pugpig.com.

Industry News

Here are some of the most important headlines about the business of news and publishing as well as strategies and tactics in product management, analytics and audience engagement.