Coffee Space


A/B Testing News

Preview Image

I happened upon a Tim Pool video that highlights a massive problem with news reporting today:

CNN changed the headline of an article after it got backlash, after people called out the comparison as absolutely ridiculous.

The original article headline was: “Joe Rogan’s use of the n-word is another January 6 moment”.

The current article headline is: “Why shrugging off Joe Rogan’s use of the n-word is so dangerous”.

I will go through how it can occur, what the problems are and what should be done about it.

A/B Testing

For quite a while now, news websites have adopted a concept called A/B testing, where different people get different versions of the same article. Based on some feedback mechanism (for example page views), one is used more than the other and eventually replaces all variations.

The purpose of this is to maximize interaction with content and make online articles maximize ‘click-bait’, i.e. more clicks is “better”.

A/B testing is incredibly common with headlines as this article shows. Normally there are at least two competing articles for the same piece of news. This is the first time I have seen the article preview image change, but it also makes sense.

Although most people don’t actually read the article content, I can’t see why they wouldn’t also change the article content - especially the first paragraph. Perhaps they could even be tailored for the chosen headlines. Going further, they may choose to even tailor the content depending on who is viewing it - they are tracking you after all, why not characterize your political ideology and serve you news that reaffirms it?

Perhaps instead of “Joe Rogan’s use of the n-word is another January 6 moment”, Republicans will read “Joe Rogan’s use of the n-word was appropriate”. Why not? They have the capability, they are already performing A/B testing - this is simply the next step.

Of course, there are also manual changes, which is likely what happened over at CNN once they were called out for the ridiculousness of the article headline. They of course then got called out for changing the headline.


So, why is this a problem? Two people can read the article from the same link, and arrive at two completely different conclusions, despite starting from the same ideological standpoint. Worse still, the same person could read the same article, at a different point, and come to a different conclusion.

I have personally dealt with this, with Facebook showing me a different cached preview to that of a different person (I believe this was a Trump-based article). We then argued about we perceived the content of the article differently. I then manually searched for the article in a private browsing session and realised I had been served a different page to them!

How on earth can we expect people to take ‘respectable’ news outlets like CNN seriously when we cannot even have a robust discussion based on the exact same article link? As if the problem wasn’t already hard enough!

The point here is that CNN doesn’t actually care about journalistic integrity, they care more for profit and political leverage. They act in a way to advance their own agenda. These people clearly cannot be trusted to act without oversight.


One immediate problem is that the CNN article does not inform readers it was edited (other than a small time stamp, which does not indicate what was changed or to what extend it was changed). It does not indicate that the headline was changed, or that the preview image was changed, or anything else for that matter.

In the future, all changes should be clearly tracked and marked, with reasons for having made the changes. If you are going to make the claim of having journalistic integrity - once you publish something, it should not be easy to edit it once done.

This is how the original article would look:

Joe Rogan’s use of the n-word is another January 6 moment

Analysis by John Blake, CNN

This is how the changed article would look:

Why shrugging off Joe Rogan’s use of the n-word is so dangerous

Analysis by John Blake, CNN

[View changes]

This is how the changes could be displayed:

Why shrugging off Joe Rogan’s use of the n-word is so dangerous another January 6 moment [1]

Analysis by John Blake, CNN

[1] (Change author: @namehere) Changed headline due to inappropriate use of ‘January 6th’ event.

It’s not entirely clear how multiple changes to the same text could be easily tracked and displayed in a readable format.

In general though, the solution here is to ban A/B testing for ‘trusted’ news media. Their readers did not opt-in to their little experimentation process and should not be subject to the results that can occur as a result. When I send a link to a friend, I expect that the article I read is the same as the article they read. We can’t have meaningful and robust debate otherwise.

This oversight should not come from the government, but an intendant body. The government of the day therefore can put pressure on the independent body, but not the news outlets directly. This degree of separation is important for journalistic integrity. The independent body would then issue fines and receive a subscription from members in order to pay for its own existence. News companies would sign-up under the threat of government regulation otherwise.