Elon Musk open sourced Twitter's recommendation algorithm

A few weeks ago, Twitter's CEO Elon Musk announced that he would open source Twitter's algorithm. Anyone can see and inspect the code that powers Twitter's recommendation system. Twitter users considered it as just another joke Elon was pulling. He posted a similar tweet two weeks back.

He tweeted something similar yesterday.

This time, it wasn't a joke. Elon indeed took Twitter's algorithm and machine learning model open source. In this blog, let's learn more about Twitter's open-source algorithm.

Twitter Algorithm is now open source

Twitter published two repositories on GitHub. The first is Twitter's The Algorithm, and the other is its ML repo. These repositories contain source code for several important modules and their recommendation algorithm.

Twitter published two blogs. The first announces Twitter taking its algorithm open source, whereas the latter explains how it works. Elon Musk hosted a Twitter Space when we presented his views on this bold move.

Twitter Space

Musk said the goal is to bring more transparency and accountability to Twitter's functions and help researchers understand how extremist speech can be countered. He also admitted that the algorithm is overly complex and not fully understood internally and that people will discover many silly things.

How does The Algorithm work?

Twitter wrote a detailed blog discussing its algorithm. The recommendation pipeline consists of three main stages. First, it gathers the best tweets from different recommendation sources. Then, it ranks them with a machine learning model. And finally, it filters out already seen tweets, tweets from blocked users, and tweets that do not follow Twitter's guidelines. The blog then explains each step in detail.

What do we know so far?

In the past, Twitter has been criticized for being opaque, biased, and restrictive. Many users and researchers have questioned how Twitter's algorithm decides which tweets to show on the For You timeline, which is personalized for each user based on their interests, preferences, and behavior. Some have also accused Twitter of censoring or amplifying certain voices or topics without disclosing the criteria or logic behind its decisions.

But we have the answers to most of the questions now. The Algorithm answered most of the questions users were having. There are several points to know based on The Algorithm.

1. Your following-to-follower ratio matters.
2. Twitter Blue subscribers do get a boost in the algorithm.
3. The algorithm identifies users in four groups: power users, democrats, republicans, and Elon Musk.
4. Negative engagement with users' tweets or profiles reduces their reputation score. If a user gets blocked, muted, gets reported for abuse or spam, it's considered a negative engagement.
5. Tweets with images & videos get a 2x boost. But things might change in the future.

There are plenty of things to know about. We will surely take a deeper dive into the algorithm soon.

User privacy

The company explains that it aimed for transparency. To achieve this, they excluded any code that would compromise user safety and privacy or the ability to protect our platform from bad actors, including undermining our efforts at combating child sexual exploitation and manipulation.

They decided not to release training data or model weights associated with the algorithm. However, they might do this in the upcoming future. Furthermore, it does not include the code that powers ad recommendations.

The announcement blog states it's the first step towards being transparent, and the company plans to take more of its code open source in the future. At the same time, it takes care that Twitter user does not possess a significant risk from this decision.

Closing notes

Twitter is still one of the largest social media platforms, with 368 million daily active users. The platform serves over 150 billion tweets every day. The blog concludes by stating, "Ensuring that we're delivering the best content possible to our users is both a challenging and an exciting problem".

The company invites the developer community to contribute to both repositories. It also started a bug bounty program through HackerOne for security concerns. 

It would be fascinating to see how this move turns out to be!