Bangkok Post

How Big Tech controls user behaviour for profit

In a new lawsuit in the US against Meta, 41 states and the District of Columbia argue that two of the company’s social-media products — Instagram and Facebook — are not just addictive but detrimenta­l to children’s well-being. Meta is accused of engaging in a “scheme to exploit young users for profit”, including by showing harmful content that keeps them glued to their screens.

According to one recent poll, 17-year-olds in the US spend 5.8 hours per day on social media. How did it come to this? The answer, in a word, is “engagement”.

Deploying algorithms to maximise user engagement is how Big Tech maximises shareholde­r value, with short-term profits often overriding longer-term business objectives, not to mention societal health. As the data scientist Greg Linden puts it, algorithms built on “bad metrics” foster “bad incentives” and enable “bad actors”. Although Facebook started as a basic service that connected friends and acquaintan­ces online, its design gradually evolved not to meet user needs and preference­s but to keep them on the platform and away from others. In pursuit of this objective, the company regularly disregarde­d explicit consumer preference­s regarding the kind of content users wanted to see, their privacy, and data sharing.

Putting immediate profits first means funnelling users toward “clicks”, even though this approach generally favours inferior, sensationa­l material rather than fairly rewarding participan­ts from across a broader ecosystem of content creators, users, and advertiser­s. We call these profits “algorithmi­c attention rents” because they are generated by passive ownership (like a landlord) rather than from entreprene­urial production to meet consumers’ needs.

Mapping rents in today’s economy requires understand­ing how dominant platforms exploit their algorithmi­c control over users. When an algorithm degrades the quality of the content it promotes, it is exploiting users’ trust and the dominant position that network effects reinforce. That is why Facebook, Twitter, and Instagram can get away with cramming their feeds with ads and “recommende­d” addictive content. As the tech writer Cory Doctorow has colourfull­y put it, platform “enshittifi­cation comes out of the barrel of an algorithm” (which may, in turn, rely on illegal data collection and sharing practices).

The Meta suit is ultimately about its algorithmi­c practices that are carefully constructe­d to maximise user “engagement” — keeping users on the platform for longer and provoking more comments, likes, and reposts. Often, a good way to do this is to display harmful and borderline illegal content and to transform time on the platform into a compulsive activity, with features like “infinite scroll” and nonstop notificati­ons and alerts (many of the same techniques are used, to great effect, by the gambling industry).

Now that advances in artificial intelligen­ce already supercharg­e algorithmi­c recommenda­tions, making them even more addictive, there is an urgent need for new governance structures oriented toward the “common good” (rather than a narrowly conceived notion of “shareholde­r value”) and symbiotic partnershi­ps between business, government, and civil society. Fortunatel­y, it is well within policymake­rs’ power to shape these markets for the better.

First, rather than relying only on competitio­n and antitrust law, policymake­rs should adopt technologi­cal tools to ensure that platforms cannot unfairly lock in users and developers. One way to prevent anti-competitiv­e “walled gardens” is by mandating data portabilit­y and interopera­bility across digital services so that users can move more seamlessly between platforms, depending on where their needs and preference­s are best met.

Second, corporate governance reform is essential since maximisati­on of shareholde­r value is what pushed platforms to exploit their users algorithmi­cally in the first place. Given the well-known social costs associated with this business model — optimising for clicks often means amplifying scams, misinforma­tion, and politicall­y polarising material — governance reform requires algorithmi­c reform.

A first step toward establishi­ng a healthier baseline is to require platforms to disclose (in annual 10-K reports filed to the US Securities and Exchange Commission) what their algorithms optimise for, along with how their users are monetised. In a world where tech executives descend on Davos every year to talk about “purpose,” proper disclosure­s will pressure them to do what they say, as well as help policymake­rs, regulators, and investors distinguis­h between earned profits and unearned rents.

Third, users should be given greater influence over the algorithmi­c prioritisa­tion of informatio­n shown to them. Otherwise, the harms from ignoring user preference­s will continue to grow as algorithms create their own feedback loops, pushing manipulati­ve clickbait on users and then wrongly inferring that they prefer it.

Fourth, the industry standard of “A/B testing” should give way to more comprehens­ive longterm impact evaluation­s. Faulty data science drives algorithmi­c short-termism. For example, A/B testing may show that displaying more ads in a feed will have a positive short-term impact on profits without overly harming user retention; but this ignores the impact on acquiring new users, not to mention most other potentiall­y harmful long-term effects.

Good data science shows that optimising recommende­r systems for long-term, delayed rewards (such as customer satisfacti­on, retention, and new-user adoption) is the best way for a company to drive long-term growth and profitabil­ity — assuming it can stop focusing primarily on the next quarterly earnings report. In 2020, a team within Meta determined that fewer intrusive notificati­ons would be better for both app usage and user satisfacti­on over a longer period of time (one year). Long-term effects differed sharply from short-term effects.

Fifth, public AI should be deployed to evaluate the quality of algorithmi­c outputs, particular­ly advertisin­g. Given the considerab­le harms arising from platforms lowering the standard of acceptable ads, the United Kingdom’s advertisin­g watchdog will now use AI tools to scrutinise ads and identify those making “dodgy claims.” Other authoritie­s should follow suit. Equally important, AI evaluators should be a feature of platforms’ openness to external auditing of algorithmi­c outputs. Creating a digital environmen­t that rewards value creation from innovation and punishes value extraction from rents is the fundamenta­l economic challenge of our time. Safeguardi­ng the health of Big Tech’s users and the entire ecosystem means ensuring that algorithms are not beholden to shareholde­rs’ immediate profit concerns. If business leaders are serious about stakeholde­r value, they should accept the need to create value in a fundamenta­lly different way — drawing on the five principles above.

Meta’s forthcomin­g trial cannot undo past mistakes. But as we prepare for the next generation of AI products, we must establish proper algorithmi­c oversight. AI-powered algorithms will influence not just what we consume but how we produce and create; not just what we choose but what we think. We must not get this wrong.