The Shadow Side of AI Part 4: Our minds in their hands

The tech bros are keen to be seen as world saviors. But should we trust them?

Freaky fingers by Stable Diffusion

“This new medium is reprogramming us, yet we fail to grasp the radical paradigm shift that is underway. We are hobbling along after the very medium that, below our threshold of conscious decision, is definitively changing the ways that we act, perceive, feel, think, and live together. We are enraptured by the digital medium yet unable to gauge the consequences of our frenzy fully. The crisis we are now experiencing follows from our blindness and stupefaction.”

Byung-Chul Han IN THE SWARM

In a previous post I wrote about the predisposition to project human-like qualities onto the technologies we interact with. By fetishizing machines and algorithms in such ways, we conceal the human labor and intentions behind such tools, and instead participate in a thaumaturgical display. A game of make believe. Similarly when we watch movies, play computer games, or consume other digital media, we are happy to suspend disbelief for an hour or two and buy into the spectacle that plays out before us. To be entertained. To relax. To escape the messiness of the real world.

If we look up from our devices for a moment or two, we are often confronted by a world in conflict. A world that perhaps doesn’t understand us. A world of unfathomable complexity that we don’t fully understand. Facing such precarity, the impulse to take refuge in virtual worlds is quite understandable. We can see this flight from the real world playing out all around us. The relief provided by an exodus into digital realms can be just as addictive as any illegal substance we may encounter in the world of physicality. Perhaps even more so. One or two hours can quickly stretch to six or seven. Glued to the screen. Lost to the great digital void.

On the sidelines we find evangelists, like Mark Zuckerberg and other tech titans, enthusiastically waving us on, edging us deeper into their virtual spaces with a religious zeal that would not seem out of place in the great revivals of the past. They preach of course to the choir, a devoted audience already fully primed by machine learning algorithms to keep them clicking, scrolling, and compliant. Algorithms that have captured the minds of millions, if not billions of people with mathematical precision. The logic driving such algorithms follows a familiar formula: increase sales to maximize profit. The products typically on sale are not physical wares like paperclips, but advertising space. As Zuckerberg famously confessed to a perplexed Congress, “We run ads”.

The human elements in this process have always been instrumental. In an initiatory moment of transcendence, as humans enter a virtual space, they disappear and emerge transformed. Commodified. No longer simply humans, they are born again as users, baptized by their tech overlords, in a rather ironic moment of Truth. Like Pavlov’s rats, users provide key behavioral data that the algorithms model to maintain attention and maximize engagement. In an endless feedback loop the algorithms work relentlessly. Recording, analyzing, profiling, and feeding users with continuous streams of content. The more they hold a user’s attention, the more ads they can push out. Ka-ching! The logic is simple. Elegant even. Capitalist. Some might feel patriotic, stand up and start waving flags.

The machine learning algorithms do not hypothesize the genus of content that would best hold a user’s attention and start pushing such content out to as many users as possible. Instead they work empirically. By aggregating the user behavior of thousands or even millions of users out there in the wild, the algorithms slowly build up a perfect knowledge base of what actually does hold their attention. They track hundreds of parameters to discern the sticky content. Drawing from this knowledge base each user is served up a bespoke feed of such sticky content to keep them engaged. Of course the most egregious content is moderated. Censored. Or to use the latest nomenclature, it is aligned to whatever ideology is currently in vogue. Woke or conservative. Take your pick. Window dressing the content in this way is always a human, manual intervention.

For most of human history, in the real world, we have been occupied with the daily drudgery of survival. Kept busy in a recurrent loop of mundane tasks. Looking through an evolutionary lens, our attention is piqued when we encounter key threats or opportunities. Events that break the monotony or boredom of everyday life. Events that cause us to look up and pay attention. Such as when we are under attack, or face some kind of existential danger. Dangers like those gripping many in the Middle East right now. Our senses awaken and we become alert. A fight takes place and a crowd gathers. Just like in the school yard, even those not directly involved join in and take sides. At the very least there is an impulse to be seen to be on the right side. Always our side, of course. Doing so, we other a whole group of people, we dehumanize them, and a drama unfolds.

To give another evolutionary example, we might pay attention when we encounter the opportunity to reproduce or find a mate. Like Pavlov’s rats, we salivate whether we want to or not. We are no different from many other animals. Whatever it might be that piques our attention, well, that’s what this first wave of machine learning algorithms gave us. In bucket loads. In an infinite feed of hyper-stimuli, we became the hyper-stimulated. Boredom was defeated. The real world pales in comparison. There are no value judgements at play here. The computer code is amoral by design. How our attention is held is quite irrelevant. Be it the good, the bad or the downright ugly. The algorithms were simply programmed to be profitable. To increase revenues. To grow the market. Modern wars are fought to maintain that freedom.

Of course, the tech bros didn’t know these algorithms would be bad for us. Not at first. I'm not naïve enough to assume they possessed any foresight beyond the profit motive and hubris driving them in the early days. As externalities materialized they simply followed Big Tobacco’s playbook. When internal reports started to uncover uncomfortable truths, they were suppressed. But leaks happened. Shortened attention spans. Children addicted. Children sexualized. Body dismorphia. Social isolation. Ubiquitous loneliness. A mental health epidemic. Online harassment. Fake news. Polarization. Doomscrolling. Democracy under threat. The list goes on.

But that is just a reflection of the real world, the tech bros have countered. Of course they are right. But that argument obfuscates the huge asymmetries of power at play here. Power that their algorithms possess to direct and shape the minds of billions of real people in the real world. The minds of our friends and families. Our daughters are particularly vulnerable it seems. In the name of freedom we have entrusted this power to narrow market forces. Aided and abetted by tech money that lubricates the corridors of power to ensure the show goes on.

Oh dear- by Stable Diffusion

Plugged into their algorithms the tech overlords gain superhuman powers of omnipresence and omniscience. Watching over their subjects, accommodating their whims, they tweak parameters to draw us deeper into their virtual webs. As their new AI algorithms start to roll out, unsatiated, they lewdly look up from their prey, turn to us and proclaim, ‘Trust us. We will  save you!’. Enraptured by the digital medium. Our minds in their hands.