The Dictatorship of the Predictariat

Effective Altruism has authoritarianism under its skin. How Toby Ord’s “The Precipice” presaged the FTX Fraud. (The Kwisatz Haderach Who Failed, Part 2)

Two quick notes: I’ve converted the post Sam Bankman-Fried is the New Face of Evil to free, so if you’re not yet a premium subscriber, enjoy. I’ll be opening up a few more posts like this in the future, and highlighting them in Sunday posts. Keep an eye out.

Also, we have a new podcast episode with my old friend Dexter Thomas, formerly of Vice News Tonight. We talked about the misguided push for a ban of TikTok; our shared transition from academia to journalism; and why you have Dexter’s permission to quit your grad program RIGHT NOW. Finally, in a first for the Dark Markets Podcast, we have a bonus segment exclusively for Premium subscribers, in which Dexter tells us about the last days of Vice News. The main episode should still be easily accessible to everyone, so please let me know if there’s an issue.

Now, on to part 2 of a key chapter in my book on Sam Bankman-Fried, in which I draw the connections between the implicit logic that drove his fraud, and the ideology he ascribed to: Effective Altruism. You can read part 1 here: The Kwisatz Haderach Who Failed: Why Sam Bankman-Fried Did What He Did.

The Dictatorship of the Predictariat

Sam Bankman-Fried has never directly admitted that he used customer deposits to fund FTX and Alameda investments and expenses, nor has he offered a coherent alternative justification of why the money went where it so clearly did. His apologists have followed along, content to argue that he deployed embezzled funds wisely. As I argued in last week’s segment, the unspoken justification for these criminal actions was rooted in Bankman-Fried’s utilitarian, determinist, and Effective Altruist constellation of worldviews.

Bankman-Fried believed, in sum, that the world is a predictable place; that he had unique powers for predicting it; and that, therefore, using customers’ money to grow FTX was actually to their benefit – even if it was technically illegal.

This viewpoint, attested to in so many words by the likes of Caroline Ellison and Judge Lewis Kaplan, was derived in large part from the attitude towards the future underpinning the Effective Altruist movement, which overlapped with the “expected value” calculations of his Jane Street trading days. As part of my project of getting beneath the bizarre surface of Sam Bankman-Fried, I recently read The Precipice: Existential Risk and the Future of Humanity by Toby Ord. With Will MacAskill, Ord is considered one of the co-founders of Effective Altruism. (All page references below are to the 2020 hardcover edition).

On the surface, The Precipice is a largely anodyne warning about asteroid strikes and climate change. But just beneath the surface of his proposals for addressing those risks, Ord adopts the same toxic premises that motivated Sam Bankman-Fried’s fraud: that there are some people who are smarter than all the others, and that this elite should be allowed to make decisions on behalf of the rest of us. Managing X-risk, Ord writes, “will ultimately require new institutions, filled with people of keen intellect and sound judgment, endowed with a substantial budget and real influence over policy” at a global scale. (196)

The Precipice calls for a single world government, dominated by committees of experts, with superseding power to “punish the deliberate or reckless imposition of unnecessary extinction risk.” (203). Ord doesn’t really address the governance and information problem at the heart of his technocratic solutionism: who gets to choose who sits on these empowered committees of experts?

But you can infer that he imagines people more or less like himself in such roles.

This boils down to a more complex version of the same fearmongering used to justify authoritarian impositions like encryption bans or the U.S. Patriot Act – Ord just replaces contemporary fears with longer-run “extinction risk.” In fact, it’s interesting to infer that the long-term nature of the risks Ord is focused on, also imply long-term powers for those tasked with fending them off.

I shouldn’t go much farther without saying that I don’t mean any of this to impugn the motives or personal character of any one person. I think many practicing and committed Effective Altruists have arrived at their framework out of very high intentions, and as I’ve learned about Sam, I’ve also learned that many of them act with enough care and thoughtfulness that, at the very least, they’re not regularly doing catastrophic harm.

Rather, I think that these good intentions, and likely Ord’s as well, have been shaped by the larger forces of our moment. They are all too representative of the thinking and practice of a much broader class – or perhaps caste – of managerial elites that has come to dominate our tech-solutionist, engineering-centric, science-driven culture. There is less and less space for thinking about human beings, and human society, as anything other than a problem to be solved, usually through sheer technological innovation. Effective Altruism is a technocratic and technological approach to fixing humanity.

The Insanity of Rationalism

The problem with treating the universe like an abacus that humanity can read is that … well, is that you can’t actually predict the future, no matter how many decimal places you use. And deciding that you can seems bad all around - including for the predictor.

Bankman-Fried has seemed truly unable to process that he failed to accurately predict the future - even in the very, very short term. Leaving aside the basic stupidity of embezzling demand deposits by putting them into illiquid venture investments, he hasn’t accepted that his bets, both on himself and others, produced far less “expected value” than he expected. In his final statements at his sentencing hearing, SBF continued to cling to the sad fantasy that all the money was still there, that customers could still be made whole if only the bankruptcy team would listen to him.

It was a genuinely pathetic moment of abject escapism, indulged in by a man who blew up his own life and thousands of others. Judge Kaplan rightly dismissed the claims that there were no losses as “misleading, logically flawed, and speculative.”

In this and other ways, Bankman-Fried seems autistically fixated on a version of events that *should* have happened, and unable to even accept what actually *did* happen. Both impulses come quite directly from his mother, Barbara Fried, who spent her academic career championing a view of human beings, and the broader world, as entirely determined and mechanistic – that is, implicitly, entirely predictable. She also, as shown in Michael Lewis’ book and later statements, has long had a delusional view of her own son’s brilliance and unimpeachable morality – views that Sam himself seems to have happily adopted.

The same mechanistic view of the world is inherent in the Effective Altruist movement that Sam was later drawn into by Will MacAskill. MacAskill made himself point man for managing SBF, who became a huge source of funding for EA – so much so that, well before the FTX crisis, EAs were worried Sam had too much influence.

Epistemic Hubris: On the Moral Equivalency of Present and Future

But Toby Ord is often described as the EA movement’s “cofounder.” It’s notable that The Precipice (2020), Ord’s only major book so far, is focused not on the ethics of charity or how to actually make your donations do the most good – the “malaria nets” narrative that helped kick off the movement back around 2015. Instead, Ord’s book is almost entirely about long-run “existential risk.” The same is largely true of MacAskill’s most well-known book, What We Owe the Future, which is oriented not towards the present, but, well ... the future.

There is another story here about how the focus of effective altruism as a movement shifted so dramatically in the few short years between the start of the movement, and when Ord and MacAskill published their books. But MacAskill was promoting the “earn to give” angle of EA philosophy at least as far back as 2016, when he encountered Bankman-Fried.

And “earn to give” shares the fundamentally flawed assumptions of Ord and MacAskill’s long-termist work, insofar as it functionally amounts to saying that people able to earn the most money in a society built around a market economy should control efforts to fix that society. Certainly they would say these “earners” should be looking at objective metrics and other expert guidance when deciding to make their judgments, but just as with calling for a unified world government to address extinction risk, this seems naive to a lot of basic human behavioral bias, to say nothing of the more overt narcissism rampant among the tech and finance types who populate the movement – such as, for instance, Sam Bankman-Fried.

The fundamental problem with Ord and MacAskill’s thinking, which is also the main problem with Sam Bankman-Fried’s thinking, is epistemic hubris. That is, these are people who don’t think very hard or subtly about how we know things, and a lot of the time seem to assume that certain knowledge will emerge. Underlying all of Ord and MacAskill’s arguments is the idea that humans could accurately predict the future consequences of their actions, if only they thought a little harder about it.

This ultimately sneaks in the technocratic authoritarianism at the heart of EA, since what such claims boil down to is that some people can predict the future – and that those people should be in charge. This was the logic Sam Bankman-Fried was following when he chose to use his customers’ money for his own speculation – he was the one who knew, and therefore the one who should have the right to decide.