The Kwisatz Haderach Who Failed: Why Sam Bankman-Fried Did What He Did
Scientism, Effective Altruism, and Frank Herbert’s Dune
Welcome to the Sunday edition of Dark Markets, when I generally share a draft portion of my Sam Bankman-Fried book in progress with my premium supporters - with a partial preview for free subscribers. This chapter is an exploration of the philsophical “theory of the case” of Sam Bankman-Fried’s criminal fraud - an exploration of the complex ideological threads that convinced him that what he did was never actually a crime.
You can find a fairly up-to-date directory of all published draft exceperts in this post.
“He was viewing the cost of getting caught, discounted against the gain of not getting caught. It started at least as early as Jane Street, and it continued to the very end. It’s in his nature. You don’t have to believe me, everyone says this.”
- Judge Lewis Kaplan, Sam Bankman-Fried sentencing hearing, March 28, 2024
Just over a week ago, the story of Sam Bankman-Fried drew to an effective close. In a sentencing hearing at the Southern District of New York, Bankman-Fried was remanded to Federal prison for a term of 25 years.
His defense merely reiterated the framing that Bankman-Fried has clung to, barely wavering, since the November 2022 collapse of the FTX cryptocurrency exchange. His lawyers argued that victims didn’t actually suffer any losses, and praised Bankman-Fried’s generous character. Bankman-Fried also spoke. He blamed the collapse on a “liquidity crisis,” proposed restarting the exchange, and didn’t at any point admit to the crimes he’d been convicted of.
Judge Kaplan righteously disemboweled the whole embarrassing mummer show. Kaplan called the claim that victims would be made whole “misleading, logically flawed, and speculative.” It was also legally irrelevant - Kaplan quipped that “A thief who takes his loot to Las Vegas and successfully bets the stolen money isn’t entitled to a reduction of his sentence.”
But Kaplan went further in that hearing – he eviscerated Sam Bankman-Fried’s character, his worldview, and his honesty. He described Bankman-Fried as essentially a sociopath, willing to risk other people’s safety on his own calculation of the odds. He added sentencing points for perjury and witness tampering. Kaplan verged on directly condemning the “Effective Altruist” movement which had shaped Bankman-Fried’s thinking and eased his path.
Kaplan was articulating the clear but complex answer to a question that remained strangely unanswered over 18 months of furious discourse about Sam Bankman-Fried and the massive FTX fraud: Why did this young man, running what at least could have been a successful business1, choose to commit massive fraud?
The psychological key to that question can be found by digging into Bankman-Fried’s own incessant claims of innocence. Those continued during and after his sentencing: he told the New York Post last week that he “never thought that what I was doing was illegal.”
More broadly, Bankman-Fried seems to have believed that the very concept of something being “illegal,” or even unethical, was irrelevant to him. He seems to have believed, in a substantial sense, that he simply didn’t have to follow normal human rules. This is why he still thinks of himself as innocent.
He thought he was above the conventional law because he saw himself as uniquely brilliant, and because his long-term intentions were good – always a dangerous combination. Added to that was a utilitarian and mechanistic worldview instilled in him by his parents. They, and mother Barbara Fried in particular, seem to have been deeply committed to a mechanistic and scientistic worldview – one that implicitly accepted the complete predictability of the future consequences of present actions.
Foundation vs. Dune
The question of determinism has been widely debated for millennia. But SBF’s determinism is specific to the context of the computing and statistics revolution, which holds out the promise that humans in a deterministic universe might be able to predict the future.
This specific flavor of determinism was the subject of two vital 20th century fiction series: Isaac Asimov’s Foundation and Frank Herbert’s Dune. Asimov’s Foundation is centered around a utilitarian ethos embodied in the hypothetical science of “psychohistory,” which allows the prediction and manipulation of the future path of human society. Herbert wrote Dune at least in part as a rebuttal to Foundation: Paul Atreides is surrounded by would-be psychohistorians seeking to shape the future, but he survives because rather than trying to influence anything, he accepts his terrible fate and, so to speak, rides the worm.
The mechanistic, science-worshipping theories of (fictional) psychohistory are being laundered into a real-world pseudo-philosophy under the label Effective Altruistm – with credit to Edward Ongweso for making this beautiful, powerful connection during my episode of This Machine Kills.
The EAs played a direct role in creating Sam Bankman-Fried, but ultimately Bankman-Fried is a mere symptom of their deeply flawed ideas. In this chapter, I dig into the work of EA cofounder Toby Ord, and reach the same effective conclusion that Frank Herbert did: that claims to a science of human history are fundamentally a mask for the authoritarian will to power.
Sam Bankman-Fried was caught, but many others who think exactly as he does still walk the Earth. As Judge Kaplan made explicit in his statements during Bankman-Fried’s sentencing, they must be shown that their beliefs are dangerous to humanity.
1This is not to say that it already was a successful business, as partisans like Michael Lewis have claimed. FTX’s fraud began very early in its existence, and every metric that could be cited to argue that it was actually “a good business” is tainted by that deep, structural fraud. But it certainly could have been successful, if Bankman-Fried had made entirely different decisions at nearly every point.
Crimes of the Future: On Believing You Are the Chosen One
Sam Bankman-Fried does seem, at certain points, to have been genuinely and literally unaware he was breaking the law.
For instance, he confessed to campaign finance fraud on tape in conversations with Tiffany Fong, and later told her he didn’t realize he was describing a crime. Bankman-Fried was never actually very knowledgeable about how the world works. In this instance, he deferred to his mother’s apparent direction to conduct straw-donor campaign finance fraud. More generally, his parents’ culpability for directly influencing him towards crime can’t be ignored.
But in many other instances, Sam Bankman-Fried clearly knew that what he was doing was illegal. That’s why he lied publicly about the use of customer funds, for instance. Or maybe he realized it when he was bribing Chinese officials, which I’m quite sure he didn’t disclose publicly at the time.
No – he doesn’t think he’s innocent in the sense that you or I might understand that term. What he seems to largely mean when he proclaims his innocence is that while he may have bent some rules, he didn’t do anything truly wrong, because normal human rules simply don’t apply to people like him.
We know Bankman-Fried believed that “rules like don’t lie and don’t steal” were merely performative “shiboleths” in a “dumb game that woke Westerners play.” He was more careful about the image of virtue, including driving cheap cars and framing a Bahamian penthouse as a college style dormitory, than he was about actually doing the right thing in the boring, day-in, day out way that most of us are stuck with.
The thought process behind Bankman-Fried’s specific crimes has become increasingly clear, though he has rarely spelled it out directly – and it’s unclear whether he is self-aware enough to truly understand the gap between his experiences and consensus reality.
Regardless, on some conscious or unconscious level, and whether or not he ever dared even to verbalize the full thought in his internal monologue, SBF believed that borrowing customer funds was okay thanks to the following chain of reasoning:
Leveraging customer funds would help him grow FTX faster (as directly stated to Caroline Ellison), and …
He would easily be able to return everything he borrowed, because …
He had calculated the odds of return for all his expenditures of customer money, so …
His success was inevitable, and therefore …
He had, in substance, committed no crime.
The fact that he was party to a signed contract that explicitly left asset ownership in the hands of customers didn’t prevent this logic from being the most effective, and therefore the most moral – regardless of what the contract said.
More notably, the fact that he actually turned out to be badly wrong didn’t fundamentally alter the moral calculus. Because, I suppose, he should have been right. Get it?
If he had to borrow his customers’ money from behind their backs, he was doing it for their own good. He would inevitably use the capital well, and everyone from whom he had merely temporarily borrowed funds would, in the long term, be better off because he, Sam Bankman-Fried, had been willing to traipse across the antiquated moral bright lines against “lying” and “stealing” – those are just words, anyway.
Judge Kaplan directed significant wrath at various versions of this logic in his statements at SBF’s sentencing hearing.
God’s Perfect Gambler
Bankman-Fried’s rationalized gambling superficially resembles the panicky self-justifications of figures like Elizabeth Holmes or Bernie Madoff, who thought a few small lies could give them the breathing room to fix everything when their initial plans failed.
What makes Bankman-Fried uniquely interesting is that he started lying before anything actually went wrong. He was deeply in the hole of stolen customer money before the market ever turned against him. He used customer money to what in sum amounted to an extremely long, high-risk bet on the success of FTX and the crypto market.
The reasons for this proactive deception have also come more and more sharply into focus. Bankman-Fried bought whole hog into the ethical framework of utilitarianism, which argues that outcomes are more important in making ethical choices than any hard and fast rules. These ideas were first instilled in him by his mother, Barbara Fried, and further weaponized by his contact with Effective Altruism leader Will MacAskill. MacAskill pushed Bankman-Fried to go into finance. In turn, his Jane Street training taught him to think of everything in terms of expected outcomes and high-stakes gambling.
Finally, and perhaps most dangerously, Bankman-Fried was raised and trained to believe that he himself was near-infallible. Both his parents and his milieu seem to have sent him the message that his reasoning ability, and even more narrowly his mathematical ability, meant he was always correct. Regardless of the causes, it is clear both from both his actions and the testimony of people close to him that Bankman-Fried thought he was the smartest guy in the room, consistently ignoring advice both from experts (most famously, lawyers) and from his own team.
This attitude seems at least partly to have come to Sam via the projections of his elders. His parents, his superiors, Effective Altruists, and the press all imbued Sam with a nearly divine level of genius. At his height, they didn’t seem to think it improbably that Bankman-Fried would save the world. He was the Chosen One, somehow separate from, and incomprehensible to, mere humans.
My favorite example of this comes via Michael Lewis, in a passage that both captures Sam’s delusion, and faithfully enacts the kind of fawning worship that helped foster that delusion. In recounting Sam’s youthful discovery that some people, against all reason, literally believed in God, Lewis describes admiringly how SBF “simply came to terms with the fact that the world could be completely wrong about something, and he could be completely right.”
In this and many other passages, and in his actions, there are signs that Lewis actually believed that Sam was uniquely gifted – even when writing about his biggest screwups. These included a bad 2016 election trade that cost Jane Street $300 million dollars – and for which Bankman-Fried faced no consequences. The takeaway, at least as Lewis recounts it, was that as long as he calculated the odds correctly, Bankman-Fried wasn’t at fault if things ultimately went badly in reality.
Novus Ordo Seclorum (or) The Ordering of the New Age
In short, Bankman-Fried was taught again and again that his desire to make high-stakes gambles was okay, because he was so smart. One important thing to note here is that his high-risk approach to decisionmaking, based on calculated odds, may superficially look like a reasonable way to account for the randomness of the world.
But the hidden reality is that the “expected value” approach to investing actually assumes a highly predictable universe. While specific outcomes may not be directly predictable, the futurist, utilitarian mindset still treats general probabilities as subject to oddsmaking.
This is where we get to the fun part – science fiction.
“In contrast to the Foundation trilogy's exaltation of rationality's march to predicted victory, Dune proclaims the power and primacy of the unconscious and the unexpected in human affairs. Paul's wild ride on the jihad, not the careful Bene Gesserit gene manipulation, provides the answer to the Empire's needs.”
- Tim O’Reilly, Frank Herbert (1981)
Isaac Asimov essentially celebrated the redeeming power of a science of predicting the future in his Foundation series of books. His fictional creation, the psychohistorian Harry Seldon, almost certainly had some lineal influence on the worldview of Sam Bankman-Fried: Foundation’s flawless managerial and mathematical science of the future had immense real-world influence.
Asimov’s most famous apparent devotee was the researcher Martin Seligman, who was partly inspired by Foundation when he pioneered the field of ‘predictive sociology’ in the 1960s. His innovation has some sinister undertones, though, partly illustrated in how his concept of “learned helplessness” has been weaponized by the right. His later work in “positive psychology” has a similar resonance with managerial technology and social control.
Things get far more dicey with another fan of Foundation: Elon Musk. Musk once tweeted that the book’s message was “you should try to take the set of actions that are likely to prolong civilization, minimize the probability of a dark age and reduce the length of a dark age if there is one.” Unfortunately but rather typically, Musk has followed that dictum to self-serving ends with his mission to send a rocket to Mars. It also seems to align with his increasingly open embrace of white supremacism, anti-immigrant paranoia, and race-war theories.
Musk is a perfect example of what Frank Herbert foresaw when he considered what Foundation was actually calling for. This is the argument made by Tim O’Reilly in his 1981 book on Frank Herbert. Yes, THAT Tim O’Reilly, the technology visionary and publisher. He very much had a dog in the fight on the question of futurism.
O’Reilly argued that Dune was “clearly a commentary on the Foundation trilogy,” citing specific comments by Herbert about the Foundation trilogy, in which he said “History… is manipulated for larger ends and for the greater good as determined by a scientific aristocracy. It is assumed, then, that the scientist-shamans know best which course humankind should take… While surprises may appear in these stories (e.g., the Mule mutant), it is assumed that no surprise will be too great or too unexpected to overcome the firm grasp of science upon human destiny. This is essentially the assumption that science can produce a surprise-free future for humankind.”
This is also the essential assumption of Effective Altruism, including its most woo-woo wings, which want us to worry about things like artificial intelligence “X-Risk” and becoming a multiplanetary species.
It’s also implicit in more conventional utilitarianism. You know who would make a perfect Reverend Mother Gaius Helene Mohaiam of the Bene Gesserit? Barbara Fried. Her rationalist moral philosophy would fit right in. And her son, of course, was even more Paul Atreides than was Mark Zuckerberg. If Zuckerberg understood the social media future, he made few claims beyond that. Bankman-Fried wanted to become the world’s first trillionaire, buy JPMorgan, and maybe become president (he put the odds at 5%).
This is where we encounter Toby Ord and Will MacAskill, the cofounders of Effective Altruism, who we’ll dive into more deeply in the second half of this chapter. Ord and MacAskill essentially believe they can both foster the real-world development of Asimov’s psychohistory, and levitate it into power over the governance structures of the human race. They believe that doing so is an imminent and pressing moral calling, because of its implications to all of the future humans who haven’t even been born yet.
Their entire system of thought hinges on a few premises. The first is that they and their friends are right, about an awful lot of things. They’re right not just about the nature of the general problem of risk to humanity, but also about their specific way of thinking about risks, and about a variety of specific calculations about risks that actually amount to little more than science fiction scenarios with numbers very roughly assigned to them.
Ord reinforces this by strongly arguing for a moral equivalency between present and future humans, rationalizing a frenetic insistence on reform and upheaval that would specifically invest more power in a global, centralized technocratic elite. Ord envisions a future of globally empowered agencies overseeing far-future risks, with sweeping supra-national oversight and enforcement powers.
The idea of a unified global government isn’t inherently authoritarian – but by combining it with this implied power to supersede the rights of present humans by invoking the rights of humans who are not yet alive, we get an Orwellian vision of truly literary subtlety and involution. We’ll be getting into more detail about that in the next part of this chapter.
As with other great thinkers, we should take Ord and MacAskill seriously, but not literally. In many settings, Effective Altruists will back away from mathematical modeling as a decision-making heuristic. The kind of vague probabilities assigned to things like an asteroid hitting Earth in the next thousand years are just aids for thinking, they’ll say. They’ll disavow any belief in scientific perfectibility, and reject the idea that utilitarianism is basically a way of laying odds that you know more than God.
But Sam Bankman-Fried was in finance, which meant anything with a probability assigned to it was an opportunity to gamble. That’s how he became an object lesson in the limits of rationality and scientistic calculation. Most EAs won’t live long enough to find out whether their “long term” bets actually pan out – all they have are their unfalsifiable calculations.
Sam Bankman-Fried, though, had to genuinely put his money where his mouth was. He bet very, very big on his own infallibility, and he lost.