I'd recommend anyone who wants to understand the emergence of AI and OpenAI in particular to read this book.
I do struggle reading non-fiction, and this book was no different. It took me nearly 2 months to finish reading (in my evening snippets) - but it was definitely worthwhile. It was this 90 minute interview by Novara Media with Karen Hao that made me immediately purchase the book.
I don't think I can fully review this book and do it justice, but I can share what I learnt from the book where my original assumptions were wrong about AI and AI companies.
The book is focused primarily around OpenAI and Sam Altman, and in particular how his ousting in November 2023 came to be, through hundreds of interviews and documents, and paints a very insightful picture of the messiah complex (though not in her words) that Altman has.
Here's a short summary of things I didn't know but learnt through reading this:
- Altman and friends started conversations (and business, albeit initially as a non-profit, though that didn't last) in 2015 because they were going to build AGI and that it was inevitable.
- The company was formed with scientific researchers originally, again as an open company with the intent of sharing, though spoiler alert: this all changed
- I had always assumed it was cowboy bro developers working on the code, but it was, originally, academic based engineering
- AI safety was constantly there and initially significant - but as we all know now, eventually lost a battle to make an impact, left "hobbled" and thrown rather to the wayside.
- The training data was, after GPT-2, ingested wholesale and attempt to clean/sanitise would happen on the results coming from prompts - i.e. the inputs were not cleaned, which means applying dizzying array of filters to catch on the output and edge cases.
- Common Crawl was introduced at GPT-3 - which is also where the input filtering stopped happening
- AI, or Western AI companies including OpenAI (but also Google and Microsoft) put their data centres in the Global South, additionally sourcing their data annotators from the poorest countries allowing them to pay (via third parties) literal pennies per hour for the work (which could also come with terrible mental health side effects as the worker would read and view the generative content that AI could come up with based on the unfiltered dark corners of the web)
- Sam Altman lies. Little lies, but from a great deal of documentation, a lot and often to tell people what they want to hear whilst (we guess?!) having some ulterior motive
- The path that OpenAI decided to take to head towards what they believe will be AGI, effectively requires unlimited compute power, when in reality, there are lots of different applications of AI that don't need that level of power, Stable Diffusion being one such example trained using 256 GPUs (still not a desktop computer, but not hundreds of thousand GPUs either)
- OpenAI's approach, to close off it's scientific findings, close it's source and refusing to share methods means that there's no way to verify any of their progress, but more importantly is stripping the academic scientific community of it's researchers (as someone who has visited CERN on two occassions, seeing science being shared is incredible and incredible for society)
My only complaint about the book (and it's likely to be my own fault) is I had trouble with the jumping backwards and forwards in time - I'd often be unsure where we were in the timeline.
If you work in tech, I'd absolutely recommend this book. If it's not possible, then definitely the interview I linked above.
36 Highlight(s)
even less time, it is now on track to rewire a great many other critical functions in society, from health care to education, from law to finance, from journalism to government. The future of AI—the shape that this technology takes—is inextricably tied to our future
OpenAI, the anti-Google, would conduct its research for everyone, open source the science, and be the paragon of transparency.
OpenAI, the anti-Google, would conduct its research for everyone, open source the science, and be the paragon of transparency.
Based on the pace of Moore's Law, how long would it take to reach the level of compute OpenAI needed for brain-scale AI? The answer was bad news: It would take far too long.
OpenAI's Law was projecting that OpenAI would need thousands, if not tens of thousands, of GPUs to train just a single model. The cost of electricity to power that training would also explode.
GPT-2 that was capable of generating passages of text that closely resembled human writing. In February that year, OpenAI had taken the unusual step of proclaiming to the press that this model, once advanced a little further, could become an exceedingly dangerous technology. Authoritarian governments or terrorist organizations could weaponize the model to mass-produce disinformation. Users could overwhelm the internet with so much trash content that it would be difficult to find high-quality information
While they hoped that AI would "be useful in reducing the costs associated with climate action, humanity also must decide to act."
A recent study from the University of Massachusetts Amherst had placed alarming numbers on the huge and growing carbon emissions of training larger and larger AI models.
It was this fundamental assumption—the need to be first or perish—that set in motion all of OpenAI's actions and their far-reaching consequences. It put a ticking clock on each of OpenAI's research advancements, based not on the timescale of careful deliberation but on the relentless pace required to cross the finish line before anyone else. It justified OpenAI's consumption of an unfathomable amount of resources: both compute, regardless of its impact on the environment; and data, the amassing of which couldn't be slowed by getting consent or abiding by regulations
To many AI developers who have long operated under this mindset, that question seems rather quaint; taking it seriously presents a direct obstacle to the moral pursuit of ever-more progress
"The human brain has about 100 trillion parameters, or synapses," Hinton told me in 2020. "What we now call a really big model, like GPT-3, has 175 billion. It's a thousand times smaller than the brain. "Deep learning is going to be able to do everything," he said.
Human psychology naturally leads us to associate intelligence, even consciousness, with anything that appears to speak to us
No matter their scale, neural networks are still statistical pattern matchers. And those patterns are still at times faulty or irrelevant
Altman has publicly tweeted that "ChatGPT is incredibly limited," especially in the case of "truthfulness," but OpenAI's website promotes GPT-4's ability to pass the bar exam and the LSAT.
The perpetuation of the empire rests as much on rewarding those with power and privilege as it does on exploiting and depriving those, often far away and hidden from view, without them.
unable to compete, losing PhD students and professors to industry, atrophying independent academic research, and spelling the beginning of the end of accountability.
GPT-2, in other words, had been peak data quality; it declined from there.
Google's commitment to compliance, ironically, gave OpenAI easier access to Google's data than Google itself.
was a warning that Big AI was increasingly going the way of Big Tobacco, as two researchers put it, distorting and censoring critical scholarship against the interests of the public to escape scrutiny
from Microsoft arriving in the third quarter with eighteen thousand Nvidia A100s, the newest, most powerful GPUs then in existence
One of the defining features that drives an empire's rapid accumulation of wealth is its ability to pay very little or nothing at all to reap the economic benefits of a broad base of human labor.
"No matter how you train it, it's going to have probabilities on things and it's going to have to guess sometimes."
Generating one thousand images used on average as much energy as 242 full smartphone charges; in other words, every AI-generated image could consume enough energy to charge a smartphone by roughly 25 percent
While scale can lead to more advanced capabilities, the inverse is not true: Advanced capabilities do not require scale.
we need more people just debunking—just looking at what people are saying and being like, 'Actually, reality is more complicated.'"
OpenAI's policy team delivered a submission to the UK House of Lords communications and digital select committee, saying that it would be "impossible" for OpenAI to train its cutting-edge models without copyrighted materials.
Among many employees, the conclusion delivered the final assurance they needed. The crisis was over. And then, suddenly, it was not.
Readying Scallion became a whole-of-company effort. To many in Applied, the breakneck pace proved exhilarating if exhausting; researchers and engineers began pulling absurd hours, including through weekends, to stay on track. But to the hobbled Safety clan, it was yet more alarming evidence of the continued deprioritization of AI safety.
If Altman was being brazen and boastful, most likely something wasn't going well.
Kokotajlo observed within his exit documents what Piper would detail in her story: If he didn't sign a nondisparagement agreement, committing to never speaking negatively about the company, he would forfeit his vested equity. If he did sign it, he could still risk losing it if he broke the agreement, which also included a gag order that barred him from disclosing its existence
The Blip and the Omnicrisis were one and the same: the convulsions that arise from the deep systemic instability that occurs when an empire concentrates so much power, through so much dispossession, leaving the majority grappling with a loss of agency and material wealth and a tiny few to vie fiercely for control. Instead, OpenAI chose to fortify itself against the criticism
It was a bizarre and incoherent strategy that only made sense under one reading: OpenAI would do whatever it needed, and interpret and reinterpret its mission accordingly, to entrench its dominance.
"Data is the last frontier of colonization,"6 Mahelona told me: The empires of old seized land from Indigenous communities and then forced them to buy it back, with new restrictive terms and services, if they wanted to regain ownership. "AI is just a land grab all over again. Big Tech likes to collect your data more or less for free—to build whatever they want to, whatever their endgame is—and then turn it around and sell it back to you as a service
What I reject is the dangerous notion that broad benefit from AI can only be derived from—indeed, will ever emerge from—a vision for the technology that requires the complete capitulation of our privacy, our agency, and our worth, including the value of our labor and art, toward an ultimately imperial centralization project.
"As the dust settles on this chapter, one thing remains clear: the human toll of unchecked power and unbridled greed,"
As Joseph Weizenbaum, MIT professor and inventor of the ELIZA chatbot, said in the 1960s, "Once a particular program is unmasked, once its inner workings are explained in language sufficiently plain to induce understanding, its magic crumbles away."