Chapter 5: What Made AI Possible?

AI did not appear overnight. It emerged from the convergence of three forces: better algorithms, massive data, and increasingly powerful computing hardware.

Individually, these ingredients are useful. Together, they unlocked the breakthroughs behind modern AI and allowed large-scale systems to learn, generate, and adapt in ways that were impossible only a decade ago.

Algorithms are the step-by-step methods that tell AI systems how to learn from data and make predictions. Over decades, researchers developed increasingly powerful learning techniques.

Neural networks allowed systems to process information in layered structures inspired (loosely) by the brain. Deep learning expanded this idea with many layers, enabling breakthroughs in vision, speech, and language.

Then, in 2017, a new architecture called the Transformer changed everything. Transformers made it possible for models to understand long-range patterns in text, enabling today’s large language models and generative AI.

As the internet grew, so did the amount of digital information available — text, images, audio, video, social interactions, logs, sensors, and more. This explosion of data provided the raw material for training increasingly capable models.

AI systems learn patterns by observing millions or billions of examples. The more diverse and higher-quality the data, the more reliable the model becomes.

In recent years, techniques like synthetic data, fine-tuning, and retrieval-augmented generation (RAG) have made it possible to train models more efficiently and safely by giving them access to curated knowledge, not just raw text.

Training large AI models requires enormous computing power. For many years, CPUs were too slow for deep learning, but new hardware accelerated the field.

GPUs (Graphics Processing Units), originally built for 3D graphics, became essential because they can perform thousands of calculations simultaneously. TPUs (Tensor Processing Units) were later developed specifically for AI workloads.

Today, clusters of thousands of GPUs and TPUs, along with cloud computing, make it possible to train models with trillions of parameters. This scale of compute was unimaginable even 10 years ago.

AI also requires significant energy and cooling, making data centers and efficiency improvements critical parts of the ecosystem.

A major discovery in the 2020s was that AI performance tends to improve predictably as models grow in size, as long as data and computing power scale alongside them.

These AI scaling laws encouraged companies to build increasingly large models — leading directly to the emergence of frontier LLMs capable of reasoning, summarizing, generating text, writing code, and learning from user input.

Recently, a parallel trend has emerged: small language models (SLMs) that run efficiently on laptops, phones, and embedded systems. This shift signals a future where powerful AI doesn’t just live in the cloud, but also on personal devices.

Modern AI is the result of three reinforcing breakthroughs: algorithms that learn deeply, abundant data to learn from, and hardware fast enough to train massive models.

Together, these forces opened the door to the generative AI systems we use today — and will continue shaping the next decade of AI innovation.