How We Got Here

In 1955, four researchers asked for money to run a summer workshop at Dartmouth College. Their proposal gave the field its name, "artificial intelligence." It also made a confident prediction:

"We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer."

One problem in the proposal was how to make machines use language. Machines that use language well arrived about seventy years later. This lesson follows the main steps along that path. Each step solved a problem the previous one could not. Together, these steps explain why older techniques are still in use, why today's models are trained the way they are, and which limits remain.

Part 1: The Timeline at a Glance

The history of AI can be told as five milestones. At each one, people did less by hand, and the computer learned more on its own.

The diagram below places the five milestones on one line.

Look at the gaps between the milestones. Rule-based systems led the field for decades, but the last three milestones arrived within about ten years of each other.

Part 2: Rule-Based Systems

The Dartmouth workshop took place in the summer of 1956. For the next few decades, building AI mostly meant writing rules by hand.

A system that stores knowledge as "if this, then that" rules written by people is called a rule-based system. In that era, people often called these systems expert systems. In the 1970s, a Stanford system called MYCIN used hundreds of such rules to suggest treatments for blood infections. To build it, people interviewed doctors, turned their knowledge into rules, and entered them.

Rule-based systems had real strengths. They needed no training data, and you could trace every decision back to a rule. But they were brittle, meaning they failed on any case their rules didn't cover. Some tasks also cannot be written as rules at all. For example, nobody can write rules to recognize a cat in a photo.

Part 3: Statistical Machine Learning

In the 1990s and 2000s, the field moved toward learning from data. Statistical machine learning finds patterns in many labeled examples, instead of following rules that a person wrote.

Spam filters are the classic example. Instead of using a list of banned words, a filter learned from thousands of emails that users had marked as spam. When spammers changed their words, the filter could retrain on new examples. Search ranking, product recommendations, and fraud scoring grew up in the same era.

This step had its own limits. An engineer still had to choose the features, which are the measurable properties a model looks at, such as the number of links in an email. Choosing good features took expert effort for every new problem. Each model also needed its own set of labeled examples.

Part 4: Deep Learning

Deep learning uses neural networks with many layers that learn useful features from raw input on their own. So engineers no longer had to choose the features by hand. The ideas were decades old, but two things made them practical. The first was very large labeled datasets. The second was graphics processors, or GPUs, which can do many calculations at once.

The turning point came in 2012. A deep network called AlexNet entered ImageNet, a well-known image recognition contest. It cut the error rate from about 26 percent to about 15 percent, a far larger improvement than anyone expected. Within a few years, deep learning led the field in images, speech, and translation.

Deep learning still had two limits. First, each model was usually trained for one task, on labeled data for that task. Second, the main text models of the time read a sentence one word at a time. This made them slow to train, and they often lost track of words far back in a long passage.

Part 5: The Transformer

In 2017, researchers at Google published a paper titled "Attention Is All You Need." It introduced the transformer, a new neural network design for sequences such as text.

The key idea in the transformer is attention. Attention lets the model look at every word in a passage at once and decide which other words matter most for understanding each word. The later lesson "Attention, Intuitively" shows how this works.

This design had two important results. First, a transformer processes all the words of a passage in parallel, so it trains efficiently on GPUs. Second, it made a simple training task practical at huge scale: predict the next word in ordinary text. This task needs no human labels, because the text itself contains the right answer. As a result, models could learn from a large share of the public internet.

Through the late 2010s, models trained this way grew from millions of learned numbers to billions. As they grew, they could write, summarize, and translate with little or no task-specific training.

Part 6: ChatGPT and the Assistant Era

At the end of 2022, OpenAI released ChatGPT. The model behind it was a large transformer, trained further to follow instructions and give answers people rated as helpful. A later lesson, From Raw Model to Helpful Assistant, explains this extra training.

For researchers, the technology was not new. What changed was access. Anyone could type a question and get a fluent answer, with no code and no setup. By widely reported estimates, ChatGPT reached more than 100 million users within a few months. Generative AI moved from research labs into everyday work almost at once.

Since then, two more steps have arrived. Reasoning models work through problems step by step before answering, which helps with math, code, and planning. Agents connect a model to tools, so the model can take actions such as searching, running code, or calling an API. Later lessons cover both.

Part 7: What Each Step Removed

The clearest way to see the whole history is to ask what people had to supply at each step. The diagram below shows how that list got shorter.

Look at the pattern. At each step, one more job moved from people to data and computing power. That is why today's models need so much training data and so many GPUs. It is also why training a large model can cost a great deal, while using one can cost a fraction of a cent per request.

Part 8: What This Changes for an Engineer

This history matters to engineers for three practical reasons.

Older techniques are still in use. Rule-based checks still protect payments, and small machine learning models still filter spam and rank search results. A new feature can use all of these tools together, and it often should.

Past limits explain today's designs. Many choices in modern AI systems come from a past limit. Models train on unlabeled text because labeling does not scale to that much data. Models also have a maximum input size, partly because attention gets expensive on very long text. Knowing the limit makes the design make sense.

Ideas last longer than models. Model names change quickly, but the core ideas change slowly. If you learn tokens, attention, and the stages of training, that knowledge stays useful after today's models are replaced.

Common Mistakes

Mistake 1: Treating generative AI as a replacement for everything before it. Rules and small models are still the right tool for many jobs, and most real systems use a mix of them.

Mistake 2: Thinking ChatGPT invented the technology. It built on decades of research and on the transformer. Its big change was making that research easy for everyone to use.

Mistake 3: Memorizing model names instead of ideas. Model names go out of date within months. The ideas behind them last much longer.

Key Takeaways

• AI moved from rules to learning. At each step, more work moved from people to data and computing power.
• Deep learning removed hand-chosen features. Large datasets and GPUs made neural networks with many layers practical, starting around 2012.
• The transformer made learning without labels practical at scale. Attention and parallel training let models learn from huge amounts of ordinary text.
• ChatGPT changed access, not only quality. A chat window put generative AI in front of everyone at once.

The next lesson, What GenAI Can and Can't Do Today, looks at where these models are strong and where they fail. It also explains why a confident but wrong answer is the problem every engineer must design around.