The Bitter Lesson

The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.

Search and learning are the two most important classes of techniques that use massive amounts of computation. Almost all methods these days use ever increasing computation.

Researchers have tried to make systems the way the researchers thought their minds worked, which proved counterproductive.

The bitter lesson is based on these historical understandings:

1. AI researchers encode human knowledge into their methods
2. This helps incrementally, and is beneficial to the researcher.
3. This plateaus and inhibits further learning.
4. Breakthrough progress comes through a new method that uses compute more effectively, and this is seen through bitter lenses.