The Bitter Lesson

The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.

Search and learning are the two most important classes of techniques that use massive amounts of computation. ¹

Researchers have tried to make systems the way the researchers thought their minds worked, which proved counterproductive.

The bitter lesson is based on these historical understandings:

1. AI researchers encode human knowledge into their methods
2. This helps incrementally, and is beneficial to the researcher.
3. This plateaus and inhibits further learning.
4. Breakthrough progress comes through a new method that uses compute more effectively, and this is seen through bitter lenses.

Footnotes

1. Almost all methods these days use ever increasing computation. ↩