Rich Sutton was right
The bitter lesson keeps proving itself — general methods that scale with compute always win over hand-engineered approaches.
Rich Sutton wrote "The Bitter Lesson" in 2019. The argument: researchers who try to build in human knowledge always lose to researchers who leverage computation. Scale wins. Every time.
Seven years later, this is more true than ever. Every hand-crafted feature, every carefully designed heuristic, every domain-specific architecture — all of it gets steamrolled by throwing more compute at a general method.
The lesson is bitter because it means your clever ideas don't matter as much as you think they do. What matters is: can it scale? If yes, it wins eventually. If no, it's a dead end with a nice paper attached.
This changes how I think about what to build. Don't build the clever thing. Build the thing that gets better with more data, more compute, more time. The boring scalable thing beats the brilliant brittle thing every single time.