The Scaling Era: An Oral History of AI, 2019–2025
Decent overall… Probably a little too low level for me
Why AI should continue to become very smart
Is human intelligence the result of scaling ?
Next token prediciton as a form of AI alignment mechanism.
As of this writting, the constraint is now the centralized electrical power required for the next scaled-up training runs.
Predicting the end point of AI is much easier than predicting the path to it
[X]Blogs
LLMs need ridiculous amounts of data to achieve mediocre levels of reasoning.