LLMs need ridiculous amounts of data to achieve mediocre levels of reasoning.
- Suggests that they are not really generalizing
- If they can't achieve human-level reasoning with 20k years worth of data a human would see, we should entertain the possibility that 1e12 years worth of data would also be insufficient.
There is no amount of jet fuel you can add to an airplane to make it reach the moon.