LeCun: "In 4 years, a child has seen 50 times more data than the biggest LLMs."

I've made that point before:
- LLM: 1E13 tokens x 0.75 word/token x 2 bytes/token = 1E13 bytes.
- 4 year old child: 16k wake hours x 3600 s/hour x 1E6 optical nerve fibers x 2 eyes x 10 bytes/s = 1E15 bytes.

In 4 years, a child has seen 50 times more data than the biggest LLMs.… https://t.co/09atbzWsFP

— Yann LeCun (@ylecun) January 25, 2024

H/t Arnold Kling.