What if LLMs are mostly crystallized intelligence?
LLMs are better at developing crystallized intelligence than fluid intelligence. That is: LLM training is good at building crystallized intelligence by learning patterns from training data, and this is sufficient to make them surprisingly skillful at lots of tasks. But for a given capability level in the areas they’ve trained on, LLMs have very weak fluid intelligence compared to humans. For example, two years ago I thought human-level SAT performance would mean AGI, but turns out LLMs can do great at the SAT while being mediocre at lots of other tasks.
I’m not saying LLMs are just parrots (that’s dumb). [1] There’s a continuity between crystallized and fluid intelligence.
Empirically, it’s unclear how fluid their intelligence is: we see both general reasoning skills and jaggedness.
It’s worth considering: what if fluid intelligence progress is relatively slow, and LLM capabilities mostly grow with relevant training data?
This could imply slower AI progress, especially if general-purpose data runs dry relatively soon. (Epoch estimates 2026-2032.) That means companies will need to prioritize specialized data collection/generation, which will lead to jagged capabilities growth favoring the prioritized areas.
[Epistemic status: I only put like 20% on worlds where this dynamic puts a serious damper on AI progress compared to e.g. the AI Futures Project ’s median timelines. It’s important to stay aware of these possibilities, though, and track the relevant evidence.]