Why domain specific LLMs won’t exist: an intuition
Why not have different models for different capabilities?
I wondered this myself and thought that it was quite a natural way for the industry to progress. We would have a healthcare model, economics model, mathematics model, coding model and so on.
But why haven’t we seen any such innovation in this space? I looked quite deeply (ok I made a GPT search) and found that there does not exist any serious competitor to general models. Maybe on cost but even that is weak. The pattern looks very clear to me: no domain specific model beats a general LLM model in its own domain reliably.
The only weak answer that ChatGPT gave was something called “medGPT” which seemed to beat general models in very limited tasks like Name Entity Recognition but I’m not fully convinced.
The big providers like OpenAI and Anthropic would have made domain specific models if there was any feasibility in it. They haven’t and that’s a clue. What we have is Codex which resembles a coding model but recent developments in GPT 5.4 (which beats Codex in all dimensions) tells me it’s not where they want to head.
Ok then why don’t they exist?
The reason no competitive domain specific models exist is because intelligence is compounding. What I mean by that is, intelligence in different domains carry over and multiply. If a model learns something intricate in mathematics, this general reasoning capability carries over to other domains. So is the case with coding: all the pre and post training done in programming datasets definitely help the model develop general intelligence.
So you have intelligence from different domains compounding on each-other, creating intelligence that is so general that it is not possible to get it from a limited data set.
Think about it: would you trust a medical specific LLM model if it can’t do general reasoning as well as GPT?
My prediction is that the industry will move this direction: have only generalised models.
It could be that we hit some physical limit in model size, in that case we might have domain specific models. And I think that situation would resemble how our own society functions: we have people specialising in different fields and working together to solve problems.