How tbd-0 stays small.
Four ideas make our models efficient. None of them is about making the model bigger.
Constant context size
Most language models treat a conversation as one growing block of text. On every turn, the whole history goes through the model again. Long conversations get slower and more expensive, and details in the middle get lost.
tbd-0 works with a context of fixed size. What the model needs right now stays in the context. Everything else moves into memory and comes back when it becomes relevant.
The result: turn 50 costs the same as turn 1, conversations have no length limit, and nothing gets lost in the middle.
Three more ideas
-
Expert models
A general-purpose model has to know a little about everything. An expert model only has to know one field well. That makes it smaller and cheaper to run, and on tasks in its field it outperforms larger general-purpose models.
-
Retrieval models
Facts don’t have to live in the model’s weights. Our retrieval models keep knowledge in a separate layer and look it up when needed. Knowledge can be updated without retraining, and the model itself stays small.
-
Distributed mixture of experts
A mixture-of-experts model splits its parameters into specialised parts and only activates the ones a request needs. We spread these experts across machines, so each request uses a fraction of the compute and the system grows by adding machines, not by building bigger ones.
What it means in practice
Lower cost
Less compute per answer and a flat cost per turn, billed per turn instead of per token.
Where the savings come fromRuns where you need it
Small models run on hardware you can operate yourself, in our European cloud or on-premise.
SovereigntyNo length limit
Long support threads, reviews and research sessions stay fast and keep their details.
Use casesWant the details?
We’re happy to go deeper in a technical conversation.