Parameters
The internal numbers (weights and biases) inside a neural network that get adjusted during training. A 70-billion-parameter model has 70 billion adjustable internal numbers encoding everything it has learned.
Also known as: weights, model size
When you hear “GPT-4 is a 1.7 trillion parameter model” or “Llama 70B,” the number refers to how many adjustable internal values the neural network has. Each parameter is a single floating-point number, usually starting as a random value and gradually being tuned during training to make the model produce useful outputs. The collection of all parameters at the end of training is the model. Loading a model means loading those parameters into GPU memory.
Parameter count is the rough proxy for model capability. More parameters means more capacity to learn patterns, store facts, and produce coherent reasoning. A 7-billion-parameter model can write basic prose and answer simple questions; a 70-billion-parameter model can handle complex reasoning, code generation, and long-form writing; a frontier model of several hundred billion parameters can do near-expert-level work in many domains. The relationship isn’t strictly linear (training quality matters at least as much as raw parameter count) but as a rough heuristic, bigger models are more capable.
Parameter count also determines hardware requirements. At 16-bit precision, each parameter takes 2 bytes of GPU memory, so a 70B model needs 140GB of VRAM just to load. That’s more than any single consumer GPU has, which is why running large models requires either expensive multi-GPU setups or quantising the model to 4-bit or 8-bit precision (which fits 70B into ~40-80GB and runs on a single RTX 4090 or A100). The OYM “Why Self-Host Your AI” article walks through the practical hardware tradeoffs.
The DeAI angle on parameters is that smaller open-weight models (7B, 13B, 32B) are the realistic target for sovereign self-hosting because the hardware to run them is affordable. Frontier 100B+ models running on consumer hardware require aggressive quantisation that hurts quality. This is part of why the open-weight model comparison page tracks parameter count alongside benchmark performance: it tells you not just how good a model is but how achievable it is to actually run on your own infrastructure.