Fine-tune an LLM.
Size the training recipe.
GPU memory requirements depend heavily on whether you train all parameters or use a parameter-efficient method. Define the recipe before choosing the hardware.
1. Define the
method.
Document the base model, trainable parameters, precision, optimizer, sequence length and target batch size. Full-parameter training and adapter-based methods have very different memory profiles.
- Base model and precision
- Full or parameter-efficient fine-tuning
- Sequence length and batch size
- Optimizer and distributed strategy
2. Establish a
memory baseline.
Account for model weights plus the training state required by your method. Then leave headroom for activations, framework allocations and communication buffers.
GPU selection guide3. Measure the
training run.
Track memory use, step time and useful training throughput. Compare configurations using the complete expected run cost rather than the accelerator's hourly price alone.
Deploy training compute