Infrastructure for your LLM.
Select a Lenovo or generic server, configure its components, and explore your model’s memory requirements.
Current catalog: Lenovo platforms and generic configuration.
Configure host
8 Lenovo + GenericAdditional networking, power, and physical conditions
Adapters are grouped by model and location. The stated quantities do not replace the assignment of each slot, PCIe width, and cable. Refer to the product guide for storage and mounting options.
Requirements and outstanding checks
CPU, RAM, disk, and network performance
Profiles automatically calculate an indicative range. Enter a measurement to override it; 0 uses the profile. For RAM, 0 means the data is still pending. The analysis uses the requested target, not an observed workload.
Disk: interface/form-factor profiles, not a catalog of purchasing part numbers. Final mounting compatibility remains to be checked. Network: the fastest selected port is used to serve the API. Formulas, scope, and sources ↗
Host planning and request arrivals
Editable allowances per host. CPU capacity is assessed using both physical cores and tokenization throughput. Inbound and outbound traffic are evaluated separately on full-duplex links.
Enable this only if the engine keeps that cache on the host. A queue without cached data does not use this allowance. Startup time compares overlapped and sequential loading; it excludes deserialization.
Enterprise software and services
Select the planned components. Local resources are added to the workload; external services are listed separately. Terms and part numbers come from a documented catalog, without prices.
Additional NIM / Triton allowance
Applies only when the runtime is selected. This is a project-specific allowance, not a requirement of the NVIDIA AI Enterprise license; avoid counting resources already included in your engine measurement twice.
Energy consumption
Editable assumptions, not measurements: CPU/GPU idle power at 20% of TDP, plus the remaining 80% proportional to the specified load; 8 W per DIMM, 6 W per drive, and 15 W per network adapter. Includes power supply losses. PUE 1 excludes external cooling.
Measured power replaces the entire host estimate. For GB300, use the power of the complete solution, not a single tray. TDP is a thermal reference: this approximation neither predicts an LLM engine’s power consumption nor sizes power supplies.
View the audited catalog, coverage, and sources ↗ · A selection for sizing purposes; not a Lenovo-certified bill of materials.
Define the LLM workload
GPU distribution and cache residency
Calculations assume homogeneous GPUs within one host. Remaining GPUs form complete replicas. PP uses the largest stage and allows for embeddings in every stage. TP/PP require engine support; parallelism across hosts is not simulated.
Model architecture
MHA, GQA, MQA, and compressed MLA. For preloaded models, embeddings and the output layer are budgeted in BF16 (one matrix if tied weights are declared; otherwise two); for custom models, adjust this value to match the quantized file. For MoE, enter all parameters, not just the active ones. Recurrent states, DSA, and linear attention are not modeled.
Quantization, paging, and engine allowances
Metadata depends on the format and engine. These controls make the estimate explicit; they do not certify that a kernel supports the combination.
Margins, execution, and training
AdamW: two FP32 states (8 bytes per trainable parameter), plus gradients and an optional master copy. Neither ZeRO nor checkpointing is applied. Replicas do not share memory. The base context allowance per GPU and the additional margin are independent allowances.
What this version calculates
Weights and cache
Quantized weights are separated from parameters that retain BF16. KV cache depends on architecture, context, residency, blocks, and reused prefixes. Batch size is treated separately from the total number of users.
MHA/GQA/MQA: 2 × layers × KV heads × dimension × tokens × bytes. Compressed MLA: layers × (latent dimension + RoPE) × tokens × bytes. TP may replicate KV heads; MLA is budgeted as replicated across TP ranks.
Peak per GPU
Weights, KV cache, activations, logits, training, communication, engine overhead, and margin are added together. TP splits weights; PP distributes layers. The largest stage is budgeted, with embeddings in every stage as a conservative upper bound.
Activations: 2 × K × tokens × hidden dimension; training also accounts for layers per stage. K is editable. Without memory-efficient attention, an approximate quadratic matrix is added. Chunked prefill limits inference buffers, not the total KV cache.
Sources and limitations
Memory and performance estimates based on model parameters, workload, and selected hardware. Results are indicative, not measurements.
Indicative speed for preloaded models with TP=1 and PP=1: memory read throughput and compute are compared using a range of efficiencies. This is not a benchmark. ZeRO/FSDP, parallelism across hosts, and speculative decoding are not included. All memory is expressed in decimal GB (1 GB = 1,000,000,000 bytes). Published nominal capacities are used; usable memory accounts for the selected reserve.
View formulas, assumptions, and references ↗