dsh-model-deploy

Before committing a large language model to specific hardware, the most common questions are: will it fit in memory? How long is the time-to-first-token? How many concurrent requests can it serve? Is the power bill acceptable? These judgments have traditionally relied on personal experience and repeated trial-and-error. dsh-model-deploy turns them into an auditable, structured report via a set of published first-order analytical formulas.

Its core capability is to take model parameters, precision, context length, concurrency, and interconnect bandwidth as inputs and produce four result blocks: memory breakdown (weights / KV cache / activations / runtime, per-GPU and cluster-level), latency (TTFT and total, idle and saturated), throughput (per-request and system tok/s, req/s, prefill tok/s), and power (idle / typical / peak, including PUE and energy per token). It also automatically searches for the optimal TP x PP x DP parallelism combination and explains the rationale. The catalog covers 38 mainstream models and 20 GPU/NPU types, spanning NVIDIA, AMD, Huawei Ascend, Pingtouge Hanguang, Cambricon, Iluvatar, MetaX, and Moore Threads. All formulas and coefficients are public, and results can be cross-checked against real deployment anchors.

It is designed for inference cluster planners, MLOps engineers, and architects who need a rapid technical sanity check before procuring or self-deploying large models. No prototype build is required: enter the model and hardware parameters to get an assessment report with remediation suggestions and explicit assumptions, helping narrow down candidates across dozens of hardware combinations.