starcoderbase-1b

StarCoderBase-1B is a compact 1-billion-parameter code language model released by the BigCode community as a smaller variant of the StarCoder family. It is trained on over 80 programming languages from The Stack v1.2 (with opt-out requests excluded), using 1 trillion tokens and the Fill-in-the-Middle (FIM) objective. The model builds on a GPT-2-style decoder architecture enhanced with multi-query attention for efficient inference and supports an 8,192-token context window.

Designed primarily for code completion and infilling rather than instruction following, StarCoderBase-1B excels at generating and completing code snippets when given sufficient context. It achieves a 15.17% pass@1 score on HumanEval and supports fill-in-the-middle generation via special FIM tokens (, , ). The model was trained on 128 Tesla A100 GPUs over 11 days using Megatron-LM and is released under the BigCode OpenRAIL-M license.

As a base (non-instruction-tuned) model, StarCoderBase-1B does not respond well to natural-language commands like "write a function that..." without appropriate prompting. It is best suited for IDE integrations, code completion plugins, and lightweight local deployment on developer machines where larger code models would be impractical. Users should note attribution requirements when generated code may reproduce training data verbatim.