starcoder

StarCoder is a 15.5-billion-parameter code language model developed by the BigCode open-scientific collaboration and released in May 2023. Built on a GPT-2-style Transformer architecture with Multi-Query Attention and Fill-in-the-Middle (FIM) training, it was pretrained on 1 trillion tokens from The Stack v1.2 spanning 80+ programming languages, then fine-tuned on an additional 35 billion Python tokens. The model supports an 8,192-token context window and is designed primarily for code completion, infilling, and technical assistance rather than general instruction following.

At release, StarCoder achieved state-of-the-art results among open-access code models, scoring 33.6% pass@1 on HumanEval and 52.7% on MBPP—matching or exceeding OpenAI's closed code-cushman-001 model. With appropriate prompting (e.g., the Tech Assistant template), it can reach 40.8% on HumanEval. The model also demonstrates strong multi-language code generation across 19 languages via MultiPL-E and excels on data science tasks (DS-1000). StarCoder is released under the BigCode OpenRAIL-M v1 license with PII redaction, attribution tracing tools, and benchmark decontamination applied to training data.

StarCoder is not an instruction-tuned chat model; it performs best on code completion and FIM tasks. Users can leverage pretraining templates for Git commits, GitHub issues, and Jupyter notebooks, or apply the Tech Assistant prompt for conversational technical Q&A. The model may generate code verbatim from its training corpus, so license attribution requirements should be respected using the provided search index.