CodeLlama-7b-hf

CodeLlama-7b-hf is the 7-billion-parameter base variant of Meta's Code Llama family, hosted on Hugging Face under the NousResearch organization with Llama-compatible modeling code. It is initialized from Llama 2 and further trained on a code-heavy mixture (roughly 500B tokens) to specialize in program synthesis, completion, and understanding across many programming languages, while retaining some natural-language ability from mixed batches.

The model uses a decoder-only Llama transformer architecture with fill-in-the-middle (FIM) training on the 7B build, enabling prefix–suffix–middle style infilling for IDE-style completion as well as left-to-right generation. Weights are released under the Llama 2 Community License for research and commercial use subject to Meta's terms.

In Meta's Code Llama paper, the 7B base model reaches strong open-model code results on standard Python benchmarks (for example, 33.5% HumanEval pass@1 and 41.4% MBPP pass@1; higher pass@100 scores on the same suites). It supports long-context fine-tuning up to very large windows in the paper's experiments, while this Hugging Face checkpoint is configured for 16,384-token positions with extended RoPE settings.