Llama-2-7b-chat-hf

Llama-2-7b-chat-hf is Meta's 7-billion-parameter dialogue-optimized variant of the Llama 2 family, released in July 2023 under the Llama 2 Community License. It is built on an auto-regressive Transformer architecture and fine-tuned with supervised fine-tuning (SFT) plus reinforcement learning from human feedback (RLHF) to align responses with human preferences for helpfulness and safety. The model was pretrained on 2 trillion tokens of publicly available data and further tuned with over one million human-annotated examples, with a 4,096-token context window.

On standard academic benchmarks reported in the official Llama 2 model card and paper, the underlying 7B base model achieves 45.3 on MMLU (5-shot), 32.6 on BBH (3-shot), 29.3 on AGIEval, 14.6 on GSM8K, and 12.8/20.8 on HumanEval/MBPP respectively. The chat-tuned variant substantially improves safety-related metrics, scoring 57.04 on TruthfulQA compared to 33.29 for the pretrained base model. Meta reports that Llama-2-Chat models outperform contemporary open-source chat models on most benchmarks tested and perform competitively with some closed-source assistants in human evaluations.

Intended for commercial and research use in English, Llama-2-7b-chat-hf is designed for assistant-style chat applications. It requires specific prompt formatting with INST tags and special tokens for optimal performance. The model weights are approximately 13.5 GB in FP16, making it deployable on a single consumer or datacenter GPU with 16 GB or more VRAM.