paligemma-3b-ft-cococap-448

PaliGemma-3B-ft-cococap-448 is a task-specific fine-tuned variant of Google's PaliGemma, a lightweight 3-billion-parameter vision-language model (VLM) released in May 2024. It combines a SigLIP-So400m/14 Vision Transformer image encoder with a Gemma-2B Transformer text decoder, following the PaLI-3 training recipe. This checkpoint is fine-tuned exclusively on the COCO Captions dataset using 448×448 pixel input images and is optimized for image captioning rather than general conversational use.

On the COCO Captions validation set, the model achieves a CIDEr score of 144.6, representing strong transfer performance for the captioning task at 448-pixel resolution. PaliGemma accepts both an image and a text prompt (e.g., "caption en" for English captions) and generates descriptive text output. It supports multiple languages and is available in float32, bfloat16, and float16 formats via Hugging Face Transformers, with optional 4-bit and 8-bit quantization for reduced memory footprint.

The model is intended for research and fine-tuning workflows in vision-language tasks such as image captioning, visual question answering, and related applications. It is a single-turn model not designed for multi-turn dialogue. Weights are released under the Gemma Terms of Use and require accepting Google's license on Hugging Face before download.