paligemma-3b-pt-224

PaliGemma-3B-pt-224 is Google's open-weights pretrained vision-language model (VLM) designed as a versatile base for transfer learning on diverse image-text tasks. This variant uses 224×224 input images and supports 128-token text input/output sequences. It accepts an image plus a text prompt (e.g., captioning, VQA, detection, segmentation) and generates text output, including structured tokens for bounding boxes and segmentation masks.

The model combines a SigLIP-So400m/14 Vision Transformer encoder with a Gemma-2B autoregressive Transformer decoder, totaling ~3 billion parameters. It was pretrained on a large multilingual mixture including WebLI, CC3M-35L, VQ²A-CC3M-35L, OpenImages, and WIT, following the PaLI-3 recipe. Weights are available in float32, bfloat16, and float16 for fine-tuning and deployment.

PaliGemma is intended for single-turn, task-specific fine-tuning rather than zero-shot chat. After transfer, pt-224 achieves strong results on vision-language benchmarks such as VQAv2 (83.2%), ScienceQA (95.4%), and GQA (65.6%). It is not a conversational chatbot; for interactive multi-task use, Google provides separate mix fine-tuned variants.