AI Agent Hub
Back to models
AquilaChat2-70B-Expr logo

AquilaChat2-70B-Expr

Open Source BAAI Released 2023-11-30
-- 70.0B params 4.1K context Open Source

About this model

Aquila_logo

English | 简体中文

We opensource our Aquila2 series, now including Aquila2, the base language models, namely Aquila2-7B, Aquila2-34B and Aquila2-70B-Expr , as well as AquilaChat2, the chat models, namely AquilaChat2-7B, AquilaChat2-34B and AquilaChat2-70B-Expr, as well as the long-text chat models, namely AquilaChat2-7B-16k and AquilaChat2-34B-16k

The additional details of the Aquila model will be presented in the official technical report. Please stay tuned for updates on official channels.

Quick Start

1. Inference

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from transformers import BitsAndBytesConfig

model_info = "BAAI/AquilaChat2-70B-Expr"
tokenizer = AutoTokenizer.from_pretrained(model_info, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_info, trust_remote_code=True, torch_dtype=torch.bfloat16)
model.eval()
text = "请给出10个要到北京旅游的理由。"
from predict import predict
out = predict(model, text, tokenizer=tokenizer, max_gen_len=200, top_p=0.95,
              seed=1234, topk=100, temperature=0.9, sft=True,
              model_name="AquilaChat2-70B")
print(out)

License

Aquila2 series open-source model is licensed under BAAI Aquila Model Licence Agreement

Acknowledgements

This work is supported by the National Science and Technology Major Project (No. 2022ZD0116300). 本项目受新一代人工智能国家科技重大专项（No. 2022ZD0116300）支持。

Technical Specs

  • Parameters: 70.0B
  • Architecture: Transformer
  • Context Window: 4,096 tokens
  • Input Modalities: text

Hardware Requirements

  • VRAM: 161.0 GB
  • Compute: Multi-GPU (~161 GB VRAM total, e.g. 2x NVIDIA A100 80GB)