CAT-Thinking-8B
About this model
CAT-Thinking 🐱
Tiny Language Model that thinks in Japanese
CAT-Thinking is trained to generate reasoning trace in Japanese by reinforcement learning. The model is based on Qwen3-Swallow-v0.2 which is a continual pretraining model based on Qwen3 to read and write fluently in Japanese.
Usage
CAT-Thinking is designed to reason in Japanese even if the input text is in English. The model is trained with the maximum output token length of 4096. We recommend setting max_new_tokens to at least 4096, and larger for difficult problems. Although the model is trained to respond within 4096 tokens, it tends to generate longer responses, especially for difficult and/or confusing instructions. It often gets stuck in repetition, especially when the instruction is confusing (e.g., two contradicting instructions are given). To mitigate the probability of repetition, we find repetition_penalty=1.05 or larger to be useful.
The model is compatible with all the libraries Qwen-3-8B is. To run with transformers, run the following command.
pip install transformers
from transformers import pipeline
# Load the model
chat_pipeline = pipeline("text-generation", model="CyberAgent/CAT-Thinking-8B")
prompt = "You have two cats, one male and one female. A female cat gives birth to up to 12 kittens per year.\n" + \
"Assume you don't spay them. In three years, how many cats might you need to take care of at most?"
user_input = [{"role": "user", "content": prompt}]
response = chat_pipeline(user_input, max_new_tokens=8192, temperature=0.8, top_p=0.95)
print(response[0]['generated_text'])
Evaluation
We conducted evaluation on coding and math tasks in Japanese and English. We compare CAT-Thinking with its counterparts, Qwen-3-8B (Qwen-3) and Qwen3-Swallow-8B-RL-v0.2 (Swallow) Random sampling (temperature=0.8, top_p=0.95, max_new_tokens=4096) is used for all runs.
- mbpp
- HumanEval
- JHumanEval
- LiveCodeBenchv6
- GPQA Main (En)
- GPQA Main (Ja)
- PolyMath (En)
- PolyMath (Ja)
- AIME 26
For all instances, CAT-Thinking generated its reasoning trace in Japanese while the comparison models generated theirs in English. The results show that it is feasible to think in Japanese while retaining the model's performance on English tasks. The benchmarks were published before the models were released, thus they should be interpreted as reference points.
A detailed experimental evaluation will be presented in a technical report.
Generation Examples
The reasoning trace of CAT-Thinking is in Japanese but not a natural one. In particular, it tends to start with a weird phrase such as:「済みの問題を解くことは許可されていますか?」「済みの数学問題かを確認する必要がありますか?」. It occurred during the second GRPO training and the model ended up learning to start with this phrase.
Example of Coding
PROMPT: Write a function to check if the given integer is a prime number. Name the function as follows: def prime_num(num):
RESPONSE:
解説
素数(prime)とは「1 とその数自身以外に約数を持たない 1 より大きい整数」のことです。
判定の基本は「√num までの整数で割り切れるものが無ければ素数」という考え方です。
- 2 は唯一の偶数の素数なので特別扱いします。
- それ以外の偶数は必ず合成数です。
- それ以外の奇数については 3 から √num までの奇数で割り切れないかを調べます。
このアルゴリズムは最悪でも O(√n) の時間で判定でき、入力サイズがそれほど大きくない限り十分高速です。
実装例(Python 3)
import math
def prime_num(num: int) -> bool:
"""
与えられた整数が素数かどうかを判定して bool を返す関数。
Parameters
----------
num : int
判定したい整数。負の数や 0, 1 は素数ではないとみなす。
Returns
-------
bool
素数であれば True、そうでなければ False。
"""
# 1 以下の数は素数ではない
if num <= 1:
return False
# 2 は唯一の偶数の素数
if num == 2:
return True
# 偶数は 2 以外すべて合成数
if num % 2 == 0:
return False
# 3 から√num までの奇数で割り切れないか調べる
limit = int(math.isqrt(num)) # 整数平方根を取得(Python3.8 以降)
for i in range(3, limit + 1, 2):
if num % i == 0:
return False
# ここまで来たら割り切れる約数が無かった → 素数
return True
ポイント
math.isqrt(またはint(math.sqrt(num)))で整数の平方根を取得し、ループ回数を最小化しています。- 偶数の除外を行うことで、ループは奇数だけを対象にし、実行速度が約半分に向上します。
- 関数は
boolを返すだけにし、呼び出し側でprint(prime_num(x))のように結果を出力できます。
使い方の簡単な例
print(prime_num(2)) # True
print(prime_num(15)) # False
print(prime_num(29)) # True
print(prime_num(1)) # False
この関数をそのままプロジェクトに組み込めば、整数が素数かどうかの判定が簡潔に行えます。
Example of Repetition on Solving Math
PROMPT: 海辺の主な港の税関職員は、輸入車の2台分のコンテナをチェックしており、各コンテナには5台の車が中に入っている。次の日、さらに多くのコンテナが運び込まれ、港の車両の数は全部で30となった。すべてのコンテナに5台の車が入っていたとすると、2日目にいくつのコンテナが輸入されたかを計算しなさい。回答は\boxed{}の中に書いてください。
RESPONSE:
まず、最初の日に税関職員がチェックしたコンテナは 2 台で、各コンテナには 5 台の車が入って いるので、最初の日に港にあった車の総数は
\[ 2 \times 5 = 10 \text{ 台} \]
です。
2 日目が終わった時点で港にある車の総数は 30 台です。したがって、2 日目 に新たに増えた車の台数は
\[ 30 - 10 = 20 \text{ 台} \]
です。
問題文の条件「すべてのコンテナに 5 台の車が入っていた」とあるので、2 日目に輸入されたコンテ ナの数は増えた車の台数を 1 コンテナあたりの車の台数で割ったものです。
\[ \frac{20}{5} = 4 \]
したがって、2 日目に輸入されたコンテナの数は 4 台 です。
\ [ \boxed{4} \]
Training Procedure
The model is trained with GRPO with a warm-start. We first generate a teacher dataset using gpt-oss-120b as a reference. Since the reasoning traces are in English, we translate them into Japanese using CAT-Translate-7b. We train the Swallow model using the generated synthesized dataset with full-parameter SFT.
Then, we run GRPO with a permissive reward model which gives partial rewards for being able to (1) follow the reasoning format, (2) generate reasoning trace and the main text in Japanese, and (3) answer the question in an instructed format. In this way, the model learns to follow the reasoning format and generate its reasoning trace in Japanese. Since this training phase focuses on learning the superficial format rather than reasoning competence itself, we use LoRA.
Finally, we train the model with GRPO using a strict reward model that gives a reward only if the model follows all format constraints and also generates the correct answer.
The training data consists of synthesized math and coding dataset generated by gpt-oss-120b. Most of the instructions used for the training are in English. We speculate this to be the reason why the model underperforms on Japanese benchmarks.
License
The model is licensed under the Apache 2.0 License.
Citation
@misc{jinnai2026costreasoningnonenglishlanguages,
title={Cost of Reasoning in non-English Languages: A Case Study on Japanese},
author={Yuu Jinnai},
year={2026},
eprint={2607.10114},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2607.10114},
}
Technical Specs
- Parameters: 8.0B
- Architecture: transformers
- Input Modalities: text
Hardware Requirements
- API-only (no local hardware needed)