Preface¶

When deploying models on RDK series boards, the training side is often in FP32, while the board-side BPU typically requires an INT8 fixed-point model. The OpenExplorer toolchain, hb_compile, and YAML configuration items are numerous; pitfalls are common in areas such as calibration data, board-side stride alignment, and post-processing. If one has to flip back and forth between documentation and forum posts every time, it is difficult to establish a stable workflow for conversational development.

The RDK Model Quantization Tool (RDK S100) on SkillHub (Coordinate @user_a793e4ac/rdk-model-quantization, version 1.0.0, community source) packages the scripts, configuration templates, and troubleshooting guides for PyTorch/ONNX to BPU (.hbm) conversion into a universal SKILL.md format. This allows Agents like Cursor to guide execution step-by-step after installation. The documentation within the Skill package is primarily aimed at RDK S100P / S600 and corresponding march configurations, coexisting with the “RDK S100” in the SkillHub title. When using it, defer to the package’s internal README.md and configurations.

What is this¶

In one sentence: A Skill for model quantization and deployment workflows targeting Horizon BPUs, maintained by the community publisher user_a793e4ac (SkillHub page displays the author as “地瓜橙”), categorized under dev-programming, and does not require an API Key (requires_api_key=false).

It solves the problem of completing ONNX export, calibration data generation, quantization compilation, and accuracy verification within a host Docker toolchain environment, while providing handling strategies for board-side deployment and common accuracy issues. The practical case study uses Paraformer-large speech recognition as the main thread, with the directory containing executable scripts like export_onnx.py and quantize_model.py, as well as templates/encoder_config.template.yaml.

Core Features¶

The following capabilities are derived from the SKILL.md, README.md, and QUICKSTART.md inside the Skill package and can be reproduced via the specified paths.

1. Five-step quantization pipeline: Export ONNX → Generate calibration data → Quantize via YAML → Verify Cosine similarity → scp to transfer the .hbm to the board.
2. Supporting scripts (scripts/):
   • export_onnx.py: Convert PyTorch to ONNX;
   • generate_calibration_data.py: Generate the PTQ calibration set;
   • quantize_model.py: Call the toolchain to execute quantization;
   • verify_quantization.py: Compare the original model with the quantized result;
   • correction_factor_cleanup.py: Post-processing for extra tokens at the tail of the Predictor.
3. Configuration and Documentation: templates/encoder_config.template.yaml; docs/best_practices.md, docs/troubleshooting.md; examples/simple_quantization.py, examples/paraformer_quantization.md.
4. Key technical notes (Summarized in the package, with Paraformer measured data): Stride-aware input copying, comparison of calibration type default with other methods, correction_factor post-processing. The documentation records metrics such as Cosine scores and speedup ratios for Encoder/Predictor for you to compare against your own model verification, rather than guaranteeing universal conclusions.

Installation and Activation¶

Install SkillHub CLI¶

If it is not already installed on your local machine, you can check the version first:

command -v skillhub && skillhub --version

If not installed, you can follow the official SkillHub instructions to execute (complete installation example):

curl -fsSL https://skillhub-1388575217.cos.ap-guangzhou.myqcloud.com/install/install.sh | bash

Install this Skill¶

The method provided on the SkillHub page is: Read the SkillHub Installation Guide and install the coordinate @user_a793e4ac/rdk-model-quantization.

When installing via CLI, you need to use --dir to point to the current Agent’s skills directory; otherwise, it defaults to ./skills/ and may not be recognized. Common paths are as follows:

In Cursor, you can execute:

skillhub install rdk-model-quantization --namespace user_a793e4ac --dir ~/.cursor/skills/

After installation is complete, restart or refresh the Agent to load the Skill. Official API Entry: https://api.skillhub.cn/user_a793e4ac/rdk-model-quantization; Display Page: https://www.skillhub.cn/skills/user_a793e4ac/rdk-model-quantization.

Typical Usage¶

The following introduces the Quick Start and host-side environment conventions within the Skill package; quantization must be performed within the Horizon OE Docker.

1. Prepare OE Docker Environment¶

The Skill documentation specifies the image ai_toolchain_ubuntu_22_s100_s600_cpu:v3.7.0 (you need to prepare the corresponding tar and docker load it yourself):

docker load -i ai_toolchain_ubuntu_22_s100_s600_cpu_v3.7.0.tar

docker run -it --rm \
  -v $(pwd):/workspace \
  -w /workspace \
  ai_toolchain_ubuntu_22_s100_s600_cpu:v3.7.0 \
  bash

pip install -r requirements.txt

2. Export ONNX¶

python3 scripts/export_onnx.py \
    --model-path models/your_model.pt \
    --output-path models/your_model.onnx \
    --input-shape 1,300,560

3. Generate Calibration Data¶

python3 scripts/generate_calibration_data.py \
    --data-dir audio_data/ \
    --output-dir calibration_data/ \
    --num-samples 50

The quick start suggests at least about 50 representative samples; README.md also mentions approximately 50–100 samples.

4. Write Quantization Config and Compile¶

Example from QUICKSTART.md (Note march: S100P uses nash-m, S600 uses nash-p):

model_parameters:
  onnx_model: 'models/your_model.onnx'
  march: "nash-m"
  output_dir: 'output/'

calibration_parameters:
  cal_data_dir: 'calibration_data/'
  calibration_type: 'default'
  num_samples: 50

compiler_parameters:
  optimize_level: 'O2'

Execute quantization:

python3 scripts/quantize_model.py \
    --config configs/my_model_config.yaml \
    --output-dir output/

You can also copy from the template: cp templates/encoder_config.template.yaml configs/encoder_config.yaml and then edit it.

5. Verify and Deploy¶

python3 scripts/verify_quantization.py \
    --original-model models/your_model.onnx \
    --quantized-model output/your_model.hbm \
    --test-data calibration_data/ \
    --max-samples 5

The quick start expects an average Cosine similarity of about 0.959 (Cosine ≥ 0.95 is considered good, but defer to your local output).

scp output/your_model.hbm root@<board-ip>:/root/models/

In an Agent conversation, you can provide the above steps along with local model paths and board IPs to the installed Skill, allowing it to troubleshoot low Cosine scores or board-side inference anomalies according to docs/troubleshooting.md.

Applicable Scenarios and Notes¶

Who is it for

• Developers already deploying INT8 models on RDK S100P / S600 who need PTQ workflow scripts and configuration templates;
• Those performing BPU quantization on Transformer-based models (the documentation uses Paraformer, BERT, etc., as examples) and are willing to prepare calibration data according to the Skill’s best practices;
• Those who want to use a fixed workflow in environments like Cursor to reduce the cost of repeatedly checking documentation.

Limitations and Notes

• You need to have your own OE toolchain Docker image and development board; the Skill does not replace the official toolchain installation and authorization process.
• README.md requires that the model can already be exported to ONNX, the input shape is relatively fixed, and the size is less than about 2GB; the host must be x86_64, and the board must be the RDK model listed in the documentation.
• SkillHub Metadata: Rating 100000, Downloads 854 (statistics at the time of scraping), which may change over time relative to the page display.
• Performance and Cosine figures in the package come from the Paraformer-large case; when migrating to other architectures, you must re-run verify_quantization.py and business metrics.
• If the Cosine score drops significantly on the board side, the documentation emphasizes checking stride alignment and whether feature extraction (CMVN/LFR, etc.) is consistent with training; speech recognition scenarios also need to pay attention to correction_factor post-processing.

Conclusion¶

RDK Model Quantization Tool (RDK S100) strings together BPU quantization from environment, scripts, to troubleshooting documentation into an installable Skill, suitable for advancing step-by-step within an Agent session. Please refer to SkillHub for verifying versions, installation, and community information:

• API: https://api.skillhub.cn/user_a793e4ac/rdk-model-quantization
• Page: https://www.skillhub.cn/skills/user_a793e4ac/rdk-model-quantization