Selected work
Efficient inference · with Huawei Research, 2026
CRE-Router, cost-aware serving across many LLMs
A two-stage routing and cascading framework. It clusters incoming queries
by difficulty to pick the most cost-effective model for each one, then
escalates only the outputs it is not confident in. The result nearly
matches the strongest single model on accuracy while running at
meaningfully lower latency.
RoutingCascadesLLM serving
Paper
Code
Speech · IWSLT 2025
Halving an 8B speech translation model
Compressed Qwen2-Audio to half its parameters and storage by combining
iterative layer pruning, 4-bit QLoRA, and knowledge distillation, while
holding on to almost all of the teacher model's quality.
50% fewer parameters
97 to 100% of teacher quality
Paper
Model compression · WMT 2025
Pruning a multilingual LLM without losing quality
Iterative layer pruning applied to Aya-Expanse 8B. Removing between a
fifth and nearly half of the model left translation quality almost intact
and made it substantially faster to serve, with a further speedup once
deployed through vLLM.
22 to 44% smaller
99% of COMET retained
4.3× faster serving
Paper
Open models · AfricaNLP 2026, Spotlight
AfriNLLB, faster open translation for African languages
Compressed translation models covering 15 language pairs in both
directions, released openly alongside a curated training set. They run
considerably faster than the NLLB-200 baseline at equal or better
quality.
30 directions
2.4× faster inference
1.6M open segments
Models and dataset