Skip to content

Translation models

Which local model should you load in LM Studio? Every model here was run through the app's own translator — with the same prompt templates and settings the app uses — on the same test sentences, so the numbers compare directly. Quality is chrF (higher is better; differences of a point or two are within noise). How it's measured: How we benchmark.

Quality vs speed

Up is more accurate, left is faster (median time per subtitle line). Bubble size: file size.

By language

chrF for each language, best-translated first. Hover to compare the models at one language, and point at a line or a model's button to pick it out; click a model to show or hide it. * marks languages the app doesn't offer yet.

All results

Click a column to sort. Rings show chrF out of 100. Hover a row for notes.

Notes

  • ⚠ in the table marks results measured before the benchmark converted Chinese output to the chosen script, as the app has done since 1.5.0. It only affects English → Chinese (part of the "From English" average), and mostly for models that write Simplified Chinese when asked for Traditional — like Hy-MT2-1.8B, whose Traditional Chinese subtitles still come out right in the app.
  • MiLMMT (Xiaomi, Gemma licence) and Hy-MT2-30B-A3B write real colloquial Cantonese when asked; Hy-MT2-7B writes formal written Chinese, which is why Cantonese is a video language only.
  • Hy-MT2 and MiLMMT get their own prompt format and sampling settings automatically (Prompt templates); every other model gets general subtitle-translator instructions. Hybrid "thinking" models are asked not to reason, so they answer immediately.
  • On AMD GPUs keep LM Studio's Vulkan runtime; set GPU offload to max and flash attention on.
  • Older, informal tests: Qwen2.5-7B-Instruct Q8_0 (~0.40 s a line, 43 tok/s) made more literal mistakes; Qwen3.8-27B UD-Q3_K_XL (1.0–1.3 s, 18–25 tok/s) had good wording but left very little VRAM.

Raw results: benchmarks.json · Test sentences: FLORES (CC BY-SA 4.0) and the Pangeanic Cantonese–English corpus (CC BY 4.0) — see Credits.