- Published on
Co-organizing the WMT26 Terminology Shared Task
- Authors
- Name
- Frédéric Odermatt
After participating twice in the WMT shared task on Terminology Translation, I joined this year's edition as a co-organizer. Terminology is only a small part of the overall WMT programme, and my main goal was modest: help get proper, well-tested metric implementations in place. As a participant I had run into several shortcomings in earlier versions of the evaluation, and I wanted to help make this year's edition better in that regard.
Over the course of the project it turned into a bit more than that:
- A public leaderboard, live before the participants' paper deadline, so every team could see where they stood while writing.
- A full release of all metric implementations, including the term success rate with its lemmatized variant.
- A release of the gold reference dataset and of all competitor submissions.
I consider all of these core to open and reproducible science, and I'm quite proud of what we did here this year.
We had more participants again, 17 teams with 27 submissions across the two tracks, and already at least one system on the leaderboard that arrived after the deadline. I intend to keep the leaderboard open until 31.12.2026 for new systems and for updates to existing ones.
Because in the end, if the metric implementations are fully open, you can find the issues in your own system and improve it. You don't want competitors to score worse because some part of the evaluation was underspecified or opaque.
The findings paper is here: Findings of the WMT26 Terminology Translation Task: The Hard Part is Finding the Terms, Not Using Them.