EMNLP 2025 глазами аналитика из Яндекса: мировые тренды и наши решения

Всем привет! Меня зовут Катя Еникеева, я руковожу командой аналитики перевода в Яндексе. Мы занимаемся оценкой качества машинного перевода — моделей, которые работают в Яндекс Переводчике, Браузере, Поиске и во множестве других сервисов. Качество перевода можно измерять по‑разному, но можно выделить два основных направления: экспертная разметка и автоматические метрики. В последние годы автометрики всё чаще строятся поверх LLM: фактически это отдельный пайплайн, который анализирует исходный текст и полученный перевод. Поэтому нас интересует не только способность моделей переводить, но и их умение анализировать качество перевода, что может быть заметно сложнее. Под катом вас ждёт обзор самых интересных решений, представленных на конференции EMNLP 2025. Наша команда перевода приехала на EMNLP 2025 не только слушать, но и рассказывать о своей работе. В этом году у нас приняли две статьи: одну — в Findings основной конференции, вторую — на WMT. О них я тоже подробно расскажу.

https://habr.com/ru/companies/yandex/articles/991144/

#яндекс #машинный_перевод #конференции #ml #языки #перевод #llm #emnlp

If you are at EMLP'25, I will be presenting shortly our work on LLM disinformation in low-resource languages, using Walliserdeutsch Swiss German dialect as an example.

TLDR; We show that a competent attacker can wreak havoc with model that have no declared performance in Germanic languages let alone WD. Since WD shares a lot of features with other low-resource languages, this means LLM disinformation in low-resource languages is much closer than thought.

#Science #EMNLP #Disinformation

And consider following the authors Sheng Lu, Ilia Kuznetsov, and Iryna Gurevych (all Ubiquitous Knowledge Processing (UKP) Lab/Technische Universität Darmstadt).

See you at the #EMNLP conference in Suzhou 🏯

(3/3)

#EMNLP2025 #UKPLab #PeerReview #LLM #AIResearch #NLProc

📊 𝗦𝗘𝗘𝗘𝗗 outperforms #GPT-4o and #Phi-4 by up to +𝟴 𝗽𝗽 across multiple datasets.

📄 𝗣𝗮𝗽𝗲𝗿: https://www.arxiv.org/abs/2509.10833
💻 𝗖𝗼𝗱𝗲: https://github.com/UKPLab/emnlp2025-automatic-error-discovery
🔗 𝗣𝗿𝗼𝗷𝗲𝗰𝘁: https://ukplab.github.io/emnlp2025-automatic-error-discovery/

Be sure to follow the authors: Dominic Petrak, Thy Thy Tran, and Iryna Gurevych from Ubiquitous Knowledge Processing (UKP) Lab/Technische Universität Darmstadt.

See you at the #EMNLP in Suzhou!

(2/2)

#NLProc #ConversationalAI #Agents #EMNLP2025

And consider following the authors Haishuo Fang (UKP Lab), Xiaodan Zhu (Department of Electrical and Computer Engineering, Smith Engineering at Queen's University), and Iryna Gurevych (UKP Lab/ATHENE Center) if you are interested in more information or an exchange of ideas.

See you at the #EMNLP conference in Suzhou 🏯

(3/3)

#NLProc #AI #EMNLP2025 #LLMAgent

𝗧𝗵𝗿𝗼𝘄𝗯𝗮𝗰𝗸 𝘁𝗼 Hashtag#EMNLP2024 𝗶𝗻 𝘀𝘂𝗻𝗻𝘆 𝗠𝗶𝗮𝗺𝗶🌞
The UKP Lab had an amazing time at this year’s #EMNLP in Florida🌴!

Our team presented a total of 13 papers, including 11 in the Main track and 2 in the Findings track, showcasing our latest research to a vibrant international audience.
(1/🧵)

RT @aroraakhilcs: Disappointed to not be at #EMNLP owing to a dislocated shoulder 😢
@DebjitPaul2 will present our poster on Multilingual E…

via https://twitter.com/WikiResearch/status/1856126524354900436

⏳ Countdown to #EMNLP2024 in Florida🌴
The #EMNLP in Miami begins on 12 November and starting tomorrow, we will be posting UKP Lab's contributions to the conference daily.

Follow along for in-depth insights and key contributions to the field of NLP innovations.

❗ #CfP EXTENDED to 27th Sep

Did your #NLP #RAG #NLU paper get rejected from @emnlpmeeting #EMNLP?

Consider submitting instead to #ALTA2024, affiliated with @aclmeeting.

📅 Submission deadline for short/long papers, presentation abstracts and industry demonstrations: 27 Sep 2024, 23:59 Anywhere on Earth, UTC -12

📅 Author notification: 29 Oct 2024

📅 Camera ready: 12 Nov 2024

📅 2nd-4th Dec 2024, #ANU, #Canberra, Australia.

#Hybrid in person and online.

https://alta2024.alta.asn.au/calls/papers

The 4th International Conference on Natural Language Processing for Digital Humanities (NLP4DH) will be co-located with EMNLP in Miami, USA, on November 15-16, 2024.
Topics include text analysis, dataset creation, cultural heritage collection research, and more. Submission deadline: September 1, 2024. Proceedings will be published in the ACL Anthology.

More info: https://www.nlp4dh.com/nlp4dh-2024

#NLP4DH #EMNLP #DigitalHumanities #NLP #Research #Conference2024

Anyone know when EMNLP'24 (i.e ACL ARR June cycle) will open for submissions?

https://2024.emnlp.org/calls/main_conference_papers/#paper-submission-information

#EMNLP2024 #EMNLP #nlp #nlproc @emnlpmeeting

@tante guter Punkt, zumindest das vorvorletzte (#EMNLP ) und das vorletzte (#NeurIPS ) scheinen Conference Paper zu sein, beide wurden anscheinend auch accepted.

Ich bin jetzt kein Experte, aber zumindest gibts bei beiden ein Review, das über Annahmen entscheidet: https://neurips.cc/Conferences/2023/ReviewerGuidelines und https://balhafni.github.io/EMNLP_2022/blog/paper-review-process/

Sieht auf den ersten Blick solide aus und schonmal besser als irgendwelche Predatory Journals (auch wenn es „nur“ Konferenzpublikationen sind).

One of a big takeaway for me for #EMNLP this year: the google T-series models (T5, T0, mT5) are really great. They seem to perform quite well on several benchmarks, have multilingual versions and the availability of very small up to large sizes really makes them great research models.

👶BabyLM will be back, but what did we learn?
the best papers from this year🍼

We know one mechanism that learns from 100M words (us), what are the main boosts to reach this in an LLM?

#EMNLP #babyLM #LLMs #ML #machinelearning #NLProc #NLP

Two works from
@dieuwke
on stability or consistency of outputs\metrics or if you want (and really, I want) reliability

Datasets for compositional generalizations do not agree with each other. It means that different models are good at different things. But that the metrics don't measure what we thought...

@Adinawilliams
@_dieuwke_

#emnlp #EMNLP2023

I am going to share some live summaries from #EMNLP #NEURIPS #CONLL conferences

Hope I am not too spammy (will pass soon)

And tell me if I can make it more useful for people not participating in any way
#machinelearning #LLM #LLMs #NLP #NLProc #ML #CV #data

Nizar Habash (https://www.nizarhabash.com/) shared the following which might be relevant to some in our community:

For students interested in attending #EMNLP 2023 (and its associated workshops & conferences, such as #ArabicNLP 2023), consider the Student #Volunteer program and the D&I #Subsidies program.

https://2023.emnlp.org/volunteers
https://2023.emnlp.org/calls/subsidies

Application deadline: October 20, 2023

I was of three minds,
Like a paper
On which there are three reviews.

#NeurIPS #EMNLP #AAAI

As expected, reviewing for #EMNLP this year is as close to academic hell as it gets, due the amount of people who never heard about #LLMs before this year deciding they have revolutionary ideas that need to be published.

Without having a slightest idea of what they are talking about.

Abstract Visual Reasoning with Tangram Shapes

https://arxiv.org/abs/2211.16492

#NLProc #NLP #IR #ML #AI #EMNLP #ACL #Conference #Paper