Using LLMs as evaluators looks like a very interesting and promising direction, enabling simpler automatic post-editing pipelines. For those interested in fine-grained MT evaluation and APE, I recommend checking out this paper by Lu et al. (2024): https://arxiv.org/abs/2409.14335
#MT #postediting #NLP #AI #evaluation #LLM

Interesting take on gender bias in LLMs and retrievers: gender information is highly extractable but gender bias in RAG tasks is likely not due to the retrievers

MultiContrievers: Analysis of Dense Retrieval Representations (Goldfarb-Tarrant et al., 2024) http://arxiv.org/abs/2402.15925

The paper is finally available on arXiv! 🎊

➡️ https://arxiv.org/abs/2405.08477

So happy to announce that our paper 'Enhancing Gender-Inclusive Machine Translation with Neomorphemes and Large Language Models' has been accepted at EAMT 2024! 🎉

Amazing co-authors: Beatrice Savoldi, Matteo Negri, Luisa Bentivogli

See you in Sheffield! 🧐
#MT #AI #LLMs #EAMT

@matthewmaybe @dippedrusk
Wikipedia adopted a gender-neutral language policy a few years ago https://en.wikipedia.org/wiki/Wikipedia:Gender-neutral_language_in_Wikipedia_policies this possibly explains why that happens. Also, looking at the paper, the authors refer to the 'Wizard' using singular they as well. This is fascinating, if you have interesting outputs you want to share feel free to do so :) Thank you!

Our pick of the week by @apierg: "Robust Pronoun Use Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased?" by @dippedrusk, @lauscher, et al., 2024.

#pickoftheweek #LLM #bias #reasoning #NLP #NLProc

@matthewmaybe @dippedrusk that’s quite interesting. May I ask what kind of data are you fine-tuning it on?

Engaging study by @dippedrusk et al. on pronoun use on a wide range of LLMs: http://arxiv.org/abs/2404.03134

Interestingly, encoder-decoder models are more accurate and more robust to distractions.
Hot take: maybe we shouldn't be using decoder-only LLMs all the time and for everything 🤔 #NLP #AI #aiethics

Ask ChatGPT to pick a number between 1 and 100 - which does it pick? (by Leniolabs)

Our pick of the week by @apierg: "Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation" by Xu et al.

https://arxiv.org/abs/2401.08417

#pickoftheweek #MT #LLM

Here's a noteworthy study, focusing on one of the most relevant questions in MT: is the 'gold standard'... gold? Or is it gilded?

📄 Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation
by Haoran Xu et al.

🔗 http://arxiv.org/abs/2401.08417

#MT #LLM

Our pick of the week by @mgaido91: "Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models" by @JHPang_r, @Fanghua_Ye, @wangly0229, @ShumingShi, @tuzhaopeng et al., 2023.

https://arxiv.org/abs/2401.08350

#translation #MT #pickoftheweek #LLM #languagemodel

Our pick of the week by @sarapapi: "SALMONN: Towards Generic Hearing Abilities for Large Language Models" by Tang* and Yu* et al., ICLR 2024.

https://arxiv.org/abs/2310.13289

#pickoftheweek #LLM #speech #language #languagemodel #whisper #beats #vicuna #translation #AI #

Our pick of the week by Dennis Fucci: "What does the EU AI Act mean for Research?", a blog post by Dirk Hovy: https://iep.unibocconi.eu/what-does-eu-ai-act-mean-research

#AI #Research #AIAct

Our pick of the week by Beatrice Savoldi: "Mind your Language (Model): Fact-Checking LLMs and their Role in NLP Research and Practice" by @SashaMTL and @annargrs, 2023.

https://arxiv.org/abs/2308.07120

#NLP #NLProc #LLM #FactCheck #research

Our pick of the week by @apierg: '"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters' by Wan et al. http://arxiv.org/abs/2310.09219 (Findings EMNLP 2023).
#NLP #NLProc #LLM #gender #genderbias #EMNLP2023

To no one's surprise, LLMs are gender biased. If you're interested in investigating how they reiterate gender stereotypes check out “Kelly is a Warm Person, Joseph is a Role Model”: Gender Biases in LLM-Generated Reference Letters" by Wan et al.: https://arxiv.org/abs/2310.09219

Our pick of the week by @mgaido91: "On decoder-only architecture for speech-to-text and large language model integration" by Wu et al., (2023).

https://arxiv.org/abs/2307.03917

#NLP #ASR #ST #speech #LLM #integration

Our pick of the week by Dennis Fucci: "Explaining Speech Classification Models via Word-Level Audio Segments and Paralinguistic Features" by Pastor et al., 2023.

https://arxiv.org/pdf/2309.07733.pdf

#NLP #NLProc #speech #audio #classification

Our pick of the week by @apierg: "Prompt-Driven Neural Machine Translation" by Li et al., Findings ACL 2022.

https://aclanthology.org/2022.findings-acl.203

#machine #translation #MT #NLP #NLProc #ACL #computational #linguistics #prompt