#TREC2023ToT

Bhaskar Mitrabmitra
2023-08-17

is now accepting runs. Please submit your runs soon!

πŸ“„ Guidelines: trec-tot.github.io/guidelines
πŸƒπŸ½β€β™€οΈ Run submission form: ir.nist.gov/trecsubmit/tot.html
⏰ Run submission deadline: Aug 31

TREC Tip-of-the-Tongue TrackTREC_ToT@idf.social
2023-08-17

#TREC2023ToT is now accepting runs. Please submit your runs soon! #TRECToT #TREC2023

πŸ“„ Guidelines: trec-tot.github.io/guidelines
πŸƒπŸ½β€β™€οΈ Run submission form: ir.nist.gov/trecsubmit/tot.htm
⏰ Run submission deadline: Aug 31

TREC Tip-of-the-Tongue TrackTREC_ToT@idf.social
2023-07-24

We have released the test queries for TREC 2023 Tip-of-the-Tongue Track! #TRECToT #TREC2023ToT

πŸ“„ Guidelines: trec-tot.github.io/guidelines
⏰ Run submission deadline: Aug 31

Here we gooooooo!

Bhaskar Mitrabmitra
2023-07-19

We got GPT4 baseline for Track! πŸ™ŒπŸ½ Test queries will be released on July 24 and run submission deadline is Aug 31.

Get your runs ready, should be an exciting track this year! 😎

Guidelines and more info: trec-tot.github.io/guidelines

Results table showing 3 baseline runs:
* BM25 -- Dev DCG=0.1314
* Dense Retrieval (SBERT, Distilbert) -- Dev DCG=0.1627
* GPT-4 -- Dev DCG=0.2407
TREC Tip-of-the-Tongue TrackTREC_ToT@idf.social
2023-07-19

#TRECToT updates:

πŸ“° News: GPT4 baseline run significantly improves DCG over BM25 and dense retrieval. What approaches will you try at the track this year?

⏰ Reminder: Test queries will be released on July 24. Run submission deadline is Aug 31.

Guidelines: trec-tot.github.io/guidelines

#TREC2023ToT

Results table showing 3 baseline runs:
* BM25 -- Dev DCG=0.1314
* Dense Retrieval (SBERT, Distilbert) -- Dev DCG=0.1627
* GPT-4 -- Dev DCG=0.2407
TREC Tip-of-the-Tongue TrackTREC_ToT@idf.social
2023-07-06

⏰Gentle reminder: We'll release the TREC 2023 Tip-of-the-Tongue test queries in just over 2 weeks. We are super excited and looking forward to seeing all the creative approaches participants employ for this year's track. #TREC2023ToT #TRECToT

More info: trec-tot.github.io

Bhaskar Mitrabmitra
2023-06-28

We've got BASELINES for our Track! Lots of room to improve, should be an exciting track this year! πŸ™ŒπŸ½

mastodon.social/@TREC_ToT@idf.

Bhaskar Mitrabmitra
2023-05-18

I'm co-organizing 2 TREC tracks this year: Tip-of-the-Tongue & Deep Learning. Excited that 20+ groups have already registered for each. πŸ™ŒπŸ½

Please Register: ir.nist.gov/trecsubmit.open/ap

* TREC-ToT Track: Test queries to be released in July. Submission deadline: August 31st.
Guidelines: trec-tot.github.io

* TREC-DL Track: Test queries to be released in June. Submission deadline: July 31st.
Guidelines: microsoft.github.io/msmarco/TR

TREC Tip-of-the-Tongue TrackTREC_ToT@idf.social
2023-05-18

🚨 Announcement 🚨
TREC 2023 Tip-of-the-Tongue (TREC-ToT) Track guidelines are now online: trec-tot.github.io. Test queries will be released in July and August 31st is the submission deadline. #TRECToT #TREC2023ToT

To participate, please pre-register: ir.nist.gov/trecsubmit.open/ap.

Bhaskar Mitrabmitra
2022-11-15

Announcing brand new track for πŸ“£
(Please boost)

We hope Tip-of-the-Tongue (ToT) Track will be an interesting & fun IR task going beyond simple keyword-based searches with opportunities for both traditional and LLM-based approaches.

If you have any questions, please reach out to me or one of my track co-organizers.

Also, if you are at please drop by our poster & join the track planning session this Friday to learn more & provide feedback!

More detailsπŸ‘‡πŸ½

An academic poster on TREC 2023  Tip-of-the-Tongue (ToT) Track that states: "The Tip-of-the-Tongue (ToT) retrieval task is a known-item identification task where the searcher has previously experienced or consumed the item but cannot recall a reliable identifier".

Client Info

Server: https://mastodon.social
Version: 2025.04
Repository: https://github.com/cyevgeniy/lmst