e509 — Maverick and Marbles

e509 with Michael and Michael – stories and discussion all around #AI, #LLMs, #llamas, generated #Quake, #grokking, #generalization and much more.

Podcast: Play in new window | Download (Duration: 32:10 — 44.8MB) | Embed

Subscribe: Apple Podcasts | Spotify | Amazon Music | Android | Podcast Index | Youtube Music | RSS | More

Share this:

• Click to share on Mastodon (Opens in new window)
• Click to share on LinkedIn (Opens in new window)
• Click to share on Facebook (Opens in new window)
• Click to share on Reddit (Opens in new window)
• Click to share on Tumblr (Opens in new window)

https://gamesatwork.biz/2025/04/14/e509-maverick-and-marbles/

When Dimensionality Hurts: The Role of #LLM Embedding Compression for Noisy Regression Tasks https://d.repec.org/n?u=RePEc:arx:papers:2502.02199&r=&r=cmp
"… suggest that the optimal dimensionality is dependent on the signal-to-noise ratio, exposing the necessity of feature compression in high noise environments. The implication of the result is that researchers should consider the #noise of a task when making decisions about the dimensionality of text.

… findings indicate that sentiment and emotion-based representations do not provide inherent advantages over learned latent features, implying that their previous success in similar tasks may be attributed to #regularisation effects rather than intrinsic informativeness."
#ML #autoencoders #Overfitting

'On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training', by Chen Liu, Zhichao Huang, Mathieu Salzmann, Tong Zhang, Sabine Süsstrunk.

http://jmlr.org/papers/v25/22-0950.html

#adversarial #overfitting #robustness

"Three married couples. Aren't we just boring now?" joked Mia.
"You're frowning," Ari told Jin.
"Mia's #overfitting. She's too used to seeing threats to relax to realise Tom's mum is fond of her."
"Of course she is, Tom loves Mia. How could his mum not love her too?" #vss365

1/

Recent commentary [1]:
escalating concern over the use of the more powerful #chatbots when they are used to go beyond the #knowledge of the human expert who uses them, rather than for simply pre-processing in a controlled way within the domain of human-expert knowledge.

1. ⁠What is often called "hallucination/confabulation” (i.e. severe #extrapolation #uncertainty and #overfitting by the chatbot model) is apparently becoming increasingly realistic with a declining human ability to detect it

I’m listening to #MITtechReview [PODCAST] Large language models can do jaw-dropping things. But nobody knows exactly why. (7 Aug 2024, 26 min)
https://pca.st/episode/527cdfed-17ed-4e59-8530-f9830eea78a7

#edtechSR #MediaLit #AI #lanhuage #learning #DeepLearning #magic #alchemy #OverFitting

AI image created by Wes Fryer with Ideogram:
https://ideogram.ai/g/FItzSD2BQJiJtpb7Ph0Ybw/3

"AI in the era of large language models [#LLMs] appears to defy textbook statistics. The most powerful models today are vast, with up to a trillion parameters (the values in a model that get adjusted during training). But statistics says that as models get bigger, they should first improve in performance but then get worse. This is because of something called #overfitting."
https://www.technologyreview.com/2024/03/04/1089403/large-language-models-amazing-but-nobody-knows-why/

🔴 Eigensolutions: composability as the antidote to overfit
by Lea Verou
@LeaVerou @leaverou
#Product #DesignThinking #CreatorTools #ProductManagement #Eigensolutions #Overfitting

https://lea.verou.me/blog/2023/eigensolutions/

'Benign Overfitting of Constant-Stepsize SGD for Linear Regression', by Difan Zou, Jingfeng Wu, Vladimir Braverman, Quanquan Gu, Sham M. Kakade.

http://jmlr.org/papers/v24/21-1297.html

#overfitting #overparameterized #sgd

🤖💡 Ever struggled with overfitting in machine learning? It can lead to poor performance and inaccurate predictions. Learn more here 👉 https://ak-codes.com/overfitting/ #MachineLearning #Overfitting

One question for the #MachineLearning people: what approach do you use to determine if a decision trees or a random forest approach should work better? Do you simply try both approaches and use whatever seems to work better?

According to what I read, decision trees are more prone to overfitting, while random forest is a more complex approach. Which means little to me 😅

#ml #DecisionTrees #RandomForest #Overfitting

Early Stopping for Deep Image Prior

https://openreview.net/forum?id=231ZzrLC8X

#imaging #overfitting #prior

Understand basic principles of underfitting and overfitting - by @dimid_ml

https://towardsdatascience.com/overfitting-and-underfitting-principles-ea8964d9c45c

#overfitting #underfitting #DataScience #MachineLearning #AI

Logistic-Normal Likelihoods for Heteroscedastic Label Noise

Erik Englesson, Amir Mehrpanah, Hossein Azizpour

Action editor: Bo Han.

https://openreview.net/forum?id=7wA65zL3B3

#label #classification #overfitting

Label Noise-Robust Learning using a Confidence-Based Sieving Strategy

https://openreview.net/forum?id=3taIQG4C7H

#label #labels #overfitting

Learning Augmentation Distributions using Transformed Risk Minimization

Evangelos Chatzipantazis, Stefanos Pertigkiozoglou, Kostas Daniilidis, Edgar Dobriban

Action editor: Andriy Mnih.

https://openreview.net/forum?id=LRYtNj8Xw0

#augmentation #augmentations #overfitting

🤖𝐂𝐡𝐚𝐭𝐆𝐏𝐓 - 𝐃𝐚𝐬 𝐩𝐞𝐫𝐟𝐞𝐤𝐭𝐞 𝐕𝐞𝐫𝐬𝐩𝐫𝐞𝐜𝐡𝐞𝐧🤖

𝑱𝒆𝒕𝒛𝒕 𝒂𝒍𝒔 𝒆𝑩𝒐𝒐𝒌 𝒆𝒓𝒉𝒂𝒆𝒍𝒕𝒍𝒊𝒄𝒉 𝒃𝒆𝒊 𝑨𝒑𝒑𝒍𝒆𝑩𝒐𝒐𝒌𝒔 𝒖𝒏𝒅 𝑨𝒎𝒂𝒛𝒐𝒏 𝑲𝒊𝒏𝒅𝒍𝒆!

Themen:

⁉️ #Interdependenz ⁉️ #Integrität ⁉️ #Authentizität ⁉️ #Glaubwürdigkeit ⁉️ #Identität ⁉️ #Overfitting ⁉️ #Verfolgungswahn ⁉️ #Ursprung ⁉️ #Kontext ⁉️ Souffleuse ⁉️ #Wert ⁉️ #IntellektuellesEigentum ⁉️ #Bequemlichkeit ⁉️ #Knast ⁉️ #Wahl ⁉️ #Vertrauen ⁉️ #Vertraulichkeit ⁉️ #Kontrollverlust ⁉️ #Sprache …

𝙰𝚞𝚝𝚘𝚛: 𝙹ü𝚛𝚐𝚎𝚗 𝚂𝚌𝚑𝚞𝚕𝚣𝚎
𝚅𝚎𝚛𝚕𝚊𝚐: 𝚒-𝚟𝚊𝚕 - 𝚒𝚗 𝚝𝚑𝚎 "𝚒" 𝚘𝚏 𝚋𝚒𝚣
𝙸𝚂𝙱𝙽: 𝟿𝟽𝟾-𝟹-𝟿𝟷𝟶𝟿𝟷𝟸-𝟶𝟶-𝟽

https://jschulze.com/projects/ChatGPT/

Catastrophic overfitting can be induced with discriminative non-robust features

Guillermo Ortiz-Jimenez, Pau de Jorge, Amartya Sanyal et al.

Action editor: Jakub Tomczak.

https://openreview.net/forum?id=10hCbu70Sr

#overfitting #adversarial #robust

"#AI #art is leaking into the mainstream in the form of #stablediffusion and #Lensa, but there are serious #ethical concerns with this unregulated tech. I'm NOT anti AI, in fact, I believe AI can be of immense benefit to us in the future. But the ethics of AI in its current state MUST be talked about, in order to steer this tech in the right direction."

https://www.youtube.com/watch?v=5Viy3Cu3DLk

#copyright #intellectualproperty #overfitting #LLM

Logistic-Normal Likelihoods for Heteroscedastic Label Noise

https://openreview.net/forum?id=7wA65zL3B3

#label #classification #overfitting