@fnord "one of the most innovative companies in the world" -- brilliant. At the current discount that sounds like a wonderful opportunity -- innovation will drive future returns. I hope you are fully invested, with leverage.

@guyreading From first principles, this seems highly unlikely. This is circumstantially confirmed by the fact that, when translating ARC problems to sequences, the largest LLMs out there (not just GPT-3, but much larger ones as well) don't work at all.

@code_star Yes

What's the highest score you can do on CIFAR10 with no data augmentation and no external data?

I was happy that it was this easy to do. I didn't even have to make it a Model or a Layer (though I did make it a layer for a separate reason -- to make sure a subclassed layer could own a FeatureSpace and have it tracked for later saving)

I'm adding serialization & saving support to the new FeatureSpace structure data processing utility: https://github.com/keras-team/keras/pull/17355/files

@Fabio IMO you'd get better results for this use case by googling your keywords and reading random paragraphs from the first few articles that show up

Essay writing is about clarity of thought. You achieve clarity of thought by using fewer words, not by generating more.

It's hard to beat a plain SGD optimizer that has a finely tuned learning rate schedule.

Combination of heartbreak and morbid fascination, like watching a bus full of orphans undergo a slow-motion crash

Watching someone get radicalized is so incredibly sad. As everyone who has lost a friend or loved one to the far-right rabbit hole knows. Social media is not a force for good.

I just sent out this week's edition of my newsletter, where I talk about the loop of progress. Read it here: https://fchollet.substack.com/p/the-loop-of-progress

Just sent out this week's newsletter entry: https://fchollet.substack.com/p/ai-is-cognitive-automation-not-cognitive

Looking at kids teaches you a lot about adult psychology

We're pretty good at solving hard problems while sleeping or while doing something else. But for that effect to kick in, you need to have spent a while banging your head at the problem first. It's only payback for prior effort.

Twitter used to be fun. Or did I imagine it?

I'm looking forward to seeing Silicon Valley once again hustle to create the social app du jour, like in the good old days of 2010.

It will be a nice break from me-too generative AI startups.

Social networking services started in the mid-to-late 2000s had anomalous breadth and longevity due to the historical circumstances of their rise -- they had a big void to fill. The future is more likely to look like a set of increasingly niche and shorter-lived apps.

Posted this week's edition of my newsletter: https://fchollet.substack.com/p/the-machine-that-makes-the-thing

I'm rather enjoying writing these! The time pressure sucks, but it's a good forcing function.

There is still so little attention devoted to the only meaningful part of machine learning -- the mechanics of generalization.