Just finished the 3ed chapter of the free "Elements of AI“ course, that's half of it, and I am really impressed with the content! It’s a fantastic course for people interested in learning about AI. The course is not crazy difficult, but some of the challenges are a little tricky like the math with the Bayes rule.
It 's for people starting out with AI and I can highly recommend it!
![A screenshot from the course reading:
Elements of AI
Course overview
Neural networks
Advanced neural network techniques
Menu
The Rise of Large Language Models (LLMs)
As mentioned above, convolutional neural networks (CNNs) reduce the number of learnable weights in a neural network so that the amount of training data required to learn all of them doesn't grow astronomically large as we keep building bigger and bigger networks. Another architectural innovation, besides the idea of a CNN, that currently powers many state-of-the-art deep learning models is called attention.
Attention mechanisms were originally introduced for machine translation where they can selectively focus the attention of the model to certain words in the input text when generating a particular word in the output. This way the model doesn't have to pay attention to all of the input at the same time, which greatly simplifies the learning task. Attention mechanisms were soon found to be extremely useful not only in machine translation.
[continued...]](https://files.mastodon.social/cache/media_attachments/files/110/886/668/013/766/009/small/8c535e5a864b7a09.jpeg)
