By Benedict Neo
So, you want to learn AI? But you don’t know how or where to get started?
Top-down approach
This curriculum follows a top-down approach — code first, theory later.
Table of contents
Mathematics
Tools
∘ Python
∘ PyTorchMachine Learning
∘ Write from Scratch
∘ Compete
∘ Do side projects
∘ Deploy them
∘ SupplementaryDeep Learning
∘ Fast.ai
∘ Do more competitions
∘ Implement papers
∘ Computer Vision
∘ NLPLarge Language Models
∘ Watch Neural Networks: Zero to Hero
∘ Free LLM boot camp
∘ Build with LLMs
∘ Participate in hackathons
∘ Read papers
∘ Write Transformers from scratch
∘ Some good blogs
∘ Watch Umar Jamil
∘ Learn how to run open-source models
∘ Prompt Engineering
∘ Fine-tuning LLMs
∘ RAGHow to stay updated
Other curriculums/listicles you may find useful
Mathematics
Machine learning relies heavily on three pillars of mathematics: linear algebra, calculus, probability, and statistics. Each plays a unique role in enabling algorithms to function effectively.
∘ Linear Algebra: the mathematical toolkit for data representation and manipulation, where matrices and vectors form the language for algorithms to interpret and process information
∘ Calculus: The engine for optimization in machine learning, enabling algorithms to learn and improve by understanding gradients and rates of change.
∘ Probability and Statistics: The foundation for decision-making under uncertainty, allowing algorithms to predict outcomes and learn from data through models of randomness and variability.
This is a great series on Math for ML from a programmer’s perspective: Math for Machine Learning by Weights & Biases (code)
If you want a code-first approach to Linear Algebra, do Computational Linear Algebra (video, code) by the creators of fast.ai.
Read Introduction to Linear Algebra for Applied Machine Learning with Python alongside the course.
If you want something more traditional, look at Imperial College London lectures — Linear Algebra & Multivariate Calculus.
Watch 3Blue1Brown’s Essence of Linear Algebra and Essence of Calculus.
Watch Statistics Fundamentals by StatQuest for statistics
Supplementary
Book: Mathematics for Machine Learning
Paper: The Matrix Calculus You Need For Deep Learning
Tools
Python
Beginners start here: Practical Python Programming.
If you’re already comfortable with Python, do this Advanced Python Mastery
They’re both great courses by David Beazley, author of Python Cookbook.
After that, watch some of James Powell’s talks.
Read Python Design Patterns.
Supplementary
Book: Fluent Python, 2nd Edition (code)
Podcasts: Real Python & Talk Python
PyTorch
Watch PyTorch Tutorials by Aladdin Persson
The PyTorch website is a great place to be.
∘ FAQ page
Test your knowledge with some puzzles
∘ srush/Tensor-Puzzles: Solve puzzles. Improve your PyTorch
Supplementary
∘ Book: Programming PyTorch for Deep Learning
Machine Learning
Read the 100-page ML book.
Write from Scratch
While you’re reading, write the algorithms from scratch.
Look at the repositories below
∘ eriklindernoren/ML-From-Scratch
∘ trekhleb/homemade-machine-learning
If you want a challenge, write PyTorch from scratch by following this course.
∘ MiniTorch: A DIY Course on Machine Learning Engineering (videos, code)
Compete
Apply what you learn in competitions.
∘ Join ML competitions on platforms like bitgrit and Kaggle; find more in this article.
∘ Look at past winning solutions and study them
Do side projects
Read Getting machine learning to production by Vicki Boykis
She also wrote about what she learned building Viberary, a semantic search for books.
Get a dataset and build a model (i.e., use earthaccess to get NASA Earth data).
Create a UI with streamlit and share it on Twitter.
Deploy them
Get the models in production. Track your experiments. Learn how to monitor models. Experience data and model drift firsthand.
Here are some excellent resources
∘ DataTalksClub/mlops-zoomcamp: Free MLOps course
∘ chiphuyen/machine-learning-systems-design
∘ Evidently AI — ML system design: 300 case studies
∘ stas00/ml-engineering: Machine Learning Engineering Online Book
Supplementary
∘ Machine Learning with PyTorch and Scikit-Learn (code)
∘ [1811.12808] Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning
∘ Introduction to Machine Learning Interviews Book · MLIB
Deep Learning
If you want top-down, start with fast.ai.
Fast.ai
∘ fast.ai (part1, part2) + W&B Study Group
Liked fast.ai? Check out Full Stack Deep Learning.
If you want a more comprehensive, traditional course, check out UNIGE 14×050 — Deep Learning by François Fleuret.
If you need to reach for theory at some point, these are great books.
∘ Dive into Deep Learning (has code examples in PyTorch, NumPy/MXNet, JAX, and TensorFlow)
∘ Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
∘ Neural networks and deep learning
∘ Understanding Deep Learning (with hands-on notebooks)
Read The Little Book of Deep Learning on your phone instead of scrolling Twitter.
Read these while your neural networks are converging.
∘ A Recipe for Training Neural Networks
∘ Deep Neural Nets: 33 years ago and 33 years from now
Do more competitions
- PlantTraits2024 — FGVC11 | Kaggle (computer vision)
Implement papers
Check out labml.ai Annotated PyTorch Paper Implementations
Papers with Code is a great resource; here’s BERT explained on their website.
Below are some resources for the specializations within Deep Learning:
Computer Vision
A lot of people recommend CS231n: Deep Learning for Computer Vision. It’s challenging but worth it if you get through it.
Reinforcement Learning
For RL, these two are great:
∘ Spinning Up in Deep RL by OpenAI
∘ 🤗 Deep Reinforcement Learning Course — Hugging Face
NLP
Another great Stanford course, CS 224N | Natural Language Processing with Deep Learning
Learn Hugging Face: Hugging Face NLP Course
Check out this Super Duper NLP Repo
Good articles and breakdowns
∘ BERT Research — Ep. 1 — Key Concepts & Sources · Chris McCormick
∘ The Illustrated Word2vec — Jay Alammar
∘ The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning
∘ Understanding LSTM Networks — colah’s blog
∘ PyTorch RNN from Scratch — Jake Tae
Supplementary
∘ Natural Language Processing with Transformers Book
Large Language Models
First, watch [1hr Talk] Intro to Large Language Models by Andrej.
Then Large Language Models in Five Formulas, by Alexander Rush — Cornell Tech
Watch Neural Networks: Zero to Hero
It starts with explaining and coding backpropagation from scratch and ends with writing GPT from scratch.
Neural Networks: Zero To Hero by Andrej Karpathy
He just released a new video → Let’s build the GPT Tokenizer
You can also look at GPT in 60 Lines of NumPy | Jay Mody while you’re at it.
Free LLM boot camp
A paid LLM Bootcamp released for free by Full Stack Deep Learning.
It teaches prompt engineering, LLMOps, UX for LLMs, and how to launch an LLM app in an hour.
Now that you’re itching to build after this boot camp.
Build with LLMs
Want to build apps with LLMs?
Watch Application Development using Large Language Models
by Andrew Ng
Read Building LLM applications for production by Huyen Chip
As well as Patterns for Building LLM-based Systems & Products by Eugene Yan
Refer to the OpenAI Cookbook for recipes.
Use Vercel AI templates to get started.
Participate in hackathons
lablab.ai has new AI hackathons every week. Let me know if you want to team up!
If you want to go deeper into the theory and understand how everything works:
Read papers
A great article by Sebastian Raschka on Understanding Large Language Models, where he lists some papers you should read.
He also recently published another article with papers you should read in January 2024, covering mistral models.
Follow his substack Ahead of AI.
Write Transformers from scratch.
Read The Transformer Family Version 2.0 | Lil’Log for an overview.
Choose whichever format suits you best and implement it from scratch.
Paper
∘ The Annotated Transformer by Harvard
Blogs
∘ Creating a Transformer From Scratch — Part One: The Attention Mechanism (part 2) (code)
∘ Understanding and Coding the Self-Attention Mechanism of Large Language Models From Scratch by Sebastian Raschka, PhD
Videos
∘ Coding a Transformer from scratch on PyTorch, with full explanation, training and inference
∘ NLP: Implementing BERT and Transformers from Scratch
You can code transformers from scratch now. But there’s still more.
Watch these Stanford CS25 — Transformers United videos.
Some good blogs
∘ Gradient Descent into Madness — Building an LLM from scratch
∘ The Illustrated Transformer — Jay Alammar
∘ Some Intuition on Attention and the Transformer by Eugene Yan
∘ Speeding up the GPT — KV cache | Becoming The Unbeatable
∘ Beyond Self-Attention: How a Small Language Model Predicts the Next Token
∘ Llama from scratch (or how to implement a paper without crying) | Brian Kitano
∘ Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch
Watch Umar Jamil
He has fantastic in-depth videos explaining papers. He also shows you the code.
∘ LoRA: Low-Rank Adaptation of Large Language Models — Explained visually + PyTorch code from scratch
∘ Mistral / Mixtral Explained: Sliding Window Attention, Sparse Mixture of Experts, Rolling Buffer
∘ Attention is all you need (Transformer) — Model explanation (including math), Inference and Training
∘ LLaMA explained: KV-Cache, Rotary Positional Embedding, RMS Norm, Grouped Query Attention, SwiGLU
∘ Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)
∘ Some more links related to LLMs that are not exhaustive. Look at LLM Syllabus for a more comprehensive syllabus for LLMs.
Learn how to run open-source models.
Use ollama: Get up and running with Llama 2, Mistral, and other large language models locally
They recently released Python & JavaScript Libraries
Prompt Engineering
Read Prompt Engineering | Lil’Log
ChatGPT Prompt Engineering for Developers by Ise Fulford (OpenAI) and Andrew Ng
DeepLearning.ai also has other short courses you can enroll in for free.
Fine-tuning LLMs
Read the Hugging Face fine-tuning guide.
A good guidebook: Fine-Tuning — The GenAI Guidebook
Check out axolotl.
This is a good article: Fine-tune a Mistral-7b model with Direct Preference Optimization | by Maxime Labonne
RAG
A great article by Anyscale: Building RAG-based LLM Applications for Production
A comprehensive overview of Retrieval Augmented Generation by Aman Chadha
How to stay updated
Combination of newsletters + podcasts + Twitter
For papers, you can follow AK (@_akhaliq)
For podcasts, the best I’ve found is Latent Space by Swyx & Alessio
Join their Discord.
They also have this newsletter, Smol Talk, which summarizes all big AI discords.
Some other newsletters I like are:
∘ The Batch | DeepLearning.AI | AI News & Insights
∘ Interconnects | Nathan Lambert
∘ More in this article.
Other curriculums/listicles you may find useful.
My list was not meant to be exhaustive, but if you still want to find more, here are some.
∘ AI Canon | Andreessen Horowitz
∘ AI Learning Curation — LLM Utils
∘ Threshold to the AI Multiverse | Open DeepLearning
∘ louisfb01/start-llms: A complete guide to start and improve your LLM skills in 2023
I’ve spent enough time writing and organizing this that it’s diminishing returns. It’s time to learn and build.
I hope this will help you in your AI journey!
Thank you for reading!
KINOMOTO.MAG 