Resources

Thanks to the treasure trove of resources readily available on the internet, I’ve been catching up on the latest in Artificial Intelligence (AI), particularly Natural Language Processing (NLP). My aim is to concentrate on educational outreach, bringing STEM to underrepresented populations.

Since Large Language Models (LLMs) are all the rage now, I’m often asked, “Where should I start?” Below are some of my favorite to learn the basics. The first 3 from DeepLearning.AI on Coursera are each 1-hour overviews – more have been released recently. While I haven’t personally used the last two resources, LLM University & LLM Bootcamp, they come highly recommended for those seeking more comprehensive coverage.

Large Language Models

• ChatGPT Prompt Engineering for Developers
• Building Systems with the ChatGPT API
• LangChain for LLM Application Development
• Cohere’s LLM University
• LLM Bootcamp

Otherwise, below are some resources that were particularly helpful in my own AI/ML learning.

Deep Learning

• Coursera DeepLearning specialization
• Coursera Natural Language Processing specialization
• Stanford CS224n: Natural Language Processing with Deep Learning
• HuggingFace Course
• Hands-on Machine Learning with Scikit-Learn and TensorFlow
• fast.ai Practical Deep Learning for Coders
• Edx Google AI for JavaScript developers with Tensorflow.js
• AGI Safety Fundamentals Technical Alignment Curriculum

NLP General Overview

• The Unreasonable Effectiveness of Recurrent Neural Networks (2015)
• Understanding LSTM Networks (2015)
• Attention and Augmented RNNs(2016)
• The Annotated Transformer (2018)
• The Illustrated Transformer (2018)
• The Illustrated BERT, ELMo, and co. (2021)

Journal Papers

• A neural probabilistic language model
• Word2Vec
  • Efficient estimation of word representations in vector space (2013)
    CBOW & Skip-Gram
  • Distributed representation of words and phrases and their compositionality (2013)
    Negative Sampling
• Show, attend and tell: Neural image caption generation with visual attention (2015)
• Transformers: Attention is All you Need (2017)

My notes include more definitions and links to other papers.