📚

Machine learning notes

I've decided to start trying to write up my notes on ML in a more structured form, and I'm posting them here.

These notes are not comprehensive overviews of the topics, but more like a re-write of some core ideas to make them seem as simple/comprehensible as possible.

⚠️

A warning: these notes vary in quality - some are very thorough and I hope rather interesting. Others, not so much 🙃

General

NeurIPS 2021 🔑

Practical Deep Learning for Coders

Graph Representation Learning Book

Probabilistic Machine Learning: An Introduction

Writing a Rebuttal

⬇️ work in progress - need to move more of my notes over

Paper Notes

Paper

Date

Link

2021

https://arxiv.org/abs/1901.00596

2016

https://arxiv.org/abs/1609.03499

Parallel WaveNet

2018

https://arxiv.org/abs/1711.10433

Chip Placement with DRL

2021

https://arxiv.org/abs/2004.10746

2019

https://arxiv.org/abs/1810.04805

Sparsely-Gated Mixture-of-Experts

2017

https://openreview.net/forum?id=B1ckMDqlg

Attention Is All You Need

2017

https://arxiv.org/abs/1706.03762

2015

https://arxiv.org/abs/1412.6980

2021

https://www.nature.com/articles/s41586-021-03819-2

Learning in High Dimension Always Amounts to Extrapolation

2021

https://arxiv.org/abs/2110.09485

2020

https://arxiv.org/abs/2002.12327

2020

https://arxiv.org/abs/2010.11929

Revealing the Dark Secrets of BERT

2019

https://arxiv.org/abs/1908.08593

What Does BERT Look At?

2019

https://arxiv.org/abs/1906.04341

What Language Model to Train if You Have One Million GPU Hours?

2022

https://openreview.net/forum?id=rI7BL3fHIZq

OPT: Open Pre-trained Transformer Language Models

2022

https://arxiv.org/abs/2205.01068

Matryoshka Representations

2022

https://arxiv.org/abs/2205.13147

Emergent Abilities of Large Language Models

2022

https://arxiv.org/abs/2206.07682v1

2022

https://arxiv.org/abs/2206.04615

2022

https://arxiv.org/abs/2205.07000

Choking Off China’s Access to the Future of AI

2022

https://www.csis.org/analysis/choking-chinas-access-future-ai

2022

https://arxiv.org/abs/2211.05100

2022

https://arxiv.org/abs/2211.10438

2023

https://arxiv.org/abs/2302.04761

Intel FP8 PTQ Paper

2023

https://arxiv.org/abs/2309.14592

Sparsity Scaling Laws

2023

https://arxiv.org/abs/2309.08520

Small-scale proxies

https://arxiv.org/abs/2309.14322

Tensor Programs V

Tensor Programs V

2022

https://arxiv.org/abs/2203.03466

Tensor Programs VI

Tensor Programs VI

2023

https://arxiv.org/abs/2310.02244v5

Adaptive Learning Rate Scheduling

Adaptive Learning Rate Scheduling

2023

https://arxiv.org/abs/2310.07831

2023

https://arxiv.org/abs/2310.18313