Flight School

The course

A chapter lists the Field Guide sections to read, then the lesson that codes the idea when it has one (6 of 12 do), then a preset that loads the idea onto Pigeon's real weights in the Coop where one fits (5 of 12).

1. 1

   Chapter 1: What a weight is

   FP32, FP16 and BF16: bits, range and precision

   • Try it in the Coop: FP16

   Not written yet: how FP32, FP16 and BF16 lay out their bits, and how they trade range for precision.

2. 2

   Chapter 2: Scale and zero-point

   The one formula behind all quantization

   • Post-training quantization: How it works
   • Round-to-nearest (RTN): How it works
   • Quantizing a detector: How it works
   • KV-cache quantization: How it works
   • Code it: Fake quantization with group scales

   Not written yet: the zero point, for ranges that aren't centered on zero, with a worked example and an exercise.

3. 3

   Chapter 3: Bits and rounding

   INT8, INT4, INT2 and binary

   • Post-training quantization: How it works
   • Round-to-nearest (RTN): How it works
   • Quantizing a detector: How it works
   • Code it: Fake quantization with group scales
4. 4

   Chapter 4: Granularity

   Per tensor, per channel, per group: how many scales a tensor gets

   • Post-training quantization: The idea
   • Round-to-nearest (RTN): How it works
   • Round-to-nearest (RTN): What the simulations show
   • Quantizing a detector: When it breaks
   • SmoothQuant (weights + activations): How it works
   • Code it: Fake quantization with group scales
   • Try it in the Coop: INT8 with one scale
   • Try it in the Coop: INT8 per channel
5. 5

   Chapter 5: Outliers

   Why a few huge values ruin everything

   • SmoothQuant (weights + activations): The idea
   • Round-to-nearest (RTN): When it breaks
   • Post-training quantization: When it breaks
   • Quantizing a detector: How it works
   • KV-cache quantization: When it breaks
6. 6

   Chapter 6: Activations and calibration

   Weight-only vs W8A8, static vs dynamic

   • SmoothQuant (weights + activations): The idea
   • SmoothQuant (weights + activations): How it works
   • SmoothQuant (weights + activations): The code, line by line
   • KV-cache quantization: The code, line by line
   • GPTQ: When it breaks
   • AWQ (activation-aware weight quantization): How it works

   Not written yet: static activation scales fitted on a calibration set, and when to choose weight-only over W8A8.

7. 7

   Chapter 7: Sensitivity and mixed precision

   Not every layer is equal

   • Quantizing a detector: When it breaks
   • Round-to-nearest (RTN): How it works
   • Try it in the Coop: mixed precision

   Not written yet: how to measure each layer's sensitivity, and how to pick bit widths that fit a size budget.

8. 8

   Chapter 8: Codebooks

   Non-uniform levels and palettes

   • Weight sharing (k-means codebooks): The idea
   • QLoRA (low-rank adapters on a quantized model): How it works
   • Try it in the Coop: a 16-colour palette

   Not written yet: fixed non-uniform grids such as NF4 and FP4, and how they compare with learned palettes.

9. 9

   Chapter 9: Pruning

   Unstructured, 2:4 and structured, and the cliff

   • Magnitude pruning: The idea
   • Structured (neuron) pruning: What the simulations show
   • Channel pruning with dependency groups: When it breaks
   • Code it: Global magnitude pruning
   • Code it: Structured pruning: removing neurons for real
   • Try it in the Coop: 2:4 sparsity

   Not written yet: 2:4 sparsity, how its mask is chosen and why sparse tensor cores run it faster.

10. 10

    Chapter 10: Recovery

    Fine-tuning and quantization-aware training

    • Quantization-aware training: The idea
    • Prune, then fine-tune (cut and regrow): The idea
    • Magnitude pruning: The idea
    • Recovery data for a specialist: What the simulations show
11. 11

    Chapter 11: Distillation

    A big teacher trains a small student

    • Knowledge distillation (Hinton): The idea
    • Code it: The distillation loss and its gradient
12. 12

    Chapter 12: Low-rank

    One big matrix becomes two thin ones

    • Low-rank factorization (SVD): The idea
    • Code it: Low-rank factorization with the SVD

Build it yourself

Write the pieces of compression in plain numpy, from softmax to GPTQ's error-compensation loop. Each lesson gives you starting code and runs hidden tests against yours: gradient checks, comparisons with the lab's own implementation, or a model that has to reach an accuracy.

Your code runs in your own browser (Python compiled to WebAssembly), with a 30-second limit per check. Nothing is sent to a server: your drafts and the lessons you pass are kept in this browser only.

• 1. Softmax, cross-entropy and its gradientUsed in: Knowledge distillation (Hinton)
• 2. A dense layer: forward and backwardUsed in: Knowledge distillation (Hinton), Magnitude pruning
• 3. Train a two-layer network from scratchUsed in: Regularizers & training tricks
• 4. The Adam optimizerUsed in: Knowledge distillation (Hinton)
• 5. The distillation loss and its gradientUsed in: Knowledge distillation (Hinton), Distillation loss variants
• 6. Global magnitude pruningUsed in: Magnitude pruning, Lottery tickets
• 7. Fake quantization with group scalesUsed in: Post-training quantization, Quantization-aware training, Quantizing a detector
• 8. Structured pruning: removing neurons for realUsed in: Structured (neuron) pruning, Channel pruning with dependency groups
• 9. Low-rank factorization with the SVDUsed in: Low-rank factorization (SVD), QLoRA (low-rank adapters on a quantized model)
• 10. GPTQ's error-compensation loopUsed in: GPTQ, SparseGPT