- theory
- math
- applied
- code
- philosophy
- external-services
•
•
•
•
•
-
Encoders - Squash Reality into a Vector Space
From SIFT histograms to CLIP's billion-parameter shared space, this post traces how vision and text communities independently discovered the same compression trick and then converged into one. Covers handcrafted features, shallow embeddings, deep CNNs, BERT, ViT, self-supervised pre-training, and multimodal alignment; with open problems and portfolio projects.
-
World Models - From Dyna to Foundation Simulators
A decade of teaching machines to imagine. From Sutton's 1990 Dyna planning loop through Ha & Schmidhuber's dreaming agents, the RSSM latent-dynamics lineage, foundation-scale video simulators, and LeCun's JEPA thesis. Includes open problems and a portfolio project guide.
-
Representation Analysis, A Reading Path
Eleven years of geometric thinking about embedding spaces, distilled into eight papers. From Mikolov's linear analogies (2013) to Bricken et al.'s sparse autoencoders (2023), discover, fix, diagnose, exploit. The throughline is one conjecture refined over a decade, high-level concepts are encoded as directions in representation space.
-
Model2Vec, A Static-Embedding Distillation Algorithm, Explained
How to compress a sentence-transformer into a static lookup table that runs on CPU in three lines of NumPy. A re-implementation guide covering the full pipeline, vocab prep, per-token forward, PCA whitening, SIF/Zipf weighting, optional vocab quantization, and dtype quantization, with a ~80-line reference implementation.
-
Agentic AI - From Symbolic Planning to Autonomous Software Engineers
Three decades of teaching machines to act. From STRIPS and BDI, through ReAct and Toolformer, to GRPO-trained agents solving real GitHub issues at superhuman rates. Covers the full lineage of agentic harnesses, tool-use training, end-to-end RL, and the MCP era. Includes open problems and portfolio projects.