Blog

Thoughts on software development, AI, and technology. Sharing insights from building real-world applications.

The process of creating GPT-2 from scratch and pre-training with DDP.

Fine tuning the pre trained model to follow instructions and emotion.

From first principles, the small mathematical change that unlocked long-context reasoning