↓Skip to main content

Neural Networks

2026

Gradient descent, Newton's method, and BFGS following different trajectories toward the minimum of an elliptical quadratic objective.

What Are Machine Learning Models Really Optimizing? From Gradient Descent to Large Language Models

7 October 2026·Updated: 11 October 2026·7808 words·37 mins

A visual metaphor depicting the evolution of ability, with a complex, expansive, and somewhat chaotic neural network on one side symbolizing raw potential and pre-training, gradually transforming or converging into a more structured, refined, and focused network on the other, representing stable, post-trained, and reliable performance, highlighting the journey from broad capability to precise skill.

Pre-training and Post-training: Your Upper and Lower Bounds of Ability

7 August 2026·3692 words·8 mins