Pre-training and Post-training: Your Upper and Lower Bounds of Ability

Let’s explore a thinking tool about thinking tools: learning. How exactly does it change a person’s abilities?

We’ve discussed methods for applying what you learn, such as “Embrace and Bridge.” But you might still wonder: I’ve learned so many thinking tools; what have they actually transformed me into? What else should I read and practice in the future?

Historically, these types of questions often fell into the realm of life philosophy, with everyone offering their own perspective. However, we can now draw inspiration from AI training.

The world’s knowledge, in the human brain, is nothing more than various patterns. Ability and learning are essentially about using patterns to predict the world, and then correcting those patterns with feedback. What neural networks do is simply “improve predictions based on massive amounts of experience.” In this sense, the human brain and AI are fundamentally no different.

Humanity’s exploration of its own learning methods spans millennia, but its achievements may not be as profound as the research into AI learning in recent years. This is because you can’t experiment rapidly with humans, but you can with AI. To see if a method is effective, you just need to change the training conditions, run a benchmark, and the results are immediately clear.

In this lecture, we will adopt a “reverse bionics” approach—using the research findings from AI training to understand human learning itself.

Of course, this is just an analogy; the technical details of the human brain and AI are not the same… but this is indeed the most insightful theory currently available regarding “what learning is actually good for.”

How Large AI Models Conduct Pre-training #

How are today’s large AI models trained? Let’s temporarily ignore the technical details and treat it as a story of a person’s growth.

A model’s “parameter scale” is analogous to the brain’s capacity: the more total parameters, the more complex patterns it can accommodate. This roughly equates to a person’s natural talent, their “factory hardware.”

“Pre-training” involves feeding the model corpus data—equivalent to a massive amount of text from the entire internet—piece by piece, having it do something seemingly simple: predict the next word. “Bright moonlight before my bed…” What’s the next word? If it guesses wrong, it’s given the correct answer, prompting it to fine-tune related parameters (i.e., “weights”); even if it guesses correctly, the probability of the correct answer is still pushed higher. It’s not being rewarded or punished; it’s being calibrated.

This “predict the next word” action is repeated trillions of times… and the marvelous thing is that prediction forces understanding.

To accurately guess the next word, simply memorizing a dictionary is useless. You need to understand grammar, facts, even causality, and human emotions and etiquette—what a character in a novel will say next depends on their situation and personality. Through these repeated predictions, the model builds its worldview.

Applied to humans, pre-training is the accumulation of experience throughout one’s life: all the books you’ve read, all the people you’ve met, all the events you’ve experienced, these are your training data.

The corpus must be sufficient. DeepMind’s famous 2022 study (code-named Chinchilla) found that with the same computational budget, a model with fewer parameters but sufficient corpus data could outperform a model with far more parameters but insufficient training [1]. This is like saying that if a person doesn’t read enough, having a big head alone is useless.

As long as there are enough parameters and ample corpus data, as you practice, two remarkable phenomena will occur that people find hard to comprehend—

One is “emergence”: when the model’s scale and corpus reach a certain magnitude, some capabilities that no one explicitly taught suddenly appear on their own [2].

The other is “grokking”: In 2022, OpenAI researchers discovered that if a model is repeatedly trained to memorize a small batch of data, it quickly masters it, but it remains helpless when encountering unseen problems; however, if you continue to instill it this way, for a very long time, suddenly at some point it has a breakthrough—it can generalize the patterns to problems it has never seen before [3].

Whether attributed to the inevitable laws of neural networks or regarded as a gift from nature, the essence is “reading a hundred times reveals its meaning”—without emergence and grokking, everything about large models would be invalid.

Post-training: Transforming Knowledge into Usable Ability #

Okay, pre-training is complete, and we have a “base model.” This model is full of knowledge but seems unfocused. If you ask it “What is the capital of France?”, it might answer conventionally, or it might not answer, instead continuing your query by asking: “What is the capital of Germany? What is the capital of Italy?"—because during training, it only learned the pattern of continuing a sequence.

It knows everything, but it doesn’t know how to interact with people. Because it doesn’t know what people want. Does this remind you of certain bookworms?

Therefore, we need “post-training” to transform this erudite but socially awkward model into a functional agent. Post-training typically involves three steps—

Step one, “Supervised Fine-Tuning (SFT)”: Show it tens of thousands of carefully written standard Q&A pairs, letting it imitate. This is like a master teaching an apprentice: I do it once, and you follow. This step teaches format and taste: what constitutes a good answer.

Step two, “Reinforcement Learning from Human Feedback (RLHF)”: Have it generate several answers to the same question, and human judges select their favorite, turning “being liked by humans” into a reward signal.

This is analogous to human socialization: learning etiquette, mastering appropriate boundaries, discerning the judges’ intentions—as you might imagine, here lie Goodhart’s Law and pseudo-alignment: a 2023 study by Anthropic observed a “sycophantic tendency” in several AI assistants, meaning the AI specifically said what the judges wanted to hear, to cater to humans [4].

One might ask, does the work ability of a particularly sycophantic person decline as a result? Exactly! There’s a term in the industry called “alignment tax”: for a model to align with human preferences, certain hard capabilities may decrease [5]. You want it to say pleasant things, but you don’t want it to only say pleasantries; you want it to tell the truth, but sometimes you can’t bear to hear the truth… Isn’t training AI the same principle as raising young people?

Step three, “Reinforcement Learning with Verifiable Rewards (RLVR)”: In domains where results can be automatically verified, such as programming, mathematics, or chess, the question isn’t whether humans like it, but whether it’s correct. Correct answers get points, incorrect answers lose points, allowing the model to interact with the real world through massive practice.

These three steps don’t necessarily follow a fixed order… Once these three steps are completed, a usable AI assistant is born.

The Essence of Post-training: Stability, Not New Capabilities #

Now, the crucial question arises: how many new capabilities are added to the model’s “brain” through these three post-training steps?

The answer will absolutely surprise you: almost none.

What constitutes “having capability”? AI engineers use two common metrics—

One is “pass@k”: allow the model to attempt the same problem k times; if it solves it correctly even once, it counts as a pass. This measures whether there is a path to the answer in the model’s brain—even if it only succeeds once out of a hundred attempts, it’s considered to have a path, and thus the capability [6].

The other is “pass@1”: only one attempt is given; the first try must be correct. This measures whether the model can retrieve the correct path at any given moment.

Simply put, pass@k measures potential, and pass@1 measures performance.

In 2025, a groundbreaking study was conducted by Professor Gao Huang’s team at Tsinghua University [7]. They systematically examined the performance of various reasoning models before and after reinforcement learning. The result was: if only pass@1 is considered, post-trained models significantly outperform base models that have not undergone post-training; however, if pass@k is considered, when k is several hundred, the base model catches up to, and even surpasses, the post-trained model.

What does this mean? The questions that post-trained models consistently solve, the base models could already solve—if you gave them hundreds of opportunities, they would eventually get it right, but their performance was very inconsistent.

In other words, the role of reinforcement learning is merely to make the model’s performance more stable—and this stability comes with a cost: for stability, the model tends to choose validated, high-scoring paths, which means it reduces exploration, to the extent that some problems it originally might have solved correctly, it can no longer solve.

Reinforcement learning adds reliability, not new capabilities.

Previously, OpenAI and Meta had already given similar signals: the 1.3 billion-parameter InstructGPT, after post-training, surpassed the 175 billion-parameter GPT-3 in human preference [5]; LIMA, with only 1000 carefully selected examples, turned a 65 billion-parameter base model into a decent assistant [8].

Researchers have a term for this: “Superficial Alignment Hypothesis” [8]: true capability stems from pre-training; post-training merely governs expression.

Putting all this evidence together leads to the core insight of this lecture—

Pre-training determines the upper bound of your ability; post-training determines the lower bound.

Pre-training determines whether there are paths in your brain; post-training doesn’t build new paths, it only determines whether you will turn onto the correct path when you reach a crossroads.

Pre-training and Post-training in Human Learning #

Everyone who has used Claude Fable says it has a certain “flavor of a large model”: a deep foundation, as if its experience far exceeds that of other models, occasionally revealing flashes of genius. This is the true skill of pre-training.

What does “reading ten thousand books, writing like a god” mean?—“Ten thousand books” is the corpus, “like a god” is pass@k.

Some believe that usable corpus data is nearly exhausted, and pre-training is about to reach its limits for further improvement [9]—but in a private conversation with an Anthropic engineer working on pre-training, they believe that the scaling law for pre-training has not yet ended. Rumor has it that OpenAI’s next-generation model might be based on an even larger pre-trained base.

However, having said that, RLVR within post-training—that is, rewarding the model based on whether the answer is objectively correct, regardless of human preference—if trained long enough and maintaining exploration, might also push up the upper bound of capability [10][11].

In summary, a model that is strong in pre-training but weak in post-training is like Duan Yu who has just mastered the Six Meridian Divine Swords: possessing both natural talent and divine skill, but with inconsistent performance.

Conversely, a model with an average pre-training upper bound but extensive post-training is like Qiao Feng: his Eighteen Dragon Subduing Palms have exactly eighteen moves, each clear and precise, and he executes every palm strike solidly, accurately, and consistently.

Thinking Tools: Your Post-training Secret Manual #

So, what are these thinking tools we discuss in this course?

If a concept is entirely new to you—for example, you hear “path dependence” for the first time, or learn about “power law” for the first time—these can be considered new corpus data. But such knowledge is too scarce for pre-training to rely solely on it.

Most thinking tools are not new knowledge. If you break down any tool, you understand each step. The role of thinking tools is to package what’s already in your brain into an automated short program.

For instance, we discussed “looking at reference classes,” just four characters, but once activated, these four characters can summon your years of accumulated case studies, your understanding of statistics, and your awareness of survivor bias.

Thinking tools are shortcuts to invoke your massive mental base model.

Therefore, I believe that the essence of learning thinking tools is to perform post-training on yourself. It doesn’t make you smarter; rather, it allows your existing intelligence to perform consistently.

How to Conduct Effective Post-training #

If you are not a genius artist, learning thinking tools will not turn you into one. You must first have sufficient pre-training for post-training to be effective.

A person who has never touched business, even if they can recite concepts like “network effects” and “marginal costs,” might not know how to apply them; whereas someone who has read extensively about company history and personally managed accounts, upon hearing “soft budget constraint,” the term will flash like lightning, instantly illuminating their experience.

Tools are not a substitute for experience; tools are compression algorithms for experience.

Moreover, post-training requires genuine practice, not just understanding. How many people can rattle off terms like “Kelly criterion,” “selection bias,” and “prospect theory,” but when it involves their own interests, status, and work, none of these tools are invoked. It’s not that they can’t do it; they can do it at pass@100, but not at pass@1.

You might ask, what’s the point of merely securing the lower bound? However, critical moments in life often test precisely the pass@1 capability. Interviews, negotiations, and crises often provide only one chance. If you can ensure pass@1, you are truly trustworthy.

How to practice? The key to post-training is reinforcement learning—

First, practice in real-world situations. When encountering a moving story, check the base rate; when facing performance indicators, think of Goodhart’s Law—“Embrace and Bridge.”

Second, there must be feedback. Without feedback, there’s no reward signal; so-called hard work is merely adding weight to one’s own mistakes.

Third, become proficient enough that it becomes automatic. Practice an action until pass@1 ≈ 1.

Does practice really work? There’s experimental evidence.

In 2019, three behavioral scientists conducted a specialized training on “confirmation bias” for 290 graduate students from three professional programs. You should be familiar with this concept, as we have mentioned it multiple times and it requires no special explanation.

A few weeks later, these students encountered a complex business case modeled after the Challenger space shuttle launch decision. There was no prior warning, and no one reminded them, “You might consider using that thinking tool you learned a few weeks ago.” The result: trained students were 19 percentage points less likely to fall into confirmation bias, meaning they were less likely to choose the inferior option [12].

19% is not a transformative change, and these students’ pass@1 was not high. But this study at least shows that thinking tools can gain behavioral control through training: in a different scenario, with no reminder, the tool activated itself.

The Advantage of Human Learning: Endless Iteration #

One way we are better than current AI is that we can continuously conduct pre-training and post-training.

AI models have version numbers: once trained, weights frozen, and released, their capabilities are fixed; further improvement requires waiting for the next generation model. Humans have no version numbers. And human weights are dynamically updated; every instance of your post-training today updates your parameters.

Moreover, today’s post-training is tomorrow’s pre-training.

Once these thinking tools become proficient, they will begin to help you select the next batch of pre-training data. Mastering “information value,” you will explore which unknown areas are worth investing time to understand; mastering “active high cognitive load,” you will actively tackle challenging books and problems, not settling for easily understandable information; mastering “unintended consequences,” you will trace a policy backward to see what it ultimately led to; mastering “exploration and exploitation,” you will also regularly leave familiar domains to supplement yourself with new experiences.

You will become more broad-minded, and then you will truly become smarter.

The Engineering of Learning: From Sentiment to Mechanism #

One thread running through this course is that everything can be solved through “engineering.” We always want to break down problems that seem to rely on intuition into mechanisms, and then turn those mechanisms into actionable steps.

Learning is precisely this. Learning is not merely sentiment, nor is it entirely effort—learning is an engineering problem: the quality of the corpus, the source of feedback, the intensity of practice, the design of reward mechanisms.

Shouting “I want to grow,” “I want to improve,” “I will read 50 books this year” is meaningless.

Everything must be translated into pre-training and post-training. Pre-training focuses on input: what you read, whom you meet, what you do, determines whether there are paths in your brain; post-training focuses on feedback and practice: after completing every important task, record errors, analyze reasons, plan improvements, and review and apply them before the next action, determining whether you can consistently follow that path.

The role of reward signals is not to provide emotional value, but to rewrite weights. If you use likes, applause, and a leader’s praise as rewards, what you train is a sycophantic tendency to please judges; if you use work, results, and hard feedback from the real world as rewards, you will both raise your lower bound and continually receive new corpus data that pushes your upper bound higher.

Pre-training determines who you occasionally can be; post-training determines who you usually are.

We learn so many thinking tools not to merely parade an arsenal in our minds. It’s so that one day, when fate suddenly poses a question—just one question, just once—you don’t have to flip through books, nor wait for inspiration, but can steadily meet the challenge.

Concluding Verses #

Parameters, natural talent; corpus, a lifetime. An occasional success, good fortune; consistent verification, achievement. No retakes in the exam, time alters the weights. This self, no fixed version, always in training.

Notes #

[1] Hoffmann, Jordan, et al. “Training Compute-Optimal Large Language Models.” Advances in Neural Information Processing Systems 35 (2022): 30016–30030.

[2] Wei, Jason, et al. “Emergent Abilities of Large Language Models.” Transactions on Machine Learning Research (2022).

[3] Power, Alethea, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. “Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets.” arXiv:2201.02177 (2022).

[4] Sharma, Mrinank, et al. “Towards Understanding Sycophancy in Language Models.” International Conference on Learning Representations (2024).

[5] Ouyang, Long, et al. “Training Language Models to Follow Instructions with Human Feedback.” Advances in Neural Information Processing Systems 35 (2022): 27730–27744.

[6] Chen, Mark, et al. “Evaluating Large Language Models Trained on Code.” arXiv:2107.03374 (2021).

[7] Yue, Yang, et al. “Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?” Advances in Neural Information Processing Systems 38 (2025).

[8] Zhou, Chunting, et al. “LIMA: Less Is More for Alignment.” Advances in Neural Information Processing Systems 36 (2023).

[9] Sutskever, Ilya. “Sequence to Sequence Learning with Neural Networks: What a Decade.” Test of Time Award talk, NeurIPS 2024, Vancouver, December 13, 2024. https://www.youtube.com/watch?v=WQQdd6qGxNs

[10] Liu, Mingjie, et al. “ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models.” Advances in Neural Information Processing Systems 38 (2025).

[11] DeepSeek-AI et al. “DeepSeek-R1 Incentivizes Reasoning in LLMs through Reinforcement Learning.” Nature 645 (2025): 633–638.

[12] Sellier, Anne-Laure, Irene Scopelliti, and Carey K. Morewedge. “Debiasing Training Improves Decision Making in the Field.” Psychological Science 30, no. 9 (2019): 1371–1379；以及 “Corrigendum: Debiasing Training Improves Decision Making in the Field.” Psychological Science 31, no. 6 (2020): 762.