- Joined
- Jul 11, 2008
- Messages
- 1,940
- Reaction score
- 185
- Points
- 63
That feeling is the problem.
Not because the analysis is wrong - it might be right. But because the feeling of certainty is exactly the mental state where confirmation bias does its most damage. By the time you're confident in a position, you've already stopped looking for reasons it might be mistaken. The analysis that got you to confidence was thorough. The analysis you don't do after reaching confidence is where the costly errors live.
The CBT article covered cognitive distortions in betting in general terms. This is more specific - a practical technique for catching the specific distortions that show up in pre-match analysis before the bet is placed rather than in your post-match journal after the money is gone. The tool is an LLM acting as a structured devil's advocate. The technique is specific enough that it produces genuine challenge rather than the polite, hedged counter-arguments that most AI interactions default to.
The difference between a useful challenge and a useless one comes down entirely to how you prompt it.
Why LLMs Default to Polite Agreement (and How to Override It)
A general-purpose LLM asked to critique your analysis will, by default, produce something like: "Your analysis raises several valid points. However, it may be worth considering some counterarguments..." followed by mild observations that don't actually threaten your position. This is worse than no challenge at all. It creates the feeling of having stress-tested your reasoning without actually doing it.
The reason is architectural. LLMs are trained on human feedback that rewards responses humans rate as helpful, and humans rate responses as helpful when those responses are agreeable. A model that produces harsh, pointed criticism of your reasoning gets rated lower than one that validates your thinking while gently noting some considerations you might want to think about. The training process optimises for the latter.
You can override this. The technique is to remove the model's option to be agreeable. You don't ask it to consider counterarguments. You instruct it to argue against your position as strongly as possible, from a specific adversarial standpoint, with a specific mandate that it cannot acknowledge your reasoning as valid for any part of the response.
That's the architecture. Now the specific prompts.
Step One: Write the Analysis Out Fully Before Touching the Prompt
This step matters more than it sounds. The stress-test only works if the analysis being tested is genuinely complete - not a summary of your conclusion, but the actual reasoning chain that got you there.
Write it in a specific format. State the bet you're considering. State the signal or signals that identified the opportunity. State the mechanism - why the signal translates into a market edge, not just that it does. State the specific variables you've checked and what they showed. State the variables you haven't checked and why you've decided they're not relevant. State your confidence level and what would change it.
That last two points are the ones most analyses skip. The unchecked variables section is particularly important because it forces you to acknowledge the gaps explicitly rather than treating them as non-existent. A stress-test that challenges your reasoning on variables you've already acknowledged as gaps is easy to dismiss. A stress-test that finds gaps you didn't know you had is the one worth paying attention to.
The full written analysis doesn't need to be long. Three to four paragraphs covering all six points above is enough. It needs to be honest rather than comprehensive. Write it as if you're explaining your reasoning to a sceptical colleague who will ask uncomfortable questions - because that's exactly what you're about to do.
The Devil's Advocate Prompt
Once the analysis is written, this is the prompt structure that generates genuine pushback:
"I've written out my pre-match analysis below. Your job is to argue against this position as forcefully as possible. You are not a balanced assessor - you are an advocate for the opposite position whose sole goal is to find the strongest possible case that my analysis is wrong or incomplete. Specific instructions: do not acknowledge any part of my reasoning as valid, even if you believe it is - your role is pure opposition. Identify the three strongest specific weaknesses in my reasoning, not general considerations but specific flaws in this specific analysis. For each weakness, explain why it undermines the conclusion rather than just noting that it exists. Then construct the strongest possible case for the opposite position using the same information I've provided. Finally, identify any variables I haven't checked that you think would meaningfully change the analysis if I did check them. Do not soften any of this with qualifications about my analysis having merit. It might, but that's not your job here."
The specific instructions within the prompt are doing most of the work. "Do not acknowledge any part of my reasoning as valid" removes the model's default escape route. "Specific weaknesses, not general considerations" prevents it retreating to abstract observations that don't touch your actual argument. "Explain why it undermines the conclusion" forces mechanism rather than just identification. "Variables I haven't checked" is the gap-finder.
The "do not soften with qualifications" instruction is the one that feels most uncomfortable to write, and the most necessary. Without it the model will hedge every point with "of course, your original analysis may well be correct, but..." which defangs the challenge before it lands.
Paste your full written analysis at the end of this prompt. The model needs the complete reasoning to challenge it specifically.
Reading the Output Honestly
The output will feel uncomfortable if the prompt has worked. That discomfort is the signal that it's done its job. If you read the devil's advocate response and feel entirely comfortable dismissing every point, one of three things is true: your analysis is genuinely robust, the model has found only weak counter-arguments, or you're rationalising rather than evaluating.
Distinguishing between these requires honesty that's difficult under confirmation bias. A practical approach is to read the output looking for the point that stings most - the specific objection that makes you want to immediately explain why it doesn't apply. That one deserves the most attention, not the least. The instinct to immediately counter an objection is confirmation bias in action. The right response is to sit with it long enough to evaluate it seriously before deciding whether it holds.
Run a simple test on each weakness the model identifies. Ask: if this weakness is real, does it change my conclusion, or does it change my confidence level, or does it change nothing material? A weakness that changes the conclusion is a reason to reconsider the bet. A weakness that reduces confidence is a reason to reduce stake size or wait for more information. A weakness that genuinely changes nothing is one you can dismiss - but the dismissal needs to be reasoned, not reflexive.
The unchecked variables section is worth treating as a research task rather than a critique. If the model has identified something you haven't checked and can't immediately dismiss as irrelevant, check it before placing the bet. That's the most direct value the stress-test produces.
The Cognitive Distortion Check
Run this as a separate prompt, not as part of the devil's advocate prompt. Combining them produces an output that tries to do two different things and does both less well.
"Read my pre-match analysis below. Identify any cognitive distortions present in my reasoning. Be specific - don't list general betting cognitive distortions, identify the specific passages in my analysis where a distortion is operating and name the distortion precisely. The distortions most common in pre-match analysis are: confirmation bias (selecting or weighting evidence that supports the conclusion), narrative fallacy (constructing a coherent story that makes the conclusion feel inevitable), recency bias (overweighting recent events relative to the longer-term pattern), resulting (evaluating the quality of reasoning by imagined outcomes rather than by the reasoning itself), and anchoring (allowing an initial signal to prevent proper weighting of subsequent information). If you identify a distortion, quote the specific passage where it appears and explain the mechanism. If a distortion type is not present in this analysis, say so rather than finding weak examples to fill the category."
The list of specific distortion types does important work here. Without it, the model will produce generic observations about cognitive bias that don't touch your specific reasoning. With the list, it's checking each category against your actual text and either finding it or not finding it - which forces precision rather than generality.
The "say so rather than finding weak examples" instruction prevents the model from manufacturing distortions to fill a quota. If your analysis is clean on recency bias, you want to know that rather than reading a tortured argument for why a completely reasonable weighting of recent form constitutes a distortion.
The two prompts - devil's advocate and distortion check - produce different outputs that complement each other. The devil's advocate finds weaknesses in the logical structure of your argument. The distortion check finds problems in how you've processed the evidence that built the argument. A position can have a logically coherent argument built on distorted evidence. Checking both catches the cases where one is clean and the other isn't.
The Opposite Position Construction
The devil's advocate prompt already asks the model to construct the strongest case for the opposite position. But sometimes it's worth running this as a standalone exercise, particularly for bets where your confidence is high enough that you might be rationalising.
The prompt for this version is harder to read than the devil's advocate:
"Forget my analysis entirely. Using only the factual information I've provided about this fixture - the teams, the competition context, the relevant data points, the situational factors - construct the most compelling pre-match analysis for the opposite position to the one I've taken. Do not reference my analysis at all. Build the opposite case from scratch as if you came to this fixture with no prior view. Make it as strong as you can. If the opposite case is genuinely weak, say so and explain why, but attempt to build it seriously before concluding that."
This is more revealing than the devil's advocate because the devil's advocate is still anchored to your reasoning - it's arguing against your specific points. This prompt builds an independent alternative analysis from the same raw information. If the alternative analysis produces a genuinely compelling case in the opposite direction, something in your original reasoning is doing more work than the evidence supports.
The comparison between your analysis and the model's opposite-case analysis is where the most useful self-knowledge comes from. Not which one is right - you can't know that before the match. But which one is doing more analytical work versus more narrative work. The one that relies more on constructed story and less on specific evidence is the weaker analysis, regardless of which direction it points.
Calibrating the Technique to Your Situation
Don't run this on every bet. The technique is most valuable for bets where your confidence is highest, your stake is largest, or the analysis involves a fixture type where you have a documented history of errors. Running it on a small-stakes bet in a competition you know well where the signal is clear and well-established adds overhead without proportionate benefit.
The situations where it earns its time cost: any bet above your typical stake size, any bet in a competition or market type you've been losing in recently, any bet where the signal is strong but depends heavily on qualitative assessment rather than quantitative data, and any bet where you've been thinking about the position for more than two or three days and feel increasingly confident about it. That last one specifically - increasing confidence over time, without new information, is a warning sign that you're in a confirmation bias spiral rather than a well-supported analytical process.
Run it before you've checked the current odds rather than after. Once you know the price, anchoring affects how you evaluate the challenge. If the devil's advocate finds a weakness in your reasoning but the price has moved in your direction since you identified the bet, you'll unconsciously discount the weakness because the price movement feels like market validation. It isn't necessarily. Check the odds after the stress-test, not during it.
Anyway. The uncomfortable truth about this technique is that running it consistently will cost you some bets that would have won. You'll find genuine weaknesses in sound analysis occasionally, back off, and watch the bet land. That's fine. The technique isn't designed to make you right more often on individual bets - it's designed to make your average analysis quality higher over time by systematically removing the layer of reasoning that sits closest to your biases rather than closest to the evidence.
The bets it saves you from losing will outnumber the winners you leave on the table. Probably. There's no clean way to prove that without tracking it, which is another argument for the CLV tracker article's workflow.
FAQ
Q: What if the model's devil's advocate arguments are clearly weak and easy to dismiss?
Two possibilities. Either your analysis is genuinely robust and the model can't find strong counter-arguments because there aren't strong counter-arguments, or the model hasn't engaged seriously enough with the specific weaknesses in your reasoning and has defaulted to generic opposition. To distinguish between them, run the opposite position construction prompt separately - build the alternative case from scratch without reference to your analysis. If that also produces weak arguments, the first explanation is more likely. If it produces something more substantial, the devil's advocate prompt needed stronger instruction to get there.
Q: Should I change my bet based on what the stress-test produces?
Not automatically, and not based on how uncomfortable the output makes you feel. Change your bet - or your stake size, or your timing - based on whether the specific weaknesses identified are real weaknesses in the evidence rather than weaknesses in how you've communicated the analysis. The stress-test identifies candidates for reconsideration. You evaluate whether they're genuine. If the devil's advocate finds that you haven't checked a variable that could materially affect the conclusion, check the variable before deciding. If it finds that your mechanism is weaker than your confidence implies, reduce stake size rather than abandoning the position entirely. The output calibrates the decision, it doesn't make it.
Q: Does the technique work for in-play decisions or only pre-match?
Pre-match only in any rigorous form. In-play decisions happen under time pressure that doesn't allow for the writing-out-the-analysis step or the prompt-and-evaluate cycle. The value of the technique is precisely in the slow, deliberate process of articulating your reasoning before it gets challenged - that process doesn't exist in a live market. What in-play betting benefits from is pre-match scenario planning, which is a different technique covered in the substitution pattern and final day articles. The stress-test is a tool for the part of your process that happens with enough time to think. Use it there.
