Calibration Check: When You Say 70%, Does It Happen 70% of the Time?

[Klaus]

Calibration is a simple idea that very few bettors measure.
If you assign a probability of 70% to a set of outcomes, roughly 70% of them should occur. Not every time. Over enough instances. If your 70s hit 58% of the time, the estimates are overconfident, whatever your overall win rate says.
I record a probability for every selection. After fourteen years I have 2,140 qualifying bets in the database, and I have only recently done the exercise properly: sorted them by stated probability, grouped them into bins, and compared stated with observed.
The result was uncomfortable. I will share it, but first I would like to know how many of you record a probability at all.
When you say 70%, what actually happens?

 

[FadeThePublic]

I record one. Google Sheets, a column called conf (I'm a developer, we name things badly). It's a number I type before the game, usually 60 to 80.
Here's what nobody wants to hear: I plotted it last month. Stated on the bottom, actual hit rate up the side. A calibrated bettor gets a diagonal line. Mine is a hill that gave up. Fine around 55 to 60, then flat the whole way up. My 80s hit about the same as my 60s.
Reverse line movement spots are the worst of it. I write 75 every time I see one. They hit just over 54. RIP public money, apparently. (It's me. I'm the public money.)

 

[Aionora]

Interesting point about your 80% predictions performing much like your 60s.

I wonder how much of that comes down to sample size, and how much is genuine overconfidence in the estimates.

Have you looked at the number of predictions in each range? I'd be curious to see whether the pattern holds over a larger sample.

 

[ParlayPrincess_88]

Okay so I've literally never written down a percentage. I just say lock lol
But I post my slips so I went back through the ones I called locks. Like actually called them that in the caption. There were 17. Four hit.
That's like 24% and I was SO sure on every single one
I don't even know what number I'd have written. Like 90? That feels really bad to type.