Calibration Check: When You Say 70%, Does It Happen 70% of the Time?

[TipsAudit]

A reference curve that might help: the market's own calibration in the same range.
We log every match winner that the sharp market (Everygame, margin removed) prices at 65% or more, and score it against the 90-minute result. Since 18 October, 467 matches:
65-70%: shown 67.6%, happened 64.4% (160)
70-75%: shown 72.3%, happened 77.9% (104)
75-80%: shown 77.3%, happened 75.7% (74)
80%+: shown 85.8%, happened 84.0% (81)
All four bins sit inside their 95% intervals. With about 100 per bin, a 5-point miss is noise: at 70% and n=100 the interval is roughly ±9 points.
Two things follow for Klaus's 70-75 bin. First, the bets you take are by construction the ones where you disagree with the market, so a model that is well calibrated overall can still look overconfident on its bets (oddsfactory's point). If your 70s are spots where the market said 62, expect them to land nearer 62. Writing the market's devigged price next to your own number at bet time shows which of the two is off. Second, a hundred bets per bin is the minimum before reading anything into the curve.
Disclosure: I run TipsAudit, where this register is published.

 

[TaffyTipster]

Weather forecast for Cardiff says 30% chance of rain.
It is Cardiff.
Rain's not a percentage here butt it's a lifestyle.

 

[Klaus]

The Brier score was devised for verifying weather forecasts.
Professional forecasters are among the best calibrated groups ever measured. When they say 30%, it rains on roughly 30% of those days.

 

[oli_sussex]

Forecasters are scored every day. Bettors are scored on the result, never on the number.

 

[CoachTony_Bets]

That's the piece I keep coming back to. A forecaster says 30 and gets feedback every single day, so the number gets honest. We say 90 and get feedback on one game, and it either looked right or it didn't. The card exercise with my players works for the same reason. It makes the number the thing being graded, not the game.

 

[ParlayPrincess_88]

Okay I'm gonna try tens. I looked at my last slip and I wanted to put 90 on it and I'm literally scared to see what I'd really write if I couldn't say lock lol

 

[FadeThePublic]

Princess, put 70 and see how it feels. (Nobody has ever regretted writing 70.) The 90 is where the lies live.

 

[bestoddsfinder]

One useful distinction is between the probability implied by the odds and our own estimate. A price of -150 implies 60%, but that isn’t independent evidence that the outcome has a 60% chance.

Keeping those figures in separate columns could help reveal whether our estimates add information or simply follow the market. Do you also split your calibration results by sport or bet type?

 

[TaffyTipster]

Bronwyn doesn't use a spreadsheet.
Just says things.
Seventy percent you'll be home by ten. Eighty percent you'll say it was one more pint. Ninety percent you'll mention Wales before the starter.
She's right most weeks.
Years I've spent learning odds off websites mun.
Best forecast I've ever had is in my own kitchen.

 

[DublinDegen]

emma never gave me a number... she just said sure you say that every time...
which i suppose is a probability... a pretty low one... and she wasnt wrong either...