The scoring math

The rest of the docs teach the intuition; this page is the receipts. Everything below is the real method — the same formulas and constants our scoring engine runs — so a quant can audit the claim in an evening. If you skipped the intuition, start with Why being right isn't enough; this page assumes it.

Notation: Φ is the standard-normal CDF. mu_h and sigma_h are the asset's drift and volatility scaled to your horizon h, estimated from price history at creation (no look-ahead) and sealed into the prediction's fingerprint. confidence ∈ [0.5, 0.99].

1. The baseline (the null model)

The probability a skill-free forecaster assigns to your side, from the asset's own drift and vol:

p_base_up = Φ( mu_h / sigma_h )
 
p_base = p_base_up          if direction == "up"
       = 1 − p_base_up      if direction == "down"

If a prediction carries no sealed volatility (legacy / out-of-scope rows), p_base falls back to 0.5 — the score degrades to a plain recentered Brier rather than crashing.

Target calls use a harder null. A precise target is scored against the odds of reaching that exact level, not merely moving in the claimed direction. With relative move m = (target − baseline) / baseline, under a normal null on the move:

z = (m − mu_h) / sigma_h
P(reach) = 1 − Φ(z)     if direction == "up"
         = Φ(z)         if direction == "down"

That reach-probability replaces p_base for a target call. There is no bonus multiplier — no 1.2×, no anything. The extra reward for precision falls out entirely from beating a lower baseline; a sharper claim beats a robot that thought it less likely, and the Brier skill below does the rest.

2. Resolution

r = (outcome_price − baseline_price) / baseline_price     # realized return
dead_band = k · sigma_h                                    # k = 0.5

For a directional call: if |r| < dead_band the move is within noise → void (scores nothing). Otherwise correct = sign(r) matches direction. When no sealed vol is available the dead-band falls back to a flat 0.005 (0.5%).

For a target call: correct = outcome_price ≥ target (up) or ≤ target (down), judged on the horizon price. A target call never voids.

3. Per-prediction skill score

With outcome y = 1 if correct else 0:

brier_author   = (confidence − y)²
brier_baseline = (p_base    − y)²
s = brier_baseline − brier_author      # >0 beat the null; <0 worse than it

s is what a prediction stores as its score_delta (rounded to 4 decimals). It is a contribution to a mean, never a running balance to sum for ranking.

4. Reputation rollup (per user × topic)

Over your resolved (non-void) calls, with age measured from each call's resolution time and a half-life of 730 days (24 months):

w_i    = 0.5 ^ (age_i_days / 730)          # recency weight
S_hat  = Σ(w_i · s_i) / Σ(w_i)             # decayed mean skill
N_eff  = (Σ w_i)² / Σ(w_i²)                # Kish effective sample size

The ranking key is a lower confidence bound — the decayed mean shrunk by its own uncertainty:

rank_key = S_hat − z · sd / √N_eff         # z = 1.0

where sd is the sample standard deviation of your skill scores, floored at 0.35 so a tiny lucky sample (observed spread ≈ 0) still pays an uncertainty penalty. The leaderboard sorts by rank_key descending — this is what buries small-N luck and lets a proven record rise.

Three numbers come out, and they are not interchangeable:

• rank_score = rank_key — the sort key (uncertainty-bounded).
• s_hat = S_hat — the decayed mean skill, the headline quality number.
• score = Σ s_i — the cumulative total, for the trajectory chart only, never for ranking.

Separately, a friendly accuracy = correct / resolved is shown but never ranked.

Display gate. Below about 10 resolved calls a profile is labelled provisional. It's cosmetic — the shrinkage above already handles small N mathematically; the label just warns readers.

5. A worked example

BTC, direction = up, confidence = 0.80, horizon 1 year. At creation the sealed difficulty is mu_h = 0.15, sigma_h = 0.40, so p_base_up = Φ(0.375) ≈ 0.646. A year later r = +0.22 (beyond the dead-band, up) → y = 1:

brier_author   = (0.80 − 1)² = 0.040
brier_baseline = (0.646 − 1)² = 0.125
s = 0.125 − 0.040 = +0.085          # right, and more confident than the null

Now the same call as a target — baseline 60,000, target 72,000, so m = 0.20, z = (0.20 − 0.15)/0.40 = 0.125, and the reach-probability is 1 − Φ(0.125) ≈ 0.450. If it lands:

brier_baseline = (0.450 − 1)² = 0.302
s = 0.302 − 0.040 = +0.262           # far more — purely because the null was harder

The target earns roughly three times the directional call here, and no multiplier was applied anywhere. A permabull, by contrast, only ever matches the drift the null already prices — so their S_hat converges to ~0 across many calls, and their confident misses are penalized hard.

6. The pinned constants

These constants may be tuned after launch as we learn — a change re-flows only through rollups recomputed afterward. The sealed baselines of each past prediction (its entry price, drift, volatility and price source) are immutable and are never re-estimated. So the method can improve without ever rewriting the difficulty of a call you already made.

See also: Reading your reputation for the reader-facing version, and The honest limits on what "derived and revisable" means for your number.