Reasoned Rule Mining

Raw language-model outputs are badly calibrated in four specific ways: token log-probabilities correlate poorly with true posterior uncertainty, small changes in decoding or prompting produce large changes in output, the implicit class prior inside the model has nothing to do with the base rate of the problem it is pointed at, and accuracy-centred metrics hide the disproportionate cost of false positives. At a 2% base rate those failures compound and precision collapses.

Reasoned Rule Mining produces stage-wise evidence as calibrated scores, maps those scores to posterior probabilities by logistic calibration, fuses the calibrated evidence additively in log-odds following Bayes' rule under conditional independence, and selects the operating point to maximise a precision-weighted objective that values precision four times as heavily as recall.

24.5%

precision at a 2% base rate

12.3x

uplift over market baseline

91.3%

of expensive evaluations avoided

Quant VC applies reproducible, measurable methods to decisions normally made on judgement. The value is not that a model outperforms a partner, but that its reasoning can be written down, tested against outcomes, and improved deliberately instead of anecdotally.

Reasoned Rule Mining sits in a line of work alongside GPTree, which builds LLM decision trees, Random Rule Forest, which aggregates features by unit-weight vote, and Policy Induction, which produces editable natural-language policies. Its particular contribution is treating the conversion of a language model into a classifier as a calibration problem.

Three phases. Rule mining extracts interpretable IF-THEN predicates from LLM reasoning traces by structured pattern matching, with perplexity-based filtering to catch hallucinated rules and multi-level safeguards against overprediction. Training compiles the validated rules into auditable decision policies.

Inference runs as a cascade: a lightweight rules-first screen passes over every candidate, and only cases the screen is confident about, above a 95% threshold, are promoted to an expensive second-stage re-evaluation deliberately designed to be harsh.

Adjudication is numerical. The stage-wise scores are calibrated into genuine posteriors, fused additively in log-odds, and thresholded at the operating point that maximises the precision-weighted objective.

The instinct when a language model classifies badly is to rewrite the prompt. That treats a symptom. The model's scores may order candidates perfectly well while being systematically wrong about magnitude, and at a 2% base rate magnitude is what determines precision.

Platt scaling preserves the ordering while mapping the scores onto reliable posteriors. Fusing in log-odds rather than averaging probabilities avoids the artifacts that probability-averaging introduces in ensembles. Both are old, well-understood techniques; the contribution is recognising that this is the problem they solve.

On a reviewed subset of 8,000 founders containing 160 positives, drawn from a pool of 9,892, the selected operating point reaches 24.5% precision at 15.0% recall, with F0.5 of 0.217, a 12.25x uplift over the market baseline.

Run precalibrated, it reaches 17.3% precision at 13.2% recall, F0.5 of 0.162, an 8.65x uplift. A single LLM baseline on the same data manages 4.5% precision at 18.0% recall, F0.5 of 0.053. Production models on a 1.9% base rate reach 19% to 38% precision, a 10x to 20x lift.

The human comparisons are normalised to the benchmark prevalence. Market-index baselines achieve roughly 1.9% precision at inception stage. Y Combinator averages 3.25% precision at 6.9% recall. Tier-one venture firms typically achieve 5.6% precision at 5.2% recall.

The second-stage evaluation is the expensive part. Running it on everything is the obvious approach and the wrong one, because most candidates are resolved confidently by the rules-first screen and gain nothing from further scrutiny.

Of 8,000 founders, only 698, about 8.7%, required stage-two evaluation. The remaining 91.3% were resolved by the screen. That is an order-of-magnitude reduction in compute with precision maintained, which is what makes the method deployable rather than merely publishable.

Three things, and auditability was a design constraint rather than a byproduct. The rule policies are human-readable IF-THEN statements, so an expert can find a rule encoding a bias they object to and remove it. The posteriors are calibrated, so a threshold means something and can be reported to limited partners as a real probability. The log-odds fusion is explicit, so the contribution of each stage to a given decision can be read off and adjusted under partnership review.

VCBench: 9,892 founder profiles from companies founded between 2010 and 2016, roughly one third of US startups raising at least $100,000 annually in that window, drawn from Crunchbase and LinkedIn. Each profile is a structured natural-language summary of the founder's background based solely on what was knowable at the investment decision point, produced by an anonymisation pipeline that preserves predictive signal while protecting the underlying records.

A founder counts as a positive on an IPO above $500M, an acquisition above $500M, or total funding above $500M, so only exceptional outcomes qualify. Negatives raised modest amounts and did not meet those criteria; ambiguous cases were excluded rather than guessed. The final set has 890 successes, a 9.0% prevalence, evaluated under six-fold cross-validation to prevent leakage.

Platt scaling assumes the calibration map is monotonic; where the relationship is non-monotone, isotonic regression would fit better. The additive log-odds decomposition assumes conditional independence between stages, which is an approximation rather than a fact about the data. The framework is binary and single-period, so it does not address multiclass outcomes or how a prediction should evolve as evidence arrives over time.

With Yigit Ihlamur. A shorter version appeared at IEEE CSCloud, SecureFinAI, 2025. The extended version received a Commendation Award at ICIM 2026 in Oxford and is in printing. Patent pending.