ProphetAI
A probability engine for arbitrary real-world questions — five language models vote, statistics decide.
The problem
Ask a single language model how likely something is and you get a confident number with nothing behind it. Ask it twice and you often get two different numbers. Neither is calibrated, and neither tells you how much to trust it.
I wanted a system where the disagreement between models was the signal, not the bug — and where the output was a defensible confidence score rather than a guess dressed up as a fact.
What I built
A request fans out to five providers — Together, OpenRouter, Groq, Cohere and Mistral — behind one API. The spread across their answers feeds a statistical smoothing step that produces the final score, clamped at 1e-6 so a probability is never reported as a flat zero.
Everything else exists to make that affordable. Providers sit in a hash map for O(1) rotation, giving roughly 30 requests per minute across the pool. Identical and near-identical questions are cached, requests are queued rather than fired in bursts, and the whole thing stays inside free-tier limits.
Considered
Rejected. No API cost and instant offline answers, but even a 2B-parameter model is around 500 MB, and iOS restrictions make cross-platform local inference impractical.
Rejected. Full control and no rate limits, but the cheapest GPU is $20–50 a month out of my own pocket, and the bill scales with demand rather than with value.
Chosen. A hash map gives O(1) provider selection at roughly 30 RPM across the pool. It will not survive real scale, and it is the right answer for a project that has to cost nothing.
Measured
- Passed 10,000 uses without a paid tier anywhere in the stack.
- Cut latency roughly 30% through caching, queueing and SQLite-backed lookups.
- Deployed as Docker microservices through Firebase Hosting and GitHub Actions, with ELK and cloud logging for visibility.
Plan
Plan
5 items · scale 1:1What I'd redraw
The weakest part is that the five models are weighted equally. They are not equally good, and they are not equally good at the same things. The honest next version scores each provider against resolved outcomes over time and weights the vote accordingly — which means storing predictions and grading them later, something the current schema has no room for.