How Merit AC is built
Exactly how Merit AC is built and hosted -- the pipeline, the data model, where it runs, and what's deliberately not built yet.
1. The ingestion & scoring pipeline
Three independent ingestion paths write into three separate tables, all attributed to a person through an identity-mapping table, and a nightly job compresses everything into one scored row per person that the API and dashboard read.
- LLM proxy / provider billing →
POST /ingest/usage→UsageEvent - GitHub / Jira / HubSpot webhooks →
POST /ingest/outcome→OutcomeEvent, andPOST /ingest/quality-signal→QualitySignal - Admin-provisioned mapping (
POST /admin/identity/POST /admin/identity-mapping) →IdentityMapping→Identity— the schema is designed for Okta/Entra SCIM provisioning, but no SCIM protocol integration is built yet (see what's deliberately not built yet)
UsageEvent, OutcomeEvent, QualitySignal, and Identity all feed scoring.recompute_all(), which runs nightly and writes one row per person per period into PersonScore — the single table every /api/* endpoint and the dashboard actually read.
The load-bearing invariant: the dashboard and every /api/* endpoint read only PersonScore, never raw events — page loads stay fast regardless of how much event history accumulates. IdentityMapping is the other load-bearing piece: if an external id resolves to the wrong (or no) person, every number downstream is wrong, which is why an unmapped id gets a 422 instead of being silently dropped — an unmapped id is a shadow-AI candidate, not something to drop quietly.
Scoring runs in two tiers today, both pure functions over a handful of bulk queries rather than per-row work, so the nightly job stays flat as event tables grow: Tier 1 correlates spend against outcomes (PRs merged, tickets closed, deals advanced) into a value-per-dollar number; Tier 2 layers a slop-risk score from quality proxies (reverts, heavy rewrites, regeneration loops). See §5 for Tier 3.
2. Data model
Every row below belongs to exactly one Organization (the tenant boundary) — see the multi-tenant isolation work referenced in the repo's own commit history.
| Table | What it holds |
|---|---|
Organization | The tenant — its own ingest_token, plan, name |
Team | A grouping of identities, scoped per org |
Identity | A real person, scoped per org (unique by org+email) |
IdentityMapping | External system id (proxy key, GitHub login, …) → Identity |
UsageEvent | One AI-spend event: tool, model, cost, tokens |
OutcomeEvent | A PR merge, ticket close, deal advance, etc. |
QualitySignal | A revert, rewrite, regeneration loop, etc. |
RubricGrade | Sampled human/LLM grading — Tier 3, not yet populated automatically |
PersonScore | One row per (identity, period) — the only table /api/* reads |
DashboardUser | A real login (password or Google), separate from Identity |
3. Where it runs
Cloudflare — Worker + static assets serving the frontend (a Vite + React app built to a plain dist/, no server-side rendering at request time) at usemeritai.com. This page and the rest of the /architecture, /setup/*, /guides, /prompts, /challenge content is prerendered from React components at build time — not client-rendered SPA routes — so each one ships as a real, crawlable file instead of an empty shell that only populates once JavaScript runs.
Fly.io (app meter, region iad) — the FastAPI backend at api.usemeritai.com, backed by a SQLite file on a persistent volume.
Both deploy paths trigger off the same push to main in one repo — no second repo, no manual deploy step in the common case.
4. Does this hosting choice make sense?
Yes, for what this actually is right now — an early-stage prototype being demoed to prospects, not yet handling real customer data.
- The frontend builds to static assets, so a CDN-native static host is the right tool — Cloudflare Workers gives global distribution, automatic HTTPS, and per-PR preview URLs for free.
- The backend is stateful (a SQLite file, an in-process nightly scoring job) and needs a place that keeps a process and a disk alive continuously — Fly.io is a reasonable, low-overhead choice at this scale.
- CORS is locked down to the real production origins, not left wide open the way the local-dev default is.
Not yet for handling real customer data. Per-user login and role-gated admin endpoints are in, but standard pre-production hardening still needs to land: rate limiting, an audit log, backup coverage on the database volume, a staging environment, and monitoring/alerting. None of that is a reason to change the underlying split; it's ordinary engineering work on top of it.
5. What's deliberately not built yet
Named here on purpose, not hidden — these are decisions, not gaps someone forgot about.
Tier 3 calibration — sampled grading needs real RubricGrade volume to be worth building against; not faked with synthetic grades.
Shadow-AI detection — the recoverable-spend estimate includes a placeholder line for it, clearly labeled as an estimate, but reconciling sanctioned spend against observed AI activity isn't implemented yet.
A job scheduler — /admin/recompute-scores is the nightly job's entry point; wiring it to cron/Airflow/a queue is a deployment decision for whoever runs this in production, not something the code assumes for you.
SCIM/SSO-driven identity provisioning — the data model's external-id-to-person shape is designed around it, but no SCIM protocol integration exists yet. POST /admin/identity and POST /admin/identity-mapping are today's admin-driven equivalent.
6. How it's licensed
Free for your own personal, non-commercial use -- read it, run it, self-host it to track your own AI spend. Using it on behalf of a company or organization -- internal use, a service to others, evaluating it for a purchase decision -- needs a separate written commercial license.
Full terms in LICENSE, or the plain-English version in LICENSE-PLAIN.md. Using the hosted dashboard itself is separate from this code license -- see /pricing. Operator OS is a different product with its own license: /operator-os.
Looking for the general field guide to AI system design -- archetypes, complex agent patterns, and the ML/AI software landscape? It's moved to its own guide. The composed and advanced multi-stage prompts that used to live on this page are now in the prompt library, alongside a link back from the daily archive.
How it compares