How Merit AC is built
The same architecture documented in the repo's own ARCHITECTURE.md and backend/README.md — reality as deployed today, expanded here with the data model and the parts deliberately left unbuilt.
1. The ingestion & scoring pipeline
Three independent ingestion paths write into three separate tables, all attributed to a person through an identity-mapping table, and a nightly job compresses everything into one scored row per person that the API and dashboard read.
- LLM proxy / provider billing →
POST /ingest/usage→UsageEvent - GitHub / Jira / HubSpot webhooks →
POST /ingest/outcome→OutcomeEvent, andPOST /ingest/quality-signal→QualitySignal - Okta / Entra SCIM →
IdentityMapping→Identity
UsageEvent, OutcomeEvent, QualitySignal, and Identity all feed scoring.recompute_all(), which runs nightly and writes one row per person per period into PersonScore — the single table every /api/* endpoint and the dashboard actually read.
The load-bearing invariant: the dashboard and every /api/* endpoint read only PersonScore, never raw events — page loads stay fast regardless of how much event history accumulates. IdentityMapping is the other load-bearing piece: if an external id resolves to the wrong (or no) person, every number downstream is wrong, which is why an unmapped id gets a 422 instead of being silently dropped — an unmapped id is a shadow-AI candidate, not something to drop quietly.
Scoring runs in two tiers today, both pure functions over a handful of bulk queries rather than per-row work, so the nightly job stays flat as event tables grow: Tier 1 correlates spend against outcomes (PRs merged, tickets closed, deals advanced) into a value-per-dollar number; Tier 2 layers a slop-risk score from quality proxies (reverts, heavy rewrites, regeneration loops). See §5 for Tier 3.
2. Data model
Every row below belongs to exactly one Organization (the tenant boundary) — see the multi-tenant isolation work referenced in the repo's own commit history.
| Table | What it holds |
|---|---|
Organization | The tenant — its own ingest_token, plan, name |
Team | A grouping of identities, scoped per org |
Identity | A real person, scoped per org (unique by org+email) |
IdentityMapping | External system id (proxy key, GitHub login, …) → Identity |
UsageEvent | One AI-spend event: tool, model, cost, tokens |
OutcomeEvent | A PR merge, ticket close, deal advance, etc. |
QualitySignal | A revert, rewrite, regeneration loop, etc. |
RubricGrade | Sampled human/LLM grading — Tier 3, not yet populated automatically |
PersonScore | One row per (identity, period) — the only table /api/* reads |
DashboardUser | A real login (password or Google), separate from Identity |
3. Where it runs
Cloudflare — Worker + static assets serving the frontend (a Vite + React app built to a plain dist/, no server-side rendering at request time) at usemeritai.com. This page and the rest of the /architecture, /setup/*, /guides, /prompts, /challenge content is prerendered from React components at build time — not client-rendered SPA routes — so each one ships as a real, crawlable file instead of an empty shell that only populates once JavaScript runs.
Fly.io (app meter, region iad) — the FastAPI backend at api.usemeritai.com, backed by a SQLite file on a persistent volume.
Both deploy paths trigger off the same push to main in one repo — no second repo, no manual deploy step in the common case.
4. Does this hosting choice make sense?
Yes, for what this actually is right now — an early-stage prototype being demoed to prospects, not yet handling real customer data.
- The frontend builds to static assets, so a CDN-native static host is the right tool — Cloudflare Workers gives global distribution, automatic HTTPS, and per-PR preview URLs for free.
- The backend is stateful (a SQLite file, an in-process nightly scoring job) and needs a place that keeps a process and a disk alive continuously — Fly.io is a reasonable, low-overhead choice at this scale.
- CORS is locked down to the real production origins, not left wide open the way the local-dev default is.
Not yet for handling real customer data. Per-user login and role-gated admin endpoints are in, but standard pre-production hardening still needs to land: rate limiting, an audit log, backup coverage on the database volume, a staging environment, and monitoring/alerting. None of that is a reason to change the underlying split; it's ordinary engineering work on top of it.
5. What's deliberately not built yet
Named here on purpose, not hidden — these are decisions, not gaps someone forgot about.
Tier 3 calibration — sampled grading needs real RubricGrade volume to be worth building against; not faked with synthetic grades.
Shadow-AI detection — the recoverable-spend estimate includes a placeholder line for it, clearly labeled as an estimate, but reconciling sanctioned spend against observed AI activity isn't implemented yet.
A job scheduler — /admin/recompute-scores is the nightly job's entry point; wiring it to cron/Airflow/a queue is a deployment decision for whoever runs this in production, not something the code assumes for you.