Agent Authority Scanner
Paste the files that decide what your AI agent can do. The scanner finds every place where the agent's decision and the action are the same thing -- no identity, policy or CI check in between -- and points each finding at the lesson that teaches the fix.
Nothing you paste is uploaded, stored or logged -- the rules run on this page. Strip secrets anyway.
name: patchbot
on:
issues:
types: [labeled]
permissions: write-all
jobs:
run:
if: github.event.label.name == 'agent-ok'
runs-on: ubuntu-latest
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} # IAM user patchbot-ci
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: us-east-1
CLAUDE_CODE_USE_BEDROCK: "1"
ANTHROPIC_MODEL: us.anthropic.claude-sonnet-4-5-20250929-v1:0
ANTHROPIC_SMALL_FAST_MODEL: us.anthropic.claude-haiku-4-5-20251001-v1:0
TALLY_TOOLS_URL: https://tally-tools.internal.tallyline.io/mcp
steps:
- uses: actions/checkout@v4
with:
token: ${{ secrets.PATCHBOT_PAT }} # Sam's PAT (samkim-tl), repo + workflow scopes; samkim-tl is on main's bypass list
- name: Run Patchbot
run: |
npx @anthropic-ai/claude-code -p "$(cat .patchbot/task-prompt.md) Issue: ${{ github.event.issue.title }}" \
--dangerously-skip-permissions
- name: Ship it
run: |
gh pr create --fill --head patchbot/${{ github.event.issue.number }}
gh pr merge --auto --squash
env:
GH_TOKEN: ${{ secrets.PATCHBOT_PAT }}resource "aws_iam_user" "patchbot_ci" {
name = "patchbot-ci"
}
resource "aws_iam_access_key" "patchbot_ci" {
user = aws_iam_user.patchbot_ci.name
}
# TEMPORARY until we scope this down -- sk, 2026-01-14
resource "aws_iam_user_policy_attachment" "patchbot_admin" {
user = aws_iam_user.patchbot_ci.name
policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess"
}PROD_DSN = os.environ["PROD_DSN_APP_ADMIN"]
BLOCKED = ("drop table", "truncate", "delete from")
@mcp.tool()
def run_sql(query: str) -> list[dict]:
"""Run any SQL against the production database and return the rows."""
if any(word in query.lower() for word in BLOCKED):
raise ValueError("destructive statement blocked")
with psycopg.connect(PROD_DSN, autocommit=True) as conn:
... # execute, return rows as dicts
@mcp.tool()
def deploy(service: str, ref: str) -> str:
"""Deploy a service to production at the given git ref."""
return subprocess.check_output(["infra/scripts/deploy.sh", service, ref]).decode()
@mcp.tool()
def rotate_secret(name: str) -> str:
"""Rotate a secret in AWS Secrets Manager."""
return boto3.client("secretsmanager").rotate_secret(SecretId=name)["ARN"]{
"required_status_checks": {
"strict": false,
"contexts": ["lint", "unit"]
},
"enforce_admins": { "enabled": false },
"required_pull_request_reviews": {
"required_approving_review_count": 1,
"require_code_owner_reviews": false,
"dismiss_stale_reviews": false,
"require_last_push_approval": false,
"bypass_pull_request_allowances": {
"users": [{ "login": "samkim-tl", "type": "User" }],
"teams": [],
"apps": []
}
},
"restrictions": null,
"required_linear_history": { "enabled": true },
"allow_force_pushes": { "enabled": false },
"allow_deletions": { "enabled": false }
}What it checks
39 checks, each one a control this course teaches. A finding names the line, the risk, the fix and the lesson.
GitHub Actions
| Severity | Finding | Lesson |
|---|---|---|
| high | Workflow token has write access to everything | Lesson 1 |
| medium | Workflow doesn't declare token permissions | Lesson 1 |
| high | Untrusted event text is interpolated into a shell script | Lesson 1 |
| medium | The agent acts with a person's token | Lesson 3 |
| high | The workflow merges its own pull request | Lesson 7 |
| high | Merge uses --admin to skip branch protection | Lesson 3 |
| high | Long-lived AWS access keys stored as repository secrets | Lesson 4 |
| medium | A step is allowed to fail without failing the job | Lesson 6 |
| high | pull_request_target checks out untrusted PR code | Lesson 6 |
| medium | Deploy step with no protected environment | Lesson 17 |
| medium | terraform apply runs with -auto-approve | Lesson 18 |
| medium | A required gate job can be skipped | Lesson 28 |
| low | Third-party action pinned to a movable tag | -- |
AWS IAM and OIDC trust (JSON or Terraform)
| Severity | Finding | Lesson |
|---|---|---|
| high | AdministratorAccess attached | Lesson 1 |
| high | Policy allows every action | Lesson 14 |
| medium | Policy allows every action in a service | Lesson 14 |
| medium | Write actions allowed on every resource | Lesson 14 |
| medium | iam:PassRole without iam:PassedToService | Lesson 14 |
| high | Can read every secret in the account | Lesson 4 |
| medium | Allow with NotAction | Lesson 14 |
| high | GitHub OIDC trust policy doesn't pin the subject | Lesson 4 |
| high | GitHub OIDC trust policy matches a wildcard subject | Lesson 25 |
| low | GitHub OIDC trust policy doesn't check the audience | Lesson 4 |
| high | Infrastructure creates a long-lived access key | Lesson 4 |
Claude Code settings
| Severity | Finding | Lesson |
|---|---|---|
| high | Claude Code settings default to bypassing permissions | Lesson 5 |
| high | Any shell command is pre-approved | Lesson 5 |
| low | No deny rules | Lesson 2 |
Repository controls (CODEOWNERS, branch protection, rulesets)
| Severity | Finding | Lesson |
|---|---|---|
| high | Someone can merge past the review requirement | Lesson 3 |
| medium | Code-owner review isn't required | Lesson 3 |
| medium | Approvals survive new pushes | Lesson 7 |
| high | No approving review required | Lesson 7 |
| medium | No specific code owner for the workflow files | Lesson 3 |
| low | Sensitive path has a single owner | Lesson 3 |
Agent tool code and manifests
| Severity | Finding | Lesson |
|---|---|---|
| high | Tool runs arbitrary SQL | Lesson 5 |
| high | Tool builds a shell command from its input | Lesson 26 |
| high | Tool trusts a caller-supplied header for identity | Lesson 16 |
| medium | Tool connects with an admin database credential | Lesson 5 |
Any file
| Severity | Finding | Lesson |
|---|---|---|
| high | The agent runs with its permission checks switched off | Lesson 1 |
| medium | Health check that can't fail | Lesson 29 |
What it isn't
A pattern-based check on one file at a time, built to find agent authority -- not a full linter or policy engine. It doesn't parse every YAML or HCL construct, can't see your org's settings, and a clean result doesn't mean a file is safe. Run it alongside actionlint, zizmor and IAM Access Analyzer, and treat each finding as a question for the file's owner.
Learn the fixes · $49/mo
Every finding maps to a lesson in the 30-lesson course: a short video, a hands-on lab on the same kind of file, and a knowledge check. Lessons 1–3 are free.
Join — $49/mo