granthi-sync v1

The signup → download → link-folders → cloud product spine for the Granthi forge, tested against the BETA forge (granthi-beta.shre.ai). Python 3 stdlib + git CLI only — same portability heritage as the estate's gitea_sync.py mesh.

   ┌──────────────┐   device flow    ┌─────────────────┐
   │ granthi-sync │ ───────────────▶ │ shre-id Zitadel │
   │  (client,    │ ◀─────────────── │  id.shre.ai     │
   │  Mac/laptop) │   access token   └─────────────────┘
   │              │
   │              │  POST /v1/link {zitadel_access_token, device_name}
   │              │ ───────────────▶ ┌───────────────────────────────┐
   │              │ ◀─────────────── │ granthi-link  :3042           │
   │              │  {login, token}  │ (granthi VPS, tailnet-only)   │
   │              │                  │  · userinfo validation        │
   │              │  POST /v1/repos  │  · ensure Gitea user (admin)  │
   │              │ ───────────────▶ │  · mint scoped user token     │
   │              │                  └──────────────┬────────────────┘
   │              │   git push/fetch (user token    │ admin API
   │              │   via credential helper)        ▼
   │              │ ───────────────▶ ┌───────────────────────────────┐
   └──────────────┘                  │ BETA forge  :3041             │
                                     │ granthi-beta.shre.ai          │
                                     └───────────────────────────────┘

Quickstart (invited user)

You need a shre-id account — an operator creates it; there is no open signup (see "Invite-only story"). You also need to be on the tailnet: the provisioning service is not publicly exposed yet.

git clone https://granthi.shre.ai/nirpa/granthi-sync.git
cd granthi-sync

# 1. Link this device. Prints a URL + code; approve it in a browser within
#    5 minutes. Creates your forge account and stores a scoped token in
#    ~/.granthi-sync/config.json (0600). You never see a forge password.
./bin/granthi-sync link

# 2. See what is already yours on the forge.
./bin/granthi-sync list

# 3. Either pull an existing repo down...
./bin/granthi-sync get <repo>           # or <owner>/<repo>, --into DIR

# 3b. ...or push a local folder up. It becomes a private repo.
./bin/granthi-sync add ~/work/notes

# 4. Keep everything synced. Autocommits, ff-pulls, pushes; skips anything
#    that has diverged rather than merging or forcing.
./bin/granthi-sync watch                # --once for a single pass
./bin/granthi-sync status               # what is linked, last sync, divergence

Run watch as a background daemon on macOS with client/launchd/ai.granthi.sync.plist (edit the script path, then launchctl bootstrap gui/$UID <plist>).

If the device code expires (5 minutes, unapproved), nothing is created — no account, no token, no partial state. Just run link again.

Merging is a forge action, not a client one. watch deliberately refuses to merge; when a folder shows DIVERGED in status, resolve it in git or on the forge web UI. The client will never force or auto-merge your work.

Components

server/granthi_link.py — provisioning service (granthi VPS)

• GET /health
• POST /v1/link {zitadel_access_token, device_name} → validates the token against https://id.shre.ai/oidc/v1/userinfo, applies the identity binding rules (below), mints a token scoped write:repository,write:user, returns {gitea_base, login, token, token_name}. POST bodies are capped at 64 KB (413 beyond; missing Content-Length → 411, invalid → 400).
• POST /v1/repos {token, name, private} → creates the user repo with the USER token; clone/html URLs are rebased onto public_gitea_base because the container ROOT_URL (https://granthi-beta.shre.ai) does not resolve for tailnet-only clients.

Deployment: /opt/granthi-link/{granthi_link.py,config.json,state.json} + systemd unit granthi-link.service; binds 127.0.0.1:3042 and 100.111.127.127:3042 (tailnet). Not publicly exposed — see promotion window. The service refuses to start (exit 2) unless config.json is mode 0600/0400 and owned by the user it runs as — the config carries the forge admin password, so permissive perms fail closed, not open.

Rate limiting

Sliding-window, per (route, client), on by default — /v1/link creates accounts and mints tokens, so unlimited has to be a deliberate config act, never an omission. Defaults: /v1/link 5/hour, /v1/repos 60/hour. GET /health is never limited. Over the limit → 429 with a Retry-After header, decided before the body is read so an abusive caller costs nothing.

"rate_limit": {
  "enabled": true,
  "trust_forwarded_for": false,
  "trusted_proxies": [],
  "rules": {"/v1/link": [5, 3600], "/v1/repos": [60, 3600]}
}

• Anything malformed refuses startup rather than silently meaning unlimited: a bad rule, a non-boolean enabled/trust_forwarded_for (JSON null, 0, or the string "false" — every non-empty string is truthy), or a rate_limit that is not an object. [0, N] disables an endpoint outright.
• Denied requests are not recorded, so a client that keeps hammering cannot push its own window forward and lock itself out permanently.
• State is an in-process dict behind a lock — granthi-link is one ThreadingHTTPServer, so that is the entire store. If this ever runs multi-process or multi-host, the limiter must move with it. The clock is read inside the lock; taken outside, racing threads append out of order and both Retry-After and window reclamation silently go wrong.
• The key store is capped (MAX_RATE_KEYS) per route, not globally. At capacity it reclaims expired windows, and if every window is still live it refuses the new key — fail closed. Evicting a live window would let an attacker who can mint many distinct keys clear their own limit on demand. The per-route budget matters just as much: with one shared table, a flood of cheap /v1/repos keys would exhaust it and lock brand-new /v1/link clients out, turning the fail-closed guard into a cross-route DoS.
• Rule values must be real integers. bool subclasses int in Python, so [5, true] would otherwise pass as a 1-second window — an hourly limit quietly becoming ~5/sec.
• trust_forwarded_for is off by default, and turning it on requires a non-empty trusted_proxies — the header is honored only when the socket peer is in that list. Without it, anyone reaching the origin directly (it also listens on the tailnet) could pick and rotate their own rate-limit key just by sending a header. Turn it on when exposing behind cloudflared, where every request otherwise arrives from the tunnel and one abuser would starve everyone. A caller can prepend anything to X-Forwarded-For; a trusted proxy appends the peer it actually saw, so the service reads the last entry, never the first, and requires it to parse as a real IP. trusted_proxies is validated at startup: it must be a list (a bare string would be iterated character by character), every entry a valid network, and wildcards (0.0.0.0/0, ::/0) are refused outright — they would restore exactly the "trust anyone" hole the setting exists to close.

Identity binding (state.json)

/v1/link originally bound purely by preferred_username / email local-part — any Zitadel identity whose derived login collided with an existing account got a token for that account (account takeover). The service now persists a map of Zitadel sub → Gitea login in /opt/granthi-link/state.json (0600, atomic tmp+rename writes) and applies:

1. Mapped sub → always the mapped login, regardless of what the current userinfo claims. If the mapped login was deleted from the forge it is re-created only when the service created it originally; adopted accounts are refused (409).
2. Unmapped sub, login free → create the user, record the mapping (created_by_service: true). A concurrent-create 409 from Gitea is handled idempotently: the user is re-fetched and accepted only if its primary email is exactly the one this request would have set.
3. Unmapped sub, login taken → bind ONLY when the Gitea user's primary email equals the Zitadel userinfo email and email_verified is true (recorded with created_by_service: false); anything else → 409 login exists and is not linked to this identity.

A Gitea token is never minted before the binding rule passes, and a corrupt/unreadable state.json fails closed (500) instead of falling back to an empty map.

Migration (pre-1.1.0 accounts): identities whose userinfo carries no verified email (e.g. Zitadel machine users) cannot self-adopt an existing forge login under rule (c) — for them the first link after the upgrade would 409 forever. Any forge account the v1 service created before this change must be seeded into state.json once, as {"<sub>": {"login": "<login>", "created_by_service": true, ...}}, written 0600 atomically. As of the beta rollout the only such account is the E2E machine user granthi-sync-e2e (seeded); the forge's human admin nirpa is never provisioned through /v1/link, so nothing else needed seeding.

Empirically verified mechanics on Gitea 1.27.1, re-probed and still holding on 1.27.2 (beta forge, 2026-08-22 — all three results below matched; the probe minted a token on the granthi-sync-e2e machine user and deleted it again, DELETE …/tokens/{id} returning 204 under basic auth):

• Token minting: POST /api/v1/users/{login}/tokens returns 401 for token-authenticated sudo (both Sudo: header and ?sudo=); it only works with admin basic auth + Sudo: <login> header (201). The config therefore carries admin_login/admin_password (root-only, 0600) in addition to admin_token (minted via gitea admin user generate-access-token, used for all other admin calls).
• User creation: source_id is omitted → local user with a random 30-char password, must_change_password=false, visibility=private. Rationale: /api/v1/admin/identity-auth-sources 404s on 1.27.1; the shre-id OAuth2 source is ID 1 (via gitea admin auth list), but users attached to an OAuth2 source cannot basic-auth and admin-created users get no external_login_user row anyway — first OIDC web login links by email regardless of this choice.
• test_mode (config flag, never in production): allows /v1/link to accept test_userinfo in the body instead of a Zitadel round-trip, so E2E can exercise the ensure-user + mint path headlessly. It is honored only when the service environment also sets GRANTHI_LINK_ALLOW_TEST_MODE=1; a config flag without the env gate is logged as an ERROR and ignored.

client/granthi_sync_client.py (+ bin/granthi-sync) — client daemon

• link [--server URL] [--token TOK] — Zitadel device flow (native app granthi-sync-device, client_id 386909715541590022, project granthi-forge 386906525790109702; grants: device_code + refresh_token): prints the verification URL + user code, polls the token endpoint (authorization_pending/slow_down handled), then calls /v1/link. --token skips the device flow with a ready Zitadel token (headless/dev). Result stored in ~/.granthi-sync/config.json (0600).
• list — every repo the linked token can see, with the local folder each is already synced to. Reads GET /api/v1/user/repos on the forge directly with the scoped user token — no granthi-link round-trip, so the read path needs no service change. Pagination is followed to a short page; if the FORGE_MAX_PAGES guard trips, the output says the list is incomplete rather than letting a bounded page read as the whole set.
• get <repo|owner/repo> [--into DIR] — the download half of add. Clones with --origin granthi (the remote name watch looks for) and -c credential.helper=… (the repo does not exist yet, so the helper cannot be installed first; git also persists it into the new config), then registers the folder in config.json with the same shape add writes — so a cloned repo is picked up by watch immediately. Refuses a non-empty destination. Branch is read with symbolic-ref (an empty repo has an unborn HEAD) and falls back to main.
• add <folder> [--name N] [--private|--public] — git init -b main if needed, creates the cloud repo via /v1/repos, adds remote granthi, initial commit + push. The token is delivered by a git credential helper (the client's hidden git-credential subcommand reading the 0600 config) — never embedded in the remote URL (estate rule).
• watch [--interval 30] [--once] — per folder: autocommit (sync: <ISO ts>) → fetch → ff-pull if remote strictly ahead → push if local strictly ahead. DIVERGED → log + record + SKIP. Never force, never merge — the same policy as the mesh. SIGTERM-clean.
• status — table of linked folders, last sync, divergence flags.
• Run as a daemon on macOS with client/launchd/ai.granthi.sync.plist (edit the script path, then launchctl bootstrap gui/$UID <plist>).

Tests

• python3 -m unittest discover -s tests — 65 tests: autocommit/ff/diverged logic against real temp git repos (including "diverged never touches the remote"), config 0600 handling (including umask-proof creation and a no-chmod guard), credential-helper quoting/injection, mocked device-flow polling, the full /v1/link + /v1/repos service flows against an in-process stub playing Zitadel + Gitea, all identity-binding rules (collision 409, verified-email adoption, deleted-login re-create/refuse, concurrent-create race, corrupt-state fail-closed), the test_mode env gate, config-permission refusal, and the 64 KB body cap. The list/get set covers pagination-to-a-short-page, truncation being reported rather than hidden, HTTP errors being fatal instead of a silent empty list, non-empty-destination refusal, owner-qualified names, unborn-HEAD branch fallback, no token in the remote URL, and — the one that matters — that a get folder is actually picked up by a subsequent sync_folder pass.
• Live E2E against the beta forge is recorded in the delivery notes (link → add → watch ff/push → forced divergence → DIVERGED skip verified via API, remote sha untouched).

Invite-only story (v1)

There is no open signup. An operator invites a user by creating them in shre-id (Zitadel org). The user downloads the client, runs granthi-sync link, signs in at id.shre.ai with the printed device code, and the provisioning service creates their forge account + scoped token on the fly — the forge never sees a password and the user never sees the forge admin. Every linked folder becomes a private repo under their account.

Promotion window (beta → prod)

1. Expose :3042 behind cloudflared (granthi.shre.ai vhost or link.granthi.shre.ai) — today it is tailnet-only by design.
2. Swap forge base URLs in /opt/granthi-link/config.json: gitea_base → prod forge, public_gitea_base → https://granthi.shre.ai; the client default server URL moves to the public endpoint.
3. The granthi-web OIDC app already lists the prod callback; the device app is host-independent. Rotate the beta admin token/password out of the config when pointing at prod (prod forge is READ-ONLY to this estate — promotion is an operator action, not an agent action).
4. Add rate limiting / abuse controls before public exposure. DONE — see "Rate limiting" above (5 links/hour per client by default). When exposing behind cloudflared, set trust_forwarded_for: true in the same change, or every request will look like the tunnel and one abuser will throttle everybody.
5. Hardening checklist (must all hold before exposing):
   • config.json is 0600 (or 0400) and owned by the service user — the service refuses to start otherwise; verify with systemctl status granthi-link after any config edit.
   • test_mode is absent from the production config and GRANTHI_LINK_ALLOW_TEST_MODE is not set in the unit environment.
   • /opt/granthi-link/state.json exists, is 0600, and is included in VPS backups — losing it orphans sub→login bindings (existing users would need verified-email re-adoption).
   • GET /health returns 200 on both binds after restart.
   • Spot-check the identity map: a repeat link for a known sub returns the same login; a colliding username with a different sub gets 409.