# granthi-sync v1 The signup → download → link-folders → cloud product spine for the Granthi forge, tested against the BETA forge (granthi-beta.shre.ai). Python 3 stdlib + git CLI only — same portability heritage as the estate's `gitea_sync.py` mesh. ``` ┌──────────────┐ device flow ┌─────────────────┐ │ granthi-sync │ ───────────────▶ │ shre-id Zitadel │ │ (client, │ ◀─────────────── │ id.shre.ai │ │ Mac/laptop) │ access token └─────────────────┘ │ │ │ │ POST /v1/link {zitadel_access_token, device_name} │ │ ───────────────▶ ┌───────────────────────────────┐ │ │ ◀─────────────── │ granthi-link :3042 │ │ │ {login, token} │ (granthi VPS, tailnet-only) │ │ │ │ · userinfo validation │ │ │ POST /v1/repos │ · ensure Gitea user (admin) │ │ │ ───────────────▶ │ · mint scoped user token │ │ │ └──────────────┬────────────────┘ │ │ git push/fetch (user token │ admin API │ │ via credential helper) ▼ │ │ ───────────────▶ ┌───────────────────────────────┐ └──────────────┘ │ BETA forge :3041 │ │ granthi-beta.shre.ai │ └───────────────────────────────┘ ``` ## Quickstart (invited user) You need a shre-id account — an operator creates it; there is no open signup (see "Invite-only story"). You also need to be on the tailnet: the provisioning service is not publicly exposed yet. ```sh git clone https://granthi.shre.ai/nirpa/granthi-sync.git cd granthi-sync # 1. Link this device. Prints a URL + code; approve it in a browser within # 5 minutes. Creates your forge account and stores a scoped token in # ~/.granthi-sync/config.json (0600). You never see a forge password. ./bin/granthi-sync link # 2. See what is already yours on the forge. ./bin/granthi-sync list # 3. Either pull an existing repo down... ./bin/granthi-sync get # or /, --into DIR # 3b. ...or push a local folder up. It becomes a private repo. ./bin/granthi-sync add ~/work/notes # 4. Keep everything synced. Autocommits, ff-pulls, pushes; skips anything # that has diverged rather than merging or forcing. ./bin/granthi-sync watch # --once for a single pass ./bin/granthi-sync status # what is linked, last sync, divergence ``` Run `watch` as a background daemon on macOS with `client/launchd/ai.granthi.sync.plist` (edit the script path, then `launchctl bootstrap gui/$UID `). **If the device code expires** (5 minutes, unapproved), nothing is created — no account, no token, no partial state. Just run `link` again. **Merging is a forge action, not a client one.** `watch` deliberately refuses to merge; when a folder shows `DIVERGED` in `status`, resolve it in git or on the forge web UI. The client will never force or auto-merge your work. ## Components ### `server/granthi_link.py` — provisioning service (granthi VPS) * `GET /health` * `POST /v1/link {zitadel_access_token, device_name}` → validates the token against `https://id.shre.ai/oidc/v1/userinfo`, applies the **identity binding rules** (below), mints a token scoped `write:repository,write:user`, returns `{gitea_base, login, token, token_name}`. POST bodies are capped at **64 KB** (413 beyond; missing `Content-Length` → 411, invalid → 400). * `POST /v1/repos {token, name, private}` → creates the user repo with the USER token; clone/html URLs are rebased onto `public_gitea_base` because the container `ROOT_URL` (https://granthi-beta.shre.ai) does not resolve for tailnet-only clients. Deployment: `/opt/granthi-link/{granthi_link.py,config.json,state.json}` + systemd unit `granthi-link.service`; binds `127.0.0.1:3042` **and** `100.111.127.127:3042` (tailnet). **Not publicly exposed** — see promotion window. The service **refuses to start** (exit 2) unless `config.json` is mode 0600/0400 and owned by the user it runs as — the config carries the forge admin password, so permissive perms fail closed, not open. #### Rate limiting Sliding-window, per `(route, client)`, **on by default** — `/v1/link` creates accounts and mints tokens, so unlimited has to be a deliberate config act, never an omission. Defaults: `/v1/link` 5/hour, `/v1/repos` 60/hour. `GET /health` is never limited. Over the limit → **429** with a `Retry-After` header, decided *before* the body is read so an abusive caller costs nothing. ```json "rate_limit": { "enabled": true, "trust_forwarded_for": false, "trusted_proxies": [], "rules": {"/v1/link": [5, 3600], "/v1/repos": [60, 3600]} } ``` * Anything malformed **refuses startup** rather than silently meaning unlimited: a bad rule, a non-boolean `enabled`/`trust_forwarded_for` (JSON `null`, `0`, or the *string* `"false"` — every non-empty string is truthy), or a `rate_limit` that is not an object. `[0, N]` disables an endpoint outright. * Denied requests are *not* recorded, so a client that keeps hammering cannot push its own window forward and lock itself out permanently. * State is an in-process dict behind a lock — granthi-link is one `ThreadingHTTPServer`, so that is the entire store. **If this ever runs multi-process or multi-host, the limiter must move with it.** The clock is read *inside* the lock; taken outside, racing threads append out of order and both `Retry-After` and window reclamation silently go wrong. * The key store is capped (`MAX_RATE_KEYS`) **per route**, not globally. At capacity it reclaims expired windows, and if every window is still live it **refuses the new key** — fail closed. Evicting a live window would let an attacker who can mint many distinct keys clear their *own* limit on demand. The per-route budget matters just as much: with one shared table, a flood of cheap `/v1/repos` keys would exhaust it and lock brand-new `/v1/link` clients out, turning the fail-closed guard into a cross-route DoS. * Rule values must be real integers. `bool` subclasses `int` in Python, so `[5, true]` would otherwise pass as a **1-second** window — an hourly limit quietly becoming ~5/sec. * `trust_forwarded_for` is **off** by default, and turning it on **requires a non-empty `trusted_proxies`** — the header is honored only when the socket peer is in that list. Without it, anyone reaching the origin directly (it also listens on the tailnet) could pick and rotate their own rate-limit key just by sending a header. Turn it on when exposing behind cloudflared, where every request otherwise arrives from the tunnel and one abuser would starve everyone. A caller can *prepend* anything to `X-Forwarded-For`; a trusted proxy *appends* the peer it actually saw, so the service reads the **last** entry, never the first, and requires it to parse as a real IP. `trusted_proxies` is validated at startup: it must be a list (a bare string would be iterated character by character), every entry a valid network, and wildcards (`0.0.0.0/0`, `::/0`) are refused outright — they would restore exactly the "trust anyone" hole the setting exists to close. #### Identity binding (`state.json`) `/v1/link` originally bound purely by `preferred_username` / email local-part — any Zitadel identity whose derived login collided with an existing account got a token for that account (account takeover). The service now persists a map of Zitadel `sub` → Gitea login in `/opt/granthi-link/state.json` (0600, atomic tmp+rename writes) and applies: 1. **Mapped sub** → always the mapped login, regardless of what the current userinfo claims. If the mapped login was deleted from the forge it is re-created only when the service created it originally; adopted accounts are refused (409). 2. **Unmapped sub, login free** → create the user, record the mapping (`created_by_service: true`). A concurrent-create 409 from Gitea is handled idempotently: the user is re-fetched and accepted only if its primary email is exactly the one this request would have set. 3. **Unmapped sub, login taken** → bind ONLY when the Gitea user's primary email equals the Zitadel userinfo `email` **and** `email_verified` is true (recorded with `created_by_service: false`); anything else → `409 login exists and is not linked to this identity`. A Gitea token is **never minted before the binding rule passes**, and a corrupt/unreadable `state.json` fails closed (500) instead of falling back to an empty map. **Migration (pre-1.1.0 accounts):** identities whose userinfo carries no verified `email` (e.g. Zitadel *machine* users) cannot self-adopt an existing forge login under rule (c) — for them the first link after the upgrade would 409 forever. Any forge account the v1 service created before this change must be seeded into `state.json` once, as `{"": {"login": "", "created_by_service": true, ...}}`, written 0600 atomically. As of the beta rollout the only such account is the E2E machine user `granthi-sync-e2e` (seeded); the forge's human admin `nirpa` is never provisioned through `/v1/link`, so nothing else needed seeding. Empirically verified mechanics on Gitea **1.27.1**, re-probed and still holding on **1.27.2** (beta forge, 2026-08-22 — all three results below matched; the probe minted a token on the `granthi-sync-e2e` machine user and deleted it again, `DELETE …/tokens/{id}` returning 204 under basic auth): * Token minting: `POST /api/v1/users/{login}/tokens` returns **401 for token-authenticated sudo** (both `Sudo:` header and `?sudo=`); it only works with **admin basic auth + `Sudo: ` header** (201). The config therefore carries `admin_login`/`admin_password` (root-only, 0600) in addition to `admin_token` (minted via `gitea admin user generate-access-token`, used for all other admin calls). * User creation: `source_id` is omitted → **local user** with a random 30-char password, `must_change_password=false`, `visibility=private`. Rationale: `/api/v1/admin/identity-auth-sources` 404s on 1.27.1; the shre-id OAuth2 source is ID 1 (via `gitea admin auth list`), but users attached to an OAuth2 source cannot basic-auth and admin-created users get no `external_login_user` row anyway — first OIDC web login links by email regardless of this choice. * `test_mode` (config flag, **never in production**): allows `/v1/link` to accept `test_userinfo` in the body instead of a Zitadel round-trip, so E2E can exercise the ensure-user + mint path headlessly. It is honored **only when the service environment also sets `GRANTHI_LINK_ALLOW_TEST_MODE=1`**; a config flag without the env gate is logged as an ERROR and ignored. ### `client/granthi_sync_client.py` (+ `bin/granthi-sync`) — client daemon * `link [--server URL] [--token TOK]` — Zitadel **device flow** (native app `granthi-sync-device`, client_id `386909715541590022`, project granthi-forge `386906525790109702`; grants: device_code + refresh_token): prints the verification URL + user code, polls the token endpoint (`authorization_pending`/`slow_down` handled), then calls `/v1/link`. `--token` skips the device flow with a ready Zitadel token (headless/dev). Result stored in `~/.granthi-sync/config.json` (0600). * `list` — every repo the linked token can see, with the local folder each is already synced to. Reads `GET /api/v1/user/repos` on the forge **directly** with the scoped user token — no granthi-link round-trip, so the read path needs no service change. Pagination is followed to a short page; if the `FORGE_MAX_PAGES` guard trips, the output says the list is incomplete rather than letting a bounded page read as the whole set. * `get [--into DIR]` — the download half of `add`. Clones with `--origin granthi` (the remote name `watch` looks for) and `-c credential.helper=…` (the repo does not exist yet, so the helper cannot be installed first; git also persists it into the new config), then registers the folder in `config.json` with the same shape `add` writes — so a cloned repo is picked up by `watch` immediately. Refuses a non-empty destination. Branch is read with `symbolic-ref` (an empty repo has an unborn HEAD) and falls back to `main`. * `add [--name N] [--private|--public]` — `git init -b main` if needed, creates the cloud repo via `/v1/repos`, adds remote `granthi`, initial commit + push. The token is delivered by a **git credential helper** (the client's hidden `git-credential` subcommand reading the 0600 config) — never embedded in the remote URL (estate rule). * `watch [--interval 30] [--once]` — per folder: autocommit (`sync: `) → fetch → ff-pull if remote strictly ahead → push if local strictly ahead. **DIVERGED → log + record + SKIP. Never force, never merge** — the same policy as the mesh. SIGTERM-clean. * `status` — table of linked folders, last sync, divergence flags. * Run as a daemon on macOS with `client/launchd/ai.granthi.sync.plist` (edit the script path, then `launchctl bootstrap gui/$UID `). ## Tests * `python3 -m unittest discover -s tests` — 65 tests: autocommit/ff/diverged logic against real temp git repos (including "diverged never touches the remote"), config 0600 handling (including umask-proof creation and a no-chmod guard), credential-helper quoting/injection, mocked device-flow polling, the full `/v1/link` + `/v1/repos` service flows against an in-process stub playing Zitadel + Gitea, all identity-binding rules (collision 409, verified-email adoption, deleted-login re-create/refuse, concurrent-create race, corrupt-state fail-closed), the test_mode env gate, config-permission refusal, and the 64 KB body cap. The `list`/`get` set covers pagination-to-a-short-page, truncation being reported rather than hidden, HTTP errors being fatal instead of a silent empty list, non-empty-destination refusal, owner-qualified names, unborn-HEAD branch fallback, no token in the remote URL, and — the one that matters — that a `get` folder is actually picked up by a subsequent `sync_folder` pass. * Live E2E against the beta forge is recorded in the delivery notes (link → add → watch ff/push → forced divergence → DIVERGED skip verified via API, remote sha untouched). ## Invite-only story (v1) There is no open signup. An operator invites a user by creating them in shre-id (Zitadel org). The user downloads the client, runs `granthi-sync link`, signs in at id.shre.ai with the printed device code, and the provisioning service creates their forge account + scoped token on the fly — the forge never sees a password and the user never sees the forge admin. Every linked folder becomes a private repo under their account. ## Promotion window (beta → prod) 1. **Expose :3042** behind cloudflared (granthi.shre.ai vhost or link.granthi.shre.ai) — today it is tailnet-only by design. 2. **Swap forge base URLs** in `/opt/granthi-link/config.json`: `gitea_base` → prod forge, `public_gitea_base` → `https://granthi.shre.ai`; the client default server URL moves to the public endpoint. 3. The `granthi-web` OIDC app already lists the prod callback; the device app is host-independent. Rotate the beta admin token/password out of the config when pointing at prod (prod forge is READ-ONLY to this estate — promotion is an operator action, not an agent action). 4. ~~Add rate limiting / abuse controls before public exposure.~~ **DONE** — see "Rate limiting" above (5 links/hour per client by default). When exposing behind cloudflared, set `trust_forwarded_for: true` in the same change, or every request will look like the tunnel and one abuser will throttle everybody. 5. **Hardening checklist (must all hold before exposing):** - [ ] `config.json` is 0600 (or 0400) and owned by the service user — the service refuses to start otherwise; verify with `systemctl status granthi-link` after any config edit. - [ ] `test_mode` is absent from the production config **and** `GRANTHI_LINK_ALLOW_TEST_MODE` is not set in the unit environment. - [ ] `/opt/granthi-link/state.json` exists, is 0600, and is included in VPS backups — losing it orphans sub→login bindings (existing users would need verified-email re-adoption). - [ ] `GET /health` returns 200 on both binds after restart. - [ ] Spot-check the identity map: a repeat link for a known sub returns the same login; a colliding username with a different sub gets 409.