# granthi-sync v1 The signup → download → link-folders → cloud product spine for the Granthi forge, tested against the BETA forge (granthi-beta.shre.ai). Python 3 stdlib + git CLI only — same portability heritage as the estate's `gitea_sync.py` mesh. ``` ┌──────────────┐ device flow ┌─────────────────┐ │ granthi-sync │ ───────────────▶ │ shre-id Zitadel │ │ (client, │ ◀─────────────── │ id.shre.ai │ │ Mac/laptop) │ access token └─────────────────┘ │ │ │ │ POST /v1/link {zitadel_access_token, device_name} │ │ ───────────────▶ ┌───────────────────────────────┐ │ │ ◀─────────────── │ granthi-link :3042 │ │ │ {login, token} │ (granthi VPS, tailnet-only) │ │ │ │ · userinfo validation │ │ │ POST /v1/repos │ · ensure Gitea user (admin) │ │ │ ───────────────▶ │ · mint scoped user token │ │ │ └──────────────┬────────────────┘ │ │ git push/fetch (user token │ admin API │ │ via credential helper) ▼ │ │ ───────────────▶ ┌───────────────────────────────┐ └──────────────┘ │ BETA forge :3041 │ │ granthi-beta.shre.ai │ └───────────────────────────────┘ ``` ## Quickstart (invited user) You need a shre-id account — an operator creates it; there is no open signup (see "Invite-only story"). You also need to be on the tailnet: the provisioning service is not publicly exposed yet. ```sh git clone https://granthi.shre.ai/nirpa/granthi-sync.git cd granthi-sync # 1. Link this device. Prints a URL + code; approve it in a browser within # 5 minutes. Creates your forge account and stores a scoped token in # ~/.granthi-sync/config.json (0600). You never see a forge password. ./bin/granthi-sync link # 2. See what is already yours on the forge. ./bin/granthi-sync list # 3. Either pull an existing repo down... ./bin/granthi-sync get # or /, --into DIR # 3b. ...or push a local folder up. It becomes a private repo. ./bin/granthi-sync add ~/work/notes # 4. Keep everything synced. Autocommits, ff-pulls, pushes; skips anything # that has diverged rather than merging or forcing. ./bin/granthi-sync watch # --once for a single pass ./bin/granthi-sync status # what is linked, last sync, divergence ``` Run `watch` as a background daemon on macOS with `client/launchd/ai.granthi.sync.plist` (edit the script path, then `launchctl bootstrap gui/$UID `). **If the device code expires** (5 minutes, unapproved), nothing is created — no account, no token, no partial state. Just run `link` again. **Merging is a forge action, not a client one.** `watch` deliberately refuses to merge; when a folder shows `DIVERGED` in `status`, resolve it in git or on the forge web UI. The client will never force or auto-merge your work. ## Components ### `server/granthi_link.py` — provisioning service (granthi VPS) * `GET /health` * `POST /v1/link {zitadel_access_token, device_name}` → validates the token against `https://id.shre.ai/oidc/v1/userinfo`, applies the **identity binding rules** (below), mints a token scoped `write:repository,write:user`, returns `{gitea_base, login, token, token_name}`. POST bodies are capped at **64 KB** (413 beyond; missing `Content-Length` → 411, invalid → 400). * `POST /v1/repos {token, name, private}` → creates the user repo with the USER token; clone/html URLs are rebased onto `public_gitea_base` because the container `ROOT_URL` (https://granthi-beta.shre.ai) does not resolve for tailnet-only clients. Deployment: `/opt/granthi-link/{granthi_link.py,config.json,state.json}` + systemd unit `granthi-link.service`; binds `127.0.0.1:3042` **and** `100.111.127.127:3042` (tailnet). **Not publicly exposed** — see promotion window. The service **refuses to start** (exit 2) unless `config.json` is mode 0600/0400 and owned by the user it runs as — the config carries the forge admin password, so permissive perms fail closed, not open. #### Rate limiting Sliding-window, per `(route, client)`, **on by default** — `/v1/link` creates accounts and mints tokens, so unlimited has to be a deliberate config act, never an omission. Defaults: `/v1/link` 5/hour, `/v1/repos` 60/hour. `GET /health` is never limited. Over the limit → **429** with a `Retry-After` header, decided *before* the body is read so an abusive caller costs nothing. ```json "rate_limit": { "enabled": true, "trust_forwarded_for": false, "rules": {"/v1/link": [5, 3600], "/v1/repos": [60, 3600]} } ``` * A malformed rule **refuses startup** rather than silently meaning unlimited; `[0, N]` disables an endpoint outright. * Denied requests are *not* recorded, so a client that keeps hammering cannot push its own window forward and lock itself out permanently. * State is an in-process dict behind a lock — granthi-link is one `ThreadingHTTPServer`, so that is the entire store. **If this ever runs multi-process or multi-host, the limiter must move with it.** The key store is capped (`MAX_RATE_KEYS`); at capacity it drops least-recent windows and logs loudly, which is fail-open, chosen over an unbounded dict that is a memory DoS. * `trust_forwarded_for` is **off** by default. Turn it on only behind cloudflared, where every request otherwise arrives from the tunnel and one abuser would starve everyone. A caller can *prepend* anything to `X-Forwarded-For`; a trusted proxy *appends* the peer it actually saw, so the service reads the **last** entry, never the first. #### Identity binding (`state.json`) `/v1/link` originally bound purely by `preferred_username` / email local-part — any Zitadel identity whose derived login collided with an existing account got a token for that account (account takeover). The service now persists a map of Zitadel `sub` → Gitea login in `/opt/granthi-link/state.json` (0600, atomic tmp+rename writes) and applies: 1. **Mapped sub** → always the mapped login, regardless of what the current userinfo claims. If the mapped login was deleted from the forge it is re-created only when the service created it originally; adopted accounts are refused (409). 2. **Unmapped sub, login free** → create the user, record the mapping (`created_by_service: true`). A concurrent-create 409 from Gitea is handled idempotently: the user is re-fetched and accepted only if its primary email is exactly the one this request would have set. 3. **Unmapped sub, login taken** → bind ONLY when the Gitea user's primary email equals the Zitadel userinfo `email` **and** `email_verified` is true (recorded with `created_by_service: false`); anything else → `409 login exists and is not linked to this identity`. A Gitea token is **never minted before the binding rule passes**, and a corrupt/unreadable `state.json` fails closed (500) instead of falling back to an empty map. **Migration (pre-1.1.0 accounts):** identities whose userinfo carries no verified `email` (e.g. Zitadel *machine* users) cannot self-adopt an existing forge login under rule (c) — for them the first link after the upgrade would 409 forever. Any forge account the v1 service created before this change must be seeded into `state.json` once, as `{"": {"login": "", "created_by_service": true, ...}}`, written 0600 atomically. As of the beta rollout the only such account is the E2E machine user `granthi-sync-e2e` (seeded); the forge's human admin `nirpa` is never provisioned through `/v1/link`, so nothing else needed seeding. Empirically verified mechanics on Gitea **1.27.1**, re-probed and still holding on **1.27.2** (beta forge, 2026-08-22 — all three results below matched; the probe minted a token on the `granthi-sync-e2e` machine user and deleted it again, `DELETE …/tokens/{id}` returning 204 under basic auth): * Token minting: `POST /api/v1/users/{login}/tokens` returns **401 for token-authenticated sudo** (both `Sudo:` header and `?sudo=`); it only works with **admin basic auth + `Sudo: ` header** (201). The config therefore carries `admin_login`/`admin_password` (root-only, 0600) in addition to `admin_token` (minted via `gitea admin user generate-access-token`, used for all other admin calls). * User creation: `source_id` is omitted → **local user** with a random 30-char password, `must_change_password=false`, `visibility=private`. Rationale: `/api/v1/admin/identity-auth-sources` 404s on 1.27.1; the shre-id OAuth2 source is ID 1 (via `gitea admin auth list`), but users attached to an OAuth2 source cannot basic-auth and admin-created users get no `external_login_user` row anyway — first OIDC web login links by email regardless of this choice. * `test_mode` (config flag, **never in production**): allows `/v1/link` to accept `test_userinfo` in the body instead of a Zitadel round-trip, so E2E can exercise the ensure-user + mint path headlessly. It is honored **only when the service environment also sets `GRANTHI_LINK_ALLOW_TEST_MODE=1`**; a config flag without the env gate is logged as an ERROR and ignored. ### `client/granthi_sync_client.py` (+ `bin/granthi-sync`) — client daemon * `link [--server URL] [--token TOK]` — Zitadel **device flow** (native app `granthi-sync-device`, client_id `386909715541590022`, project granthi-forge `386906525790109702`; grants: device_code + refresh_token): prints the verification URL + user code, polls the token endpoint (`authorization_pending`/`slow_down` handled), then calls `/v1/link`. `--token` skips the device flow with a ready Zitadel token (headless/dev). Result stored in `~/.granthi-sync/config.json` (0600). * `list` — every repo the linked token can see, with the local folder each is already synced to. Reads `GET /api/v1/user/repos` on the forge **directly** with the scoped user token — no granthi-link round-trip, so the read path needs no service change. Pagination is followed to a short page; if the `FORGE_MAX_PAGES` guard trips, the output says the list is incomplete rather than letting a bounded page read as the whole set. * `get [--into DIR]` — the download half of `add`. Clones with `--origin granthi` (the remote name `watch` looks for) and `-c credential.helper=…` (the repo does not exist yet, so the helper cannot be installed first; git also persists it into the new config), then registers the folder in `config.json` with the same shape `add` writes — so a cloned repo is picked up by `watch` immediately. Refuses a non-empty destination. Branch is read with `symbolic-ref` (an empty repo has an unborn HEAD) and falls back to `main`. * `add [--name N] [--private|--public]` — `git init -b main` if needed, creates the cloud repo via `/v1/repos`, adds remote `granthi`, initial commit + push. The token is delivered by a **git credential helper** (the client's hidden `git-credential` subcommand reading the 0600 config) — never embedded in the remote URL (estate rule). * `watch [--interval 30] [--once]` — per folder: autocommit (`sync: `) → fetch → ff-pull if remote strictly ahead → push if local strictly ahead. **DIVERGED → log + record + SKIP. Never force, never merge** — the same policy as the mesh. SIGTERM-clean. * `status` — table of linked folders, last sync, divergence flags. * Run as a daemon on macOS with `client/launchd/ai.granthi.sync.plist` (edit the script path, then `launchctl bootstrap gui/$UID `). ## Tests * `python3 -m unittest discover -s tests` — 65 tests: autocommit/ff/diverged logic against real temp git repos (including "diverged never touches the remote"), config 0600 handling (including umask-proof creation and a no-chmod guard), credential-helper quoting/injection, mocked device-flow polling, the full `/v1/link` + `/v1/repos` service flows against an in-process stub playing Zitadel + Gitea, all identity-binding rules (collision 409, verified-email adoption, deleted-login re-create/refuse, concurrent-create race, corrupt-state fail-closed), the test_mode env gate, config-permission refusal, and the 64 KB body cap. The `list`/`get` set covers pagination-to-a-short-page, truncation being reported rather than hidden, HTTP errors being fatal instead of a silent empty list, non-empty-destination refusal, owner-qualified names, unborn-HEAD branch fallback, no token in the remote URL, and — the one that matters — that a `get` folder is actually picked up by a subsequent `sync_folder` pass. * Live E2E against the beta forge is recorded in the delivery notes (link → add → watch ff/push → forced divergence → DIVERGED skip verified via API, remote sha untouched). ## Invite-only story (v1) There is no open signup. An operator invites a user by creating them in shre-id (Zitadel org). The user downloads the client, runs `granthi-sync link`, signs in at id.shre.ai with the printed device code, and the provisioning service creates their forge account + scoped token on the fly — the forge never sees a password and the user never sees the forge admin. Every linked folder becomes a private repo under their account. ## Promotion window (beta → prod) 1. **Expose :3042** behind cloudflared (granthi.shre.ai vhost or link.granthi.shre.ai) — today it is tailnet-only by design. 2. **Swap forge base URLs** in `/opt/granthi-link/config.json`: `gitea_base` → prod forge, `public_gitea_base` → `https://granthi.shre.ai`; the client default server URL moves to the public endpoint. 3. The `granthi-web` OIDC app already lists the prod callback; the device app is host-independent. Rotate the beta admin token/password out of the config when pointing at prod (prod forge is READ-ONLY to this estate — promotion is an operator action, not an agent action). 4. ~~Add rate limiting / abuse controls before public exposure.~~ **DONE** — see "Rate limiting" above (5 links/hour per client by default). When exposing behind cloudflared, set `trust_forwarded_for: true` in the same change, or every request will look like the tunnel and one abuser will throttle everybody. 5. **Hardening checklist (must all hold before exposing):** - [ ] `config.json` is 0600 (or 0400) and owned by the service user — the service refuses to start otherwise; verify with `systemctl status granthi-link` after any config edit. - [ ] `test_mode` is absent from the production config **and** `GRANTHI_LINK_ALLOW_TEST_MODE` is not set in the unit environment. - [ ] `/opt/granthi-link/state.json` exists, is 0600, and is included in VPS backups — losing it orphans sub→login bindings (existing users would need verified-email re-adoption). - [ ] `GET /health` returns 200 on both binds after restart. - [ ] Spot-check the identity map: a repeat link for a known sub returns the same login; a colliding username with a different sub gets 409.