nirpa/granthi-sync
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request 'feat: back up uncommitted work, restore points, scoped bulk pull' (#1) from feat/backup-snapshots-and-scoped-pull into main

Browse Source
This commit is contained in:
Nirav Patel
2026-08-23 11:58:07 -04:00
parent 1091aa61f2 eb70ca08ad
commit a9321590f5
3 changed files with 1712 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+226 -16
View File
@@ -1,4 +1,4 @@
# granthi-sync v1
# granthi-sync v1.2
The signup → download → link-folders → cloud product spine for the Granthi
forge, tested against the BETA forge (granthi-beta.shre.ai). Python 3 stdlib +
@@ -50,10 +50,17 @@ cd granthi-sync
# 3b. ...or push a local folder up. It becomes a private repo.
./bin/granthi-sync add ~/work/notes
# 3c. ...or pull down everything this account is allowed to see.
./bin/granthi-sync get --all --into ~/granthi
# 4. Keep everything synced. Autocommits, ff-pulls, pushes; skips anything
# that has diverged rather than merging or forcing.
./bin/granthi-sync watch # --once for a single pass
./bin/granthi-sync status # what is linked, last sync, divergence
./bin/granthi-sync status # what is linked, mode, last sync
# 5. Go back to how a folder looked at some point in time.
./bin/granthi-sync snapshots ~/work/notes
./bin/granthi-sync restore ~/work/notes --at 20260823T142530Z
```
Run `watch` as a background daemon on macOS with
@@ -67,6 +74,104 @@ no account, no token, no partial state. Just run `link` again.
to merge; when a folder shows `DIVERGED` in `status`, resolve it in git or on
the forge web UI. The client will never force or auto-merge your work.
## Two modes, because two very different folders ask for this
A folder people sync is either *their documents* or *their git project*, and
the correct behaviour is opposite in each case. Each linked folder therefore
carries a `mode`.
| | `mirror` | `snapshot` |
|---|---|---|
| chosen for | a plain folder `add` turned into a repo | a folder that was already a git repo, and anything `get` clones |
| commits on your behalf | yes, `sync: <ISO ts>` | **never** |
| where work lands | the branch | `refs/granthi-backup/<device>/<ts>` |
| a restore point is | every commit | every snapshot |
`snapshot` mode is what "the work may not be committed, but it is still
backed up" means in git terms. Each pass loads a scratch index from HEAD,
stages the working tree into *that* index, writes a tree, and commits it with
`commit-tree`. HEAD, your index, your stash and every file on disk are
untouched — you can be mid-rebase with a dirty tree and the backup still
records exactly what is on the disk right now. The user's history stays the
user's.
Why a custom ref namespace: verified on the beta forge (Gitea 1.27.2) that
`refs/granthi-backup/...` is accepted, is readable through `ls-remote`, and
does **not** appear in the branch list. Under `refs/heads` a machine taking a
backup every 30 seconds would bury the branches a person actually made.
Snapshots are parented on HEAD and deliberately **not** chained to the
previous snapshot: chaining would keep every old snapshot reachable from the
newest, so pruning a ref would free nothing and retention would be
decorative.
**Retention** (or 30-second backups become a disk leak nobody can navigate):
everything is kept for 24 h, then thinned to hourly for 7 days, then daily.
Pruning runs at most hourly, per device, and only over that device's own
refs. A ref whose timestamp this version cannot parse is **kept** — deleting
the unrecognised is how a backup system loses the one thing someone needed.
**Restore never writes over the working tree.** `restore` materialises a
restore point into a *new* directory and refuses a non-empty destination.
Someone restoring a backup is already having a bad day; overwriting the files
they still have would make the recovery tool the second disaster.
## The credential helper must be the ONLY helper (found by live QA)
`credential.helper` is a list that accumulates across system, global and repo
config, and git asks every helper in it. A stock mac already has two —
`osxkeychain` from Xcode's gitconfig, and `store` from many people's
`~/.gitconfig` — and they lose in both directions:
* **reading:** a stale entry for the forge host answers before our helper, so
pushes fail `remote: Failed to authenticate user` long after the token was
rotated, and nothing in this tool's config explains why. This is exactly
how the first live-QA run failed;
* **writing:** git calls `approve` on every helper after a successful auth,
so `store` copies the forge token into `~/.git-credentials` **in
plaintext**. Keeping the token in a 0600 file and out of remote URLs buys
nothing if git then hands it to a plaintext store.
So `install_credential_helper` (and the `git clone` in `get`) sets an **empty**
`credential.helper` first, which resets the inherited list, then adds ours.
Exactly one helper serves this repo.
Corollary worth remembering: a token embedded in a remote URL gets saved by
`store` on first use. During QA a verification clone with a URL-embedded
token re-created the very entry that had just been cleaned out. That is the
whole reason this client passes tokens through a helper and never a URL.
## Device identity
`link` mints a uuid on first run and persists it in `~/.granthi-sync/config.json`
as `device_id`, and sends it to `/v1/link`. Hostnames are neither stable
(people rename laptops) nor unique (every new mac is "Mac mini"), so a
hostname cannot key a backup ref or a device registry — two machines would
overwrite each other's snapshots. The service-side device registry is the
next phase; the client leads so the id already exists when it lands.
## What "add the computer to the network" means — and does not
The onboarding shape is: download → login → **the device is federated to the
account** → the device can reach its repos.
The middle step is a *device registration*, not a network membership. Those
sound like one step and must not be built as one: this estate's tailnet is a
single flat private network carrying the granthi VPS, aros-vps, the Shadow
box and the Mac. Putting a customer's laptop on it to let them sync a folder
would hand that laptop L3 reach to every piece of infrastructure we run.
So:
* **Our own machines** may join the tailnet — that is an operator action with
an operator's judgement behind it.
* **Customer devices never do.** Their transport is public HTTPS to
granthi-link and the forge through cloudflared. That is the same exposure
step already in the promotion window below, and it is what makes a genuinely
new computer able to onboard itself at all — today `link` only works from
inside the tailnet, which means "you can't set up a new computer without an
operator first" is the honest status.
## Components
### `server/granthi_link.py` — provisioning service (granthi VPS)
@@ -211,13 +316,32 @@ deleted it again, `DELETE …/tokens/{id}` returning 204 under basic auth):
(`authorization_pending`/`slow_down` handled), then calls `/v1/link`.
`--token` skips the device flow with a ready Zitadel token (headless/dev).
Result stored in `~/.granthi-sync/config.json` (0600).
* `list` — every repo the linked token can see, with the local folder each
is already synced to. Reads `GET /api/v1/user/repos` on the forge
* `list [pattern]` — every repo the linked token can see, with the local
folder each is already synced to. `pattern` narrows the table by name
(substring, or a glob like `work-*`), case-insensitively, against both
`owner/name` and the bare name. Filtering is display-only: the set already
came from the forge under this account's token. Reads
`GET /api/v1/user/repos` on the forge
**directly** with the scoped user token — no granthi-link round-trip, so
the read path needs no service change. Pagination is followed to a short
page; if the `FORGE_MAX_PAGES` guard trips, the output says the list is
incomplete rather than letting a bounded page read as the whole set.
* `get <repo|owner/repo> [--into DIR]` — the download half of `add`. Clones
* `get --all [--into DIR] [--mode M]` — clone every repo this account can
see, skipping the ones already linked here. **"Only the repos they are
granted" needs no client-side permission logic**: `/api/v1/user/repos` is
evaluated by the forge against this account's own scoped token, so the
list *is* the grant. A client-side filter would be a second opinion about
someone else's authorisation. One repo failing does not abandon the rest,
and a truncated listing is reported loudly — `--all` must never quietly
mean "the first 2000". `alice/notes` and `bob/notes` both want
`<base>/notes`; the second is cloned to `<base>/bob-notes` and the clash is
logged, because reporting it as "already present" would leave the user
believing they had pulled both.
* `get <repo|owner/repo> [--into DIR] [--mode M]` — the download half of
`add`. Defaults to `snapshot` mode unless the repo carries a
`.granthi-sync.json` marker saying otherwise, so a plain synced folder
behaves the same on the second machine while someone's real project is
never autocommitted onto. Clones
with `--origin granthi` (the remote name `watch` looks for) and
`-c credential.helper=…` (the repo does not exist yet, so the helper
cannot be installed first; git also persists it into the new config), then
@@ -225,22 +349,76 @@ deleted it again, `DELETE …/tokens/{id}` returning 204 under basic auth):
so a cloned repo is picked up by `watch` immediately. Refuses a non-empty
destination. Branch is read with `symbolic-ref` (an empty repo has an
unborn HEAD) and falls back to `main`.
* `add <folder> [--name N] [--private|--public]` — `git init -b main` if
needed, creates the cloud repo via `/v1/repos`, adds remote `granthi`,
initial commit + push. The token is delivered by a **git credential
helper** (the client's hidden `git-credential` subcommand reading the 0600
config) — never embedded in the remote URL (estate rule).
* `watch [--interval 30] [--once]` — per folder: autocommit
(`sync: <ISO ts>`) → fetch → ff-pull if remote strictly ahead → push if
local strictly ahead. **DIVERGED → log + record + SKIP. Never force, never
merge** — the same policy as the mesh. SIGTERM-clean.
* `status` — table of linked folders, last sync, divergence flags.
* `add <folder> [--name N] [--private|--public] [--mode M] [--force]` —
`git init -b main` if needed, creates the cloud repo via `/v1/repos`, adds
remote `granthi`, initial commit + push. The token is delivered by a **git
credential helper** (the client's hidden `git-credential` subcommand
reading the 0600 config) — never embedded in the remote URL (estate rule).
Pushes the folder's **current** branch, not a hardcoded `main`: an existing
repo may sit on `master` or a feature branch, and publishing that work
under the wrong name is not a cosmetic error.
Two guards, because `add -A` takes whatever it is given: a starter
`.gitignore` is seeded when the folder has none (an existing one is never
touched — it is the user's), and a folder over 20 000 files / 512 MB is
refused unless `--force`. The seeded ignore file covers `.env`, `*.key`,
`*.pem`, `id_rsa` and friends, and it governs snapshots too — the scratch
index honours `.gitignore` exactly as a normal commit does.
The size guard measures what git *would* sync, ignore rules included
(including the machine's global excludes), because its own advice is "add
a .gitignore for what should not sync" and advice that changes nothing is
worse than none. It asks git through a **throwaway git dir outside the
folder**, so a refused `add` leaves no `.git` behind in a directory the
user never agreed to turn into a repo.
* `watch [--interval 30] [--once]` — per folder, by mode. `mirror`:
autocommit (`sync: <ISO ts>`) → fetch → ff-pull if remote strictly ahead →
push if local strictly ahead. `snapshot`: fetch → push a snapshot of the
working tree to this device's backup ref → ff-pull only when the tree is
clean (local edits are already safe on the backup ref, so it reports and
leaves the tree alone rather than failing) → push the user's own commits
when they are strictly ahead. **DIVERGED → log + record + SKIP. Never
force, never merge** — the same policy as the mesh — **but the backup
still happens**, because divergence is when work is most at risk.
Retention pruning runs at most hourly. SIGTERM-clean.
* `snapshots <folder> [--limit 20]` — restore points, newest first, **across
every device**, with the device that took each one. Read from the
**remote**, not a local cache: the feature exists for the case where this
machine is gone.
The three scopes differ deliberately. Writing is device-scoped, so two
machines never overwrite each other. Pruning is device-scoped, so machine A
never applies its clock to machine B's refs. **Reading is not scoped** — a
replacement laptop has a new id, and scoping the read to it would print
"no restore points yet" while the backups sit on the forge. That defect
was live in the first draft and is now pinned by a test that restores a
dead machine's work from a fresh clone.
* `restore <folder> --at <ts|sha> [--into DIR]` — materialise one restore
point into a new directory; refuses a non-empty destination. Accepts what
`snapshots` printed in either mode, including a mirror-mode `%cI`
timestamp. Two commits inside the same second share that timestamp, so an
ambiguous `--at` is **refused with the candidate ids** rather than
resolved by guessing.
* `status` — table of linked folders, mode, last sync, divergence flags.
* Run as a daemon on macOS with `client/launchd/ai.granthi.sync.plist`
(edit the script path, then `launchctl bootstrap gui/$UID <plist>`).
## Tests
* `python3 -m unittest discover -s tests` — 65 tests: autocommit/ff/diverged
* `python3 -m unittest discover -s tests` — 153 tests. The v1.2 additions
cover: a snapshot capturing uncommitted work while HEAD, the index and the
working tree stay byte-identical; snapshots landing outside `refs/heads`;
an unchanged tree not being re-pushed; a diverged folder still being backed
up; a dirty tree blocking the ff-pull but not the backup; retention keeping
everything recent, thinning to hourly then daily, and **keeping**
unparseable timestamps; prune deleting only the thinned refs; `.gitignore`
seeding never overwriting an existing one and keeping `.env` out of
snapshots; the folder-size guard being bounded rather than walking the
disk; mode detection; `list` filtering; `get --all` skipping what is
already present, defaulting to snapshot mode, and shouting about
truncation; `restore` writing a new folder, refusing a non-empty
destination, and leaving the working tree alone; and the credential helper
being the only one the repo consults, proven by driving
`git credential fill` against a deliberately poisoned outer helper.
* Earlier suite: autocommit/ff/diverged
logic against real temp git repos (including "diverged never touches the
remote"), config 0600 handling (including umask-proof creation and a
no-chmod guard), credential-helper quoting/injection, mocked device-flow
@@ -267,6 +445,38 @@ the provisioning service creates their forge account + scoped token on the
fly — the forge never sees a password and the user never sees the forge admin.
Every linked folder becomes a private repo under their account.
## Next phase — invites and per-repo access (designed, not built)
Today `/v1/link` creates an account and every folder becomes a private repo
under it. What is missing is the multi-person case: an existing account
inviting somebody, and that person's device waking up with access to *some*
repos and not others.
Shape this should take, so the next session does not re-litigate it:
* **Where grants live: granthi-link's own store, not Zitadel orgs.** The
estate's house pattern is app-side tenancy tables with the IdP only
providing identity (see the AROS `tenants` / `tenant_members` split).
Grants therefore sit beside `state.json`, and **Gitea is the enforcement
point** — a grant is materialised as a repo collaborator or an org team
membership, so the forge itself refuses unauthorised reads. Nothing in the
client decides access, which is why `get --all` needs no permission logic.
* **Token scope does not change.** `write:repository,write:user` stays; per
repo permission is collaborator/team state, not a token property.
* **`POST /v1/invite`** (account admin → new member): creates the shre-id
user (Zitadel admin PAT, on aros-vps at
`/opt/shre-id/deploy/secrets/shre_id_zitadel_pat`), records the intended
grants, and returns an invite the person redeems by running `link`. Until
they redeem it, nothing exists on the forge.
* **`POST /v1/grants`** (account admin): add/remove repo access for a member;
applies the change to Gitea and records it. Removing a grant must also
remove the collaborator — a grant store that drifts from the forge is
worse than no store.
* Both endpoints are account-admin-only and rate-limited like `/v1/link`.
* The grant store inherits the same fragility already noted for the rate
limiter: a flat JSON file behind an in-process lock, fine for one
`ThreadingHTTPServer` and **not** fine the day this runs multi-process.
## Promotion window (beta → prod)
1. **Expose :3042** behind cloudflared (granthi.shre.ai vhost or
client/granthi_sync_client.py
+805 -41
View File
File diff suppressed because it is too large Load Diff
tests/test_client.py
+681 -9
View File
@@ -10,6 +10,7 @@ import shutil
import subprocess
import sys
import tempfile
import time
import unittest
from unittest import mock
@@ -33,6 +34,17 @@ def run_git(cwd, *args):
capture_output=True, text=True, env=GIT_ENV).stdout.strip()
def get_ns(**kw):
"""Namespace for cmd_get with the parser's defaults filled in, so a test
exercises the same shape argparse hands the command."""
kw.setdefault("all", False)
kw.setdefault("mode", None)
kw.setdefault("into", None)
kw.setdefault("repo", None)
return argparse.Namespace(**kw)
class GitScenarioBase(unittest.TestCase):
"""bare 'cloud' repo + two working clones to simulate device vs remote."""
@@ -341,7 +353,7 @@ class TestGet(GitScenarioBase):
def test_get_clones_registers_and_is_watchable(self):
dest = os.path.join(self.tmp, "pulled")
client.cmd_get(argparse.Namespace(repo="cloud", into=dest))
client.cmd_get(get_ns(repo="cloud", into=dest))
# cloned content
self.assertTrue(os.path.exists(os.path.join(dest, "hello.txt")))
@@ -366,7 +378,7 @@ class TestGet(GitScenarioBase):
def test_get_accepts_owner_qualified_name(self):
dest = os.path.join(self.tmp, "pulled2")
client.cmd_get(argparse.Namespace(repo="alice/cloud", into=dest))
client.cmd_get(get_ns(repo="alice/cloud", into=dest))
self.assertTrue(os.path.exists(os.path.join(dest, "hello.txt")))
def test_get_refuses_non_empty_destination(self):
@@ -374,14 +386,14 @@ class TestGet(GitScenarioBase):
os.makedirs(dest)
self.write(dest, "mine.txt", "do not clobber")
with self.assertRaises(SystemExit):
client.cmd_get(argparse.Namespace(repo="cloud", into=dest))
client.cmd_get(get_ns(repo="cloud", into=dest))
self.assertEqual(open(os.path.join(dest, "mine.txt")).read(),
"do not clobber")
def test_get_unlinked_exits_like_add(self):
with mock.patch.object(client, "load_config", lambda: {}):
with self.assertRaises(SystemExit) as cm:
client.cmd_get(argparse.Namespace(repo="cloud", into=None))
client.cmd_get(get_ns(repo="cloud", into=None))
self.assertIn("not linked", str(cm.exception))
def test_get_empty_repo_falls_back_to_main(self):
@@ -389,7 +401,7 @@ class TestGet(GitScenarioBase):
subprocess.run(["git", "init", "--bare", "-b", "main", empty],
check=True, capture_output=True, env=GIT_ENV)
dest = os.path.join(self.tmp, "blank")
client.cmd_get(argparse.Namespace(repo="blank", into=dest))
client.cmd_get(get_ns(repo="blank", into=dest))
meta = client.load_config()["folders"][os.path.abspath(dest)]
self.assertEqual(meta["branch"], "main")
@@ -399,17 +411,17 @@ class TestGet(GitScenarioBase):
"", "alice/"]:
with self.subTest(repo=bad):
with self.assertRaises(SystemExit):
client.cmd_get(argparse.Namespace(repo=bad, into=None))
client.cmd_get(get_ns(repo=bad, into=None))
def test_get_records_full_name_so_list_matches_the_right_owner(self):
dest = os.path.join(self.tmp, "pulled4")
client.cmd_get(argparse.Namespace(repo="cloud", into=dest))
client.cmd_get(get_ns(repo="cloud", into=dest))
meta = client.load_config()["folders"][os.path.abspath(dest)]
self.assertEqual(meta["full_name"], "alice/cloud")
def test_list_does_not_mark_a_same_named_other_owner_repo_as_local(self):
dest = os.path.join(self.tmp, "pulled5")
client.cmd_get(argparse.Namespace(repo="cloud", into=dest))
client.cmd_get(get_ns(repo="cloud", into=dest))
cfg = client.load_config()
repos = [{"name": "cloud", "full_name": "alice/cloud", "private": True,
"updated_at": "2026-08-20T00:00:00Z"},
@@ -426,9 +438,669 @@ class TestGet(GitScenarioBase):
def test_get_never_puts_token_in_remote_url(self):
dest = os.path.join(self.tmp, "pulled3")
client.cmd_get(argparse.Namespace(repo="cloud", into=dest))
client.cmd_get(get_ns(repo="cloud", into=dest))
self.assertNotIn("sekrit", run_git(dest, "remote", "get-url", "granthi"))
class TestSnapshots(GitScenarioBase):
"""The whole point of snapshot mode: work that was never committed still
leaves the machine, and the user's own history is not touched."""
DEV = "dev0123456789"
def test_snapshot_captures_uncommitted_work_without_moving_head(self):
self.write(self.local, "a.txt", "committed")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "real commit")
head_before = run_git(self.local, "rev-parse", "HEAD")
self.write(self.local, "a.txt", "UNCOMMITTED EDIT")
self.write(self.local, "new.txt", "never staged")
status_before = run_git(self.local, "status", "--porcelain")
# (run_git strips, so the leading space of ' M' is gone here)
self.assertEqual(status_before, "M a.txt\n?? new.txt")
commit, tree = client.build_snapshot(self.local)
self.assertEqual(run_git(self.local, "rev-parse", "HEAD"), head_before)
# the index is untouched: a.txt is still merely modified, not staged,
# and new.txt is still untracked. A snapshot that quietly staged the
# user's files would corrupt whatever they were in the middle of.
self.assertEqual(run_git(self.local, "status", "--porcelain"),
status_before)
# working tree still holds exactly what the user left there
with open(os.path.join(self.local, "a.txt")) as f:
self.assertEqual(f.read(), "UNCOMMITTED EDIT")
# ...and the snapshot commit holds it too
blob = run_git(self.local, "show", f"{commit}:a.txt")
self.assertEqual(blob, "UNCOMMITTED EDIT")
self.assertIn("new.txt", run_git(self.local, "ls-tree", "--name-only",
tree))
self.assertEqual(run_git(self.local, "log", "-1", "--format=%P",
commit), head_before)
def test_snapshot_is_none_when_nothing_is_uncommitted(self):
self.write(self.local, "a.txt", "one")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "c")
self.assertIsNone(client.build_snapshot(self.local))
def test_push_snapshot_lands_in_the_backup_namespace(self):
self.write(self.local, "a.txt", "one")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "c")
run_git(self.local, "push", "-u", "granthi", "main")
self.write(self.local, "a.txt", "work in progress")
ref = client.push_snapshot(self.local, self.DEV)
self.assertTrue(ref.startswith(f"refs/granthi-backup/{self.DEV}/"), ref)
refs = run_git(self.bare, "for-each-ref", "--format=%(refname)")
self.assertIn(ref, refs.splitlines())
# it is NOT a branch: the user's branch list stays theirs
self.assertNotIn("refs/heads/granthi-backup", refs)
def test_push_snapshot_skips_an_unchanged_tree(self):
self.write(self.local, "a.txt", "one")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "c")
run_git(self.local, "push", "-u", "granthi", "main")
self.write(self.local, "a.txt", "work in progress")
self.assertIsNotNone(client.push_snapshot(self.local, self.DEV))
self.assertIsNone(client.push_snapshot(self.local, self.DEV))
def test_snapshot_mode_never_commits_for_the_user(self):
self.write(self.local, "a.txt", "one")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "mine")
run_git(self.local, "push", "-u", "granthi", "main")
head_before = run_git(self.local, "rev-parse", "HEAD")
self.write(self.local, "a.txt", "dirty")
outcome, detail = client.sync_folder(self.local, mode="snapshot",
dev=self.DEV)
self.assertEqual(run_git(self.local, "rev-parse", "HEAD"), head_before)
self.assertIn("backed up", detail)
self.assertEqual(outcome, "clean")
self.assertTrue(run_git(self.local, "status", "--porcelain"))
def test_diverged_folder_is_still_backed_up(self):
"""Divergence is when work is most at risk -- the least acceptable
moment to skip the backup."""
self.write(self.local, "a.txt", "one")
client.sync_folder(self.local) # mirror push to establish the branch
other = self.other_clone()
self.write(other, "b.txt", "remote side")
run_git(other, "add", "-A")
run_git(other, "commit", "-m", "remote")
run_git(other, "push", "origin", "main")
remote_sha = run_git(other, "rev-parse", "HEAD")
self.write(self.local, "a.txt", "local side")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "local")
self.write(self.local, "c.txt", "and uncommitted too")
outcome, detail = client.sync_folder(self.local, mode="snapshot",
dev=self.DEV)
self.assertEqual(outcome, "diverged")
self.assertIn("backed up", detail)
self.assertEqual(run_git(self.bare, "rev-parse", "main"), remote_sha)
snaps = client.list_snapshots(self.local, self.DEV)
self.assertEqual(len(snaps), 1)
self.assertIn("c.txt", run_git(self.local, "ls-tree", "--name-only",
snaps[0]["sha"]))
def test_dirty_tree_blocks_the_pull_but_not_the_backup(self):
self.write(self.local, "a.txt", "one")
client.sync_folder(self.local)
other = self.other_clone()
self.write(other, "b.txt", "remote side")
run_git(other, "add", "-A")
run_git(other, "commit", "-m", "remote")
run_git(other, "push", "origin", "main")
self.write(self.local, "wip.txt", "half-finished")
head_before = run_git(self.local, "rev-parse", "HEAD")
outcome, detail = client.sync_folder(self.local, mode="snapshot",
dev=self.DEV)
self.assertEqual(run_git(self.local, "rev-parse", "HEAD"), head_before)
self.assertFalse(os.path.exists(os.path.join(self.local, "b.txt")))
self.assertIn("not pulling", detail)
self.assertIn("backed up", detail)
def test_snapshots_are_listed_from_the_remote(self):
self.write(self.local, "a.txt", "one")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "c")
run_git(self.local, "push", "-u", "granthi", "main")
self.write(self.local, "a.txt", "wip")
ref = client.push_snapshot(self.local, self.DEV)
snaps = client.list_snapshots(self.local, self.DEV)
self.assertEqual([s["ref"] for s in snaps], [ref])
# writing is device-scoped: the ref carries THIS device's id, so two
# machines cannot overwrite each other
self.assertEqual(snaps[0]["device"], self.DEV)
self.assertEqual(client.list_snapshots(self.local, "someone-else"), [])
def test_a_new_machine_can_see_the_dead_machine_s_backups(self):
"""The case the whole feature exists for. Reads must NOT be scoped to
this device's id -- a replacement laptop has a new id, and scoping
would show an empty list while the backups sit on the forge."""
self.write(self.local, "a.txt", "committed")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "c")
run_git(self.local, "push", "-u", "granthi", "main")
self.write(self.local, "a.txt", "PRECIOUS UNCOMMITTED WORK")
ref = client.push_snapshot(self.local, "laptop-that-died")
# a fresh clone standing in for the replacement machine
newbox = os.path.join(self.tmp, "newbox")
subprocess.run(["git", "clone", "-q", "--origin", "granthi",
self.bare, newbox], check=True, capture_output=True,
env=GIT_ENV)
client.save_config({"gitea_base": self.tmp, "login": "alice",
"token": "t", "device_id": "brand-new-laptop",
"folders": {newbox: {"name": "cloud",
"full_name": "alice/cloud",
"branch": "main",
"mode": "snapshot"}}})
seen = client.list_snapshots(newbox)
self.assertEqual([s["ref"] for s in seen], [ref])
self.assertEqual(seen[0]["device"], "laptop-that-died")
# and it can actually restore it
dest = os.path.join(self.tmp, "recovered")
client.cmd_restore(argparse.Namespace(folder=newbox, at=seen[0]["ts"],
into=dest))
with open(os.path.join(dest, "a.txt")) as f:
self.assertEqual(f.read(), "PRECIOUS UNCOMMITTED WORK")
class TestRetention(unittest.TestCase):
"""Retention is what keeps 30-second backups from being a disk leak --
and what must never quietly eat the one restore point someone needs."""
NOW = client.datetime(2026, 8, 23, 12, 0, 0, tzinfo=client.timezone.utc)
def snap(self, when):
ts = when.strftime(client.SNAPSHOT_TS_FMT)
return {"ts": ts, "ref": f"refs/granthi-backup/d/{ts}", "sha": "x"}
def test_everything_recent_is_kept(self):
snaps = [self.snap(self.NOW - client.timedelta(minutes=m))
for m in range(0, 24 * 60, 30)]
self.assertEqual(client.snapshots_to_prune(snaps, now=self.NOW), [])
def test_older_than_a_day_thins_to_hourly(self):
base = self.NOW - client.timedelta(days=2)
snaps = [self.snap(base + client.timedelta(minutes=m))
for m in (0, 10, 20, 60, 70)]
pruned = client.snapshots_to_prune(snaps, now=self.NOW)
self.assertEqual(len(pruned), 3) # 5 in 2 hourly buckets -> keep 2
def test_older_than_a_week_thins_to_daily(self):
base = self.NOW - client.timedelta(days=30)
snaps = [self.snap(base + client.timedelta(hours=h))
for h in (0, 1, 2, 25)]
pruned = client.snapshots_to_prune(snaps, now=self.NOW)
self.assertEqual(len(pruned), 2) # 2 days -> keep 1 each
def test_unparseable_timestamps_are_kept_not_deleted(self):
snaps = [{"ts": "not-a-timestamp",
"ref": "refs/granthi-backup/d/not-a-timestamp", "sha": "x"}]
self.assertEqual(client.snapshots_to_prune(snaps, now=self.NOW), [])
class TestPrune(GitScenarioBase):
DEV = "devprune"
def test_prune_deletes_stale_refs_on_the_remote(self):
self.write(self.local, "a.txt", "one")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "c")
run_git(self.local, "push", "-u", "granthi", "main")
sha = run_git(self.local, "rev-parse", "HEAD")
old = "20260101T000000Z"
older = "20260101T001000Z"
for ts in (old, older):
run_git(self.local, "push", "granthi",
f"{sha}:refs/granthi-backup/{self.DEV}/{ts}")
self.assertEqual(len(client.list_snapshots(self.local, self.DEV)), 2)
gone = client.prune_snapshots(self.local, self.DEV)
self.assertEqual(gone, 1) # same hour, older one dropped
left = client.list_snapshots(self.local, self.DEV)
self.assertEqual([s["ts"] for s in left], [older])
class TestAddGuards(GitScenarioBase):
def test_gitignore_is_seeded_only_when_absent(self):
self.assertTrue(client.seed_gitignore(self.local))
with open(os.path.join(self.local, ".gitignore")) as f:
body = f.read()
self.assertIn(".env", body)
with open(os.path.join(self.local, ".gitignore"), "w") as f:
f.write("mine-only\n")
self.assertFalse(client.seed_gitignore(self.local))
with open(os.path.join(self.local, ".gitignore")) as f:
self.assertEqual(f.read(), "mine-only\n")
def test_seeded_gitignore_keeps_secrets_out_of_snapshots(self):
client.seed_gitignore(self.local)
self.write(self.local, ".env", "SECRET=hunter2")
self.write(self.local, "ok.txt", "fine")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "c")
self.write(self.local, "ok.txt", "changed")
commit, tree = client.build_snapshot(self.local)
names = run_git(self.local, "ls-tree", "-r", "--name-only", tree)
self.assertIn("ok.txt", names)
self.assertNotIn(".env", names.splitlines())
def test_measure_folder_stops_counting_past_the_cap(self):
for i in range(12):
self.write(self.local, f"f{i}.txt", "x" * 10)
files, size = client.measure_folder(self.local, max_files=5)
self.assertEqual(files, 6) # bounded: stopped one past the cap
self.assertLess(size, 12 * 10)
def test_measure_folder_ignores_dot_git(self):
files, _ = client.measure_folder(self.local)
self.assertEqual(files, 0)
def test_detect_mode(self):
plain = os.path.join(self.tmp, "plain")
os.makedirs(plain)
self.assertEqual(client.detect_mode(plain, had_git=False), "mirror")
self.write(self.local, "a.txt", "one")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "real history")
self.assertEqual(client.detect_mode(self.local, had_git=True),
"snapshot")
empty = os.path.join(self.tmp, "empty-repo")
os.makedirs(empty)
client.ensure_repo(empty)
self.assertEqual(client.detect_mode(empty, had_git=True), "mirror")
class TestMatchRepo(unittest.TestCase):
def repo(self, full):
return {"full_name": full, "name": full.split("/")[-1]}
def test_substring_is_case_insensitive_and_matches_bare_name(self):
self.assertTrue(client.match_repo(self.repo("alice/Notes"), "notes"))
self.assertTrue(client.match_repo(self.repo("alice/notes"), "ALICE"))
self.assertFalse(client.match_repo(self.repo("alice/notes"), "ledger"))
def test_glob_syntax_switches_to_glob(self):
self.assertTrue(client.match_repo(self.repo("alice/work-2026"),
"work-*"))
self.assertFalse(client.match_repo(self.repo("alice/homework"),
"work-*"))
def test_empty_pattern_matches_everything(self):
self.assertTrue(client.match_repo(self.repo("alice/x"), None))
class TestRestore(GitScenarioBase):
DEV = "devrestore"
def setUp(self):
super().setUp()
client.save_config({"gitea_base": self.tmp, "login": "alice",
"token": "sekrit", "device_id": self.DEV,
"folders": {self.local: {
"name": "cloud", "full_name": "alice/cloud",
"branch": "main", "mode": "snapshot"}}})
def test_restore_writes_a_new_folder_and_leaves_the_working_tree_alone(self):
self.write(self.local, "a.txt", "original")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "c")
run_git(self.local, "push", "-u", "granthi", "main")
self.write(self.local, "a.txt", "the version I want back")
ref = client.push_snapshot(self.local, self.DEV)
ts = ref.rsplit("/", 1)[-1]
self.write(self.local, "a.txt", "what I have now")
dest = os.path.join(self.tmp, "restored")
client.cmd_restore(argparse.Namespace(folder=self.local, at=ts,
into=dest))
with open(os.path.join(dest, "a.txt")) as f:
self.assertEqual(f.read(), "the version I want back")
with open(os.path.join(self.local, "a.txt")) as f:
self.assertEqual(f.read(), "what I have now")
def test_restore_refuses_a_non_empty_destination(self):
self.write(self.local, "a.txt", "one")
run_git(self.local, "add", "-A")
run_git(self.local, "commit", "-m", "c")
run_git(self.local, "push", "-u", "granthi", "main")
self.write(self.local, "a.txt", "two")
ref = client.push_snapshot(self.local, self.DEV)
busy = os.path.join(self.tmp, "busy")
os.makedirs(busy)
with open(os.path.join(busy, "keepme"), "w") as f:
f.write("do not clobber")
with self.assertRaises(SystemExit):
client.cmd_restore(argparse.Namespace(
folder=self.local, at=ref.rsplit("/", 1)[-1], into=busy))
self.assertTrue(os.path.exists(os.path.join(busy, "keepme")))
def test_unknown_restore_point_is_an_error_not_an_empty_folder(self):
with self.assertRaises(SystemExit):
client.cmd_restore(argparse.Namespace(
folder=self.local, at="20990101T000000Z", into=None))
def test_restore_refuses_a_folder_that_is_not_linked(self):
with self.assertRaises(SystemExit):
client.cmd_restore(argparse.Namespace(
folder=os.path.join(self.tmp, "nowhere"), at="x", into=None))
class TestDeviceId(unittest.TestCase):
def test_device_id_is_stable_and_persisted(self):
cfg = {}
first = client.device_id(cfg)
self.assertEqual(client.device_id(cfg), first)
self.assertEqual(cfg["device_id"], first)
def test_two_installs_get_different_ids(self):
self.assertNotEqual(client.device_id({}), client.device_id({}))
class TestGetAll(GitScenarioBase):
"""--all must pull exactly what the forge grants, and one bad repo must
not abandon the rest."""
def setUp(self):
super().setUp()
self.forge = os.path.join(self.tmp, "forge")
for full in ("alice/one", "alice/two"):
path = os.path.join(self.forge, full + ".git")
os.makedirs(os.path.dirname(path), exist_ok=True)
subprocess.run(["git", "init", "--bare", "-b", "main", path],
check=True, capture_output=True, env=GIT_ENV)
seed = os.path.join(self.tmp, "seed-" + full.replace("/", "-"))
subprocess.run(["git", "clone", path, seed], check=True,
capture_output=True, env=GIT_ENV)
run_git(seed, "config", "user.name", "s")
run_git(seed, "config", "user.email", "s@s")
self.write(seed, "f.txt", full)
run_git(seed, "add", "-A")
run_git(seed, "commit", "-m", "seed")
run_git(seed, "push", "origin", "main")
client.save_config({"gitea_base": self.forge, "login": "alice",
"token": "sekrit", "folders": {}})
self.repos = [{"name": "one", "full_name": "alice/one"},
{"name": "two", "full_name": "alice/two"}]
def test_all_clones_every_granted_repo(self):
into = os.path.join(self.tmp, "workspace")
os.makedirs(into)
with mock.patch.object(client, "list_repos",
lambda c: (self.repos, False)):
rc = client.cmd_get(get_ns(all=True, into=into))
self.assertEqual(rc, 0)
for name in ("one", "two"):
self.assertTrue(os.path.exists(os.path.join(into, name, "f.txt")))
self.assertEqual(len(client.load_config()["folders"]), 2)
def test_all_skips_what_is_already_here(self):
into = os.path.join(self.tmp, "workspace2")
os.makedirs(into)
with mock.patch.object(client, "list_repos",
lambda c: (self.repos, False)):
client.cmd_get(get_ns(all=True, into=into))
with mock.patch("sys.stdout", new_callable=io.StringIO) as out:
client.cmd_get(get_ns(all=True, into=into))
self.assertIn("already present", out.getvalue())
self.assertEqual(len(client.load_config()["folders"]), 2)
def test_all_defaults_cloned_repos_to_snapshot_mode(self):
into = os.path.join(self.tmp, "workspace3")
os.makedirs(into)
with mock.patch.object(client, "list_repos",
lambda c: (self.repos, False)):
client.cmd_get(get_ns(all=True, into=into))
modes = {m["mode"] for m in client.load_config()["folders"].values()}
self.assertEqual(modes, {"snapshot"})
def test_all_says_so_loudly_when_the_listing_was_truncated(self):
into = os.path.join(self.tmp, "workspace4")
os.makedirs(into)
with mock.patch.object(client, "list_repos",
lambda c: (self.repos, True)), \
mock.patch("sys.stdout", new_callable=io.StringIO) as out:
client.cmd_get(get_ns(all=True, into=into))
self.assertIn("NOT every repo", out.getvalue())
class TestMarkerRoundTrip(GitScenarioBase):
"""A plain folder synced on machine A must behave the same on machine B:
the intent travels in the repo, not in one machine's config."""
def test_marker_written_by_add_makes_get_choose_mirror(self):
client.write_marker(self.local, "mirror")
self.assertEqual(client.read_marker(self.local)["mode"], "mirror")
def test_missing_or_corrupt_marker_falls_back_to_the_safe_mode(self):
self.assertEqual(client.read_marker(self.local), {})
with open(os.path.join(self.local, client.MARKER_FILE), "w") as f:
f.write("{not json")
self.assertEqual(client.read_marker(self.local), {})
class TestCredentialHelperIsolation(GitScenarioBase):
"""A repo-local helper is not enough on a normal machine: git consults
system + global helpers too, and they both shadow us and copy the token
into plaintext. Found by live QA against the beta forge, not by a unit
test -- so it gets one now."""
def test_install_leaves_exactly_one_helper(self):
run_git(self.local, "config", "--add", "credential.helper", "store")
client.install_credential_helper(self.local)
# --get-all merges system + global + local, so entries inherited from
# the machine still appear. What matters is that the last two are the
# reset and ours: git reads an empty value as "forget every helper
# inherited so far", so nothing before it can answer.
helpers = run_git(self.local, "config", "--get-all",
"credential.helper").splitlines()
self.assertEqual(helpers[-2], "", helpers)
self.assertIn("git-credential", helpers[-1])
# the repo-level 'store' this test added is gone, not merely outvoted
self.assertNotIn("store", helpers)
def test_inherited_helper_cannot_answer_for_the_forge(self):
"""The end-to-end property: with a poisoned outer helper configured,
the credential git actually resolves is ours."""
fake = os.path.join(self.tmp, "poison.sh")
with open(fake, "w") as f:
f.write("#!/bin/sh\n"
"echo username=wrong-user\necho password=stale-token\n")
os.chmod(fake, 0o755)
run_git(self.local, "config", "--add", "credential.helper",
f"!{shlex.quote(fake)}")
client.save_config({"gitea_base": "http://forge.example:3041",
"login": "alice", "token": "the-right-token"})
client.install_credential_helper(self.local)
out = subprocess.run(
["git", "-C", self.local, "credential", "fill"],
input="protocol=http\nhost=forge.example:3041\n\n",
capture_output=True, text=True, env=dict(
GIT_ENV, GRANTHI_SYNC_HOME=os.environ["GRANTHI_SYNC_HOME"]))
self.assertIn("password=the-right-token", out.stdout)
self.assertNotIn("stale-token", out.stdout)
class TestMirrorRestore(GitScenarioBase):
"""Mirror is the default for a plain folder, so its restore path is the
one most people will use -- and the timestamp `snapshots` prints has to
be a timestamp `restore` accepts, or the user just loops."""
def setUp(self):
super().setUp()
client.save_config({"gitea_base": self.tmp, "login": "alice",
"token": "t", "device_id": "d1",
"folders": {self.local: {
"name": "cloud", "full_name": "alice/cloud",
"branch": "main", "mode": "mirror"}}})
def test_restore_accepts_the_timestamp_snapshots_printed(self):
self.write(self.local, "a.txt", "the version I want back")
client.sync_folder(self.local, mode="mirror")
time.sleep(1.1) # %cI has one-second resolution
self.write(self.local, "a.txt", "later junk")
client.sync_folder(self.local, mode="mirror")
with mock.patch("sys.stdout", new_callable=io.StringIO) as out:
client.cmd_snapshots(argparse.Namespace(folder=self.local,
limit=20))
listed = [l.split() for l in out.getvalue().splitlines()
if l.startswith(" ")]
wanted_ts = listed[-1][0] # first column of the oldest entry
dest = os.path.join(self.tmp, "mirror-restore")
client.cmd_restore(argparse.Namespace(folder=self.local,
at=wanted_ts, into=dest))
with open(os.path.join(dest, "a.txt")) as f:
self.assertEqual(f.read(), "the version I want back")
def test_two_commits_in_the_same_second_are_refused_not_guessed(self):
"""%cI has one-second resolution. Picking one silently would restore
something the user did not choose."""
self.write(self.local, "a.txt", "first")
client.sync_folder(self.local, mode="mirror")
self.write(self.local, "a.txt", "second")
client.sync_folder(self.local, mode="mirror")
stamps = run_git(self.local, "log", "--format=%cI").splitlines()
if len(set(stamps)) != 1:
self.skipTest("commits did not land in the same second")
with self.assertRaises(SystemExit) as caught:
client.cmd_restore(argparse.Namespace(folder=self.local,
at=stamps[0], into=None))
self.assertIn("matches 2 restore points", str(caught.exception))
def test_restore_still_accepts_a_sha(self):
self.write(self.local, "a.txt", "one")
client.sync_folder(self.local, mode="mirror")
sha = run_git(self.local, "rev-parse", "HEAD")
dest = os.path.join(self.tmp, "by-sha")
client.cmd_restore(argparse.Namespace(folder=self.local, at=sha,
into=dest))
self.assertTrue(os.path.exists(os.path.join(dest, "a.txt")))
def test_default_destination_is_a_usable_path(self):
self.write(self.local, "a.txt", "one")
client.sync_folder(self.local, mode="mirror")
with mock.patch("sys.stdout", new_callable=io.StringIO) as out:
client.cmd_snapshots(argparse.Namespace(folder=self.local,
limit=5))
ts = [l.split() for l in out.getvalue().splitlines()
if l.startswith(" ")][0][0]
client.cmd_restore(argparse.Namespace(folder=self.local, at=ts,
into=None))
made = [d for d in os.listdir(self.tmp) if d.startswith("local-restore-")]
self.assertEqual(len(made), 1, made)
self.assertNotIn(":", made[0])
class TestMeasureHonoursGitignore(GitScenarioBase):
"""The guard tells people to add a .gitignore. That advice has to work."""
def test_ignored_files_are_not_counted(self):
os.makedirs(os.path.join(self.local, "bulkdata"))
for i in range(30):
self.write(self.local, f"bulkdata/x{i}.bin", "y" * 100)
self.write(self.local, "real.txt", "mine")
before, _ = client.measure_folder(self.local)
self.write(self.local, ".gitignore", "bulkdata/\n")
after, _ = client.measure_folder(self.local)
self.assertGreater(before, 30)
self.assertEqual(after, 2) # real.txt + .gitignore
def test_measure_leaves_no_git_dir_behind(self):
plain = os.path.join(self.tmp, "untouched")
os.makedirs(plain)
self.write(plain, "a.txt", "x")
client.measure_folder(plain)
self.assertEqual(os.listdir(plain), ["a.txt"])
class TestGetAllRobustness(GitScenarioBase):
def setUp(self):
super().setUp()
self.forge = os.path.join(self.tmp, "forge")
for full in ("alice/notes", "bob/notes"):
path = os.path.join(self.forge, full + ".git")
os.makedirs(os.path.dirname(path), exist_ok=True)
subprocess.run(["git", "init", "-q", "--bare", "-b", "main", path],
check=True, capture_output=True, env=GIT_ENV)
seed = os.path.join(self.tmp, "seed-" + full.replace("/", "-"))
subprocess.run(["git", "clone", "-q", path, seed], check=True,
capture_output=True, env=GIT_ENV)
run_git(seed, "config", "user.name", "s")
run_git(seed, "config", "user.email", "s@s")
self.write(seed, "who.txt", full)
run_git(seed, "add", "-A")
run_git(seed, "commit", "-m", "seed")
run_git(seed, "push", "-q", "origin", "main")
client.save_config({"gitea_base": self.forge, "login": "alice",
"token": "t", "folders": {}})
self.repos = [{"name": "notes", "full_name": "alice/notes"},
{"name": "notes", "full_name": "bob/notes"}]
def test_same_named_repos_from_two_owners_both_land(self):
into = os.path.join(self.tmp, "ws")
os.makedirs(into)
with mock.patch.object(client, "list_repos",
lambda c: (self.repos, False)), \
mock.patch("sys.stdout", new_callable=io.StringIO) as out:
client.cmd_get(get_ns(all=True, into=into))
self.assertIn("name clash", out.getvalue())
with open(os.path.join(into, "notes", "who.txt")) as f:
self.assertEqual(f.read(), "alice/notes")
with open(os.path.join(into, "bob-notes", "who.txt")) as f:
self.assertEqual(f.read(), "bob/notes")
self.assertEqual(len(client.load_config()["folders"]), 2)
def test_a_git_failure_on_one_repo_does_not_abandon_the_rest(self):
into = os.path.join(self.tmp, "ws2")
os.makedirs(into)
real_clone = client.clone_one
def flaky(cfg, full_name, dest, mode=None):
if full_name == "alice/notes":
raise RuntimeError("git clone failed: pretend network blip")
return real_clone(cfg, full_name, dest, mode)
with mock.patch.object(client, "list_repos",
lambda c: (self.repos, False)), \
mock.patch.object(client, "clone_one", flaky), \
mock.patch("sys.stdout", new_callable=io.StringIO) as out:
rc = client.cmd_get(get_ns(all=True, into=into))
self.assertEqual(rc, 1) # reported, not hidden
self.assertIn("FAILED alice/notes", out.getvalue())
self.assertTrue(os.path.exists(os.path.join(into, "notes", "who.txt")))
class TestPruneClockIsPersisted(GitScenarioBase):
def test_watch_once_does_not_prune_every_run(self):
client.save_config({"gitea_base": self.tmp, "login": "alice",
"token": "t", "device_id": "d1", "folders": {}})
with mock.patch.object(client, "prune_snapshots") as pruner:
client.watch_pass(now=1_000_000.0)
self.assertEqual(client.load_config()["last_prune"], 1_000_000.0)
client.watch_pass(now=1_000_060.0) # a minute later: not due
client.watch_pass(now=1_003_700.0) # an hour later: due again
self.assertEqual(client.load_config()["last_prune"], 1_003_700.0)
self.assertEqual(pruner.call_count, 0) # no snapshot folders linked
if __name__ == "__main__":
unittest.main()