nirpa/granthi-sync
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(link): rate-limit /v1/link and /v1/repos

Browse Source 
Last unbuilt item on the promotion-window hardening checklist. /v1/link
round-trips Zitadel, can CREATE a forge account and always mints a token, so
it is the endpoint that must not be free to hammer.

Sliding window per (route, client), ON by default -- unlimited has to be a
deliberate config act, not an omission. Defaults 5/hour and 60/hour; /health
never limited. 429 + Retry-After, decided BEFORE the body is read so an
abusive caller costs nothing.

Decisions worth naming:
- State is an in-process dict behind a lock. granthi-link is ONE
  ThreadingHTTPServer, so that IS the store -- no redis. Kept behind a class
  so a future multi-process move has one thing to change.
- Denied requests are NOT recorded. Recording them lets a hammering client
  push its own window forward and lock itself out forever.
- Key store is capped; at capacity it drops least-recent windows and logs
  loudly. Fail-open under key pressure, chosen over an unbounded dict that
  is a memory DoS.
- trust_forwarded_for OFF by default. Behind cloudflared every request comes
  from the tunnel, so limiting on the socket peer starves everyone; but XFF
  is client-controlled. A caller can PREPEND, a trusted proxy APPENDS what it
  actually saw -- so we read the LAST entry, never the first.
- A malformed rule refuses startup instead of silently meaning unlimited.

Tests 69 -> 87, including a 40-thread race proving the lock holds, the
self-lockout case, XFF spoof-resistance, and a real 429 on the wire.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LTARYHX7GPepi3CH3tp5pg
This commit is contained in:
Nirav Patel
2026-08-22 23:14:54 -04:00
co-authored by Claude Opus 5
parent 137d2cdd53
commit e2fed5886f
4 changed files with 351 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
server/granthi_link.py
+135
View File
@@ -61,6 +61,7 @@ import signal
import string
import sys
import threading
import time
import urllib.error
import urllib.request
from datetime import datetime, timezone
@@ -79,6 +80,93 @@ USER_CREATE_CONFLICT = object()
LOGIN_SAFE = re.compile(r"[^a-zA-Z0-9._-]+")
# Rate limiting. /v1/link is the expensive endpoint -- it round-trips Zitadel,
# can CREATE a forge account and always mints a token -- so its default is
# deliberately tight. /v1/repos only spends the caller's own token.
DEFAULT_RATE_RULES = {"/v1/link": (5, 3600), "/v1/repos": (60, 3600)}
MAX_RATE_KEYS = 10000
# --------------------------------------------------------------------------
# Rate limiting
# --------------------------------------------------------------------------
class RateLimiter:
"""Sliding-window limiter keyed by (route, client).
In-process and lock-guarded: granthi-link is ONE ThreadingHTTPServer
process, so a dict is the entire store -- no redis, no shared cache, and
nothing to keep consistent across nodes. If this ever runs multi-process
the limiter must move with it; that is why the store is behind this class
rather than sprinkled through the handler.
"""
def __init__(self, rules=None, max_keys=MAX_RATE_KEYS, clock=None):
self.rules = dict(rules or DEFAULT_RATE_RULES)
self.max_keys = max_keys
self._clock = clock or time.monotonic
self._hits = {}
self._lock = threading.Lock()
def check(self, route, client):
"""-> (allowed: bool, retry_after: int). Records the hit when allowed.
Denied requests are NOT recorded: a client that keeps hammering must
not push its own window forward and lock itself out indefinitely.
"""
rule = self.rules.get(route)
if not rule:
return True, 0
limit, window = rule
if limit <= 0: # 0 = endpoint disabled entirely
return False, window
now = self._clock()
key = (route, client)
with self._lock:
hits = [t for t in self._hits.get(key, ()) if now - t < window]
if len(hits) >= limit:
self._hits[key] = hits
return False, max(1, int(window - (now - hits[0])) + 1)
hits.append(now)
self._hits[key] = hits
if len(self._hits) > self.max_keys:
self._prune(now)
return True, 0
def _prune(self, now):
"""Caller holds the lock. Drop expired keys; if still over the cap,
drop the least-recently-hit. Dropping RESETS those clients' windows,
which is fail-open -- but an unbounded dict is a memory DoS, and a
bounded one is the lesser failure. Say so loudly either way."""
for k in [k for k, v in self._hits.items()
if not v or now - v[-1] >= self.rules[k[0]][1]]:
del self._hits[k]
if len(self._hits) > self.max_keys:
over = len(self._hits) - self.max_keys
for k in sorted(self._hits, key=lambda k: self._hits[k][-1])[:over]:
del self._hits[k]
LOG.warning("rate limiter at capacity (%d keys): dropped %d "
"least-recent windows; those clients start fresh",
self.max_keys, over)
def client_ip(handler, trust_forwarded_for):
"""The address to rate-limit on.
Behind cloudflared every request arrives from the tunnel, so limiting on
the socket peer would let one abuser starve everyone. X-Forwarded-For is
client-controlled, though: a caller can prepend anything. A trusted proxy
APPENDS the peer it actually saw, so the LAST entry is the only one we did
not let the client choose -- take that, never the first. Off by default;
only enable when something trustworthy really is in front.
"""
if trust_forwarded_for:
xff = handler.headers.get("X-Forwarded-For", "")
parts = [p.strip() for p in xff.split(",") if p.strip()]
if parts:
return parts[-1]
return handler.client_address[0]
# --------------------------------------------------------------------------
# HTTP helper (patchable in tests)
@@ -203,6 +291,31 @@ class LinkService:
"zitadel_userinfo", "https://id.shre.ai/oidc/v1/userinfo")
self.state = IdentityStore(config.get("state_path", DEFAULT_STATE))
# Rate limiting is ON by default: this endpoint creates accounts and
# mints tokens, so the safe default is limited, and disabling it has
# to be a deliberate config act rather than an omission.
rl = config.get("rate_limit") or {}
self.trust_forwarded_for = bool(rl.get("trust_forwarded_for", False))
if rl.get("enabled", True):
rules = dict(DEFAULT_RATE_RULES)
for route, spec in (rl.get("rules") or {}).items():
if (not isinstance(spec, (list, tuple)) or len(spec) != 2
or not all(isinstance(x, int) for x in spec)
or spec[1] <= 0):
# A malformed rule must not silently mean "unlimited".
raise SystemExit(
f"config rate_limit.rules[{route!r}] must be "
f"[max_requests, window_seconds] with window > 0")
rules[route] = (spec[0], spec[1])
self.limiter = RateLimiter(rules)
LOG.info("rate limiting active: %s (trust_forwarded_for=%s)",
{k: f"{v[0]}/{v[1]}s" for k, v in rules.items()},
self.trust_forwarded_for)
else:
self.limiter = None
LOG.warning("rate limiting DISABLED by config -- /v1/link will "
"mint tokens without any throttle")
def test_mode_enabled(self):
"""Config test_mode is honored ONLY with the env gate also set."""
if not self.cfg.get("test_mode"):
@@ -452,6 +565,28 @@ class Handler(BaseHTTPRequestHandler):
self._send(404, {"error": "not found"})
def do_POST(self):
# Rate-limit BEFORE reading the body or doing any work -- the point is
# to spend nothing on an abusive caller. Same close-the-connection
# treatment the 413 path uses, for the same reason: we are not going
# to drain a body we already decided to reject.
limiter = getattr(self.service, "limiter", None)
if limiter is not None:
who = client_ip(self, self.service.trust_forwarded_for)
allowed, retry_after = limiter.check(self.path, who)
if not allowed:
LOG.warning("rate limited %s %s (retry after %ss)",
who, self.path, retry_after)
self.close_connection = True
payload = json.dumps({
"error": "rate limit exceeded",
"retry_after": retry_after}).encode()
self.send_response(429)
self.send_header("Content-Type", "application/json")
self.send_header("Retry-After", str(retry_after))
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
return
cl = self.headers.get("Content-Length")
if cl is None:
self.close_connection = True