Repository navigation
Woltspace 0.6: the Mill for apps, ways in, a lodge at its own address (sites stay put) - #663
Open
woltspace-jerpint[bot] wants to merge 84 commits into
Open
woltspace-jerpint[bot] wants to merge 84 commits into
woltspace-jerpint[bot] wants to merge 84 commits into
Conversation
The app proxy forwarded every request header to the app, including the Cloudflare Access token and cookie, so an app could replay the owner's login. It now strips Access credentials (headers and cookies) both ways, drops hop-by-hop headers, keeps repeated headers and every Set-Cookie, and makes app cookies host-only. The app's own Authorization header and cookies still pass through. Request bodies stream to the app instead of being read whole, responses are relayed raw so compressed replies keep a correct length, and app WebSockets carry binary frames as well as text, with both sides closed when either ends.
Access can read a service token from a configurable header, Authorization included, as JSON with the client id and secret, and accepts its JWT as a bearer token. Either shape is now stripped before the request reaches an app, on HTTP and WebSocket. Any other Authorization value is the app's own and still passes.
Managed OAuth issues opaque Access tokens sent as `Bearer oauth:...`. They are now stripped like the other Access credential shapes.
- Authorization JSON is judged by its decoded keys, so an escaped cf-access-* key is stripped like a plain one. - Every wait on an app is bounded: connect 10 s, each upload write 60 s, each read (the reply's start, then any quiet gap in a streamed body) 300 s. A hung app gets a 504 instead of holding the request forever. - WebSocket messages from an app are capped at 16 MiB, as uvicorn caps the browser's; an over-cap message closes the relay.
The /tools/<name>/ proxy forwarded every request header, Access token and cookie included, to a tool's port. It now uses the same filters as the app proxy, both ways, and sends a length that matches the decoded body.
Four of them need a writable /workspace, on main as well, so they can't run on the CI runner yet.
…-in) Being on localhost isn't proof of being the owner: any program on the machine can send whatever Host and Origin it likes. With WOLTSPACE_LODGE_AUTH=1, every lodge request (HTTP and WebSocket) needs a lodge key or a paired browser's cookie. - Keys live in <data root>/.space/run/lodge/ (folder 0700, files 0600), machine-local, never on a command line. The owner key serves the terminal and the lodge's own calls. The wolt key serves tools inside a wolt session, and can't grant auto mode, publish an app or change lodge settings. - `woltspace pair` prints a one-time link (local, and through the tunnel when there is one): single use, ten minutes, rate-limited, and stored hashed. Opening it sets an HttpOnly cookie (__Host- and Secure over https). It refuses to run inside a wolt session. - notify, push-view, the woltspace client, wolf, vulture, the chat lodge and the terminal cockpit send their key (curl reads it from stdin). - App hosts are not gated here, and apps never inherit a lodge key. - Off by default: nothing changes until it is turned on.
- A request must carry exactly one Host header. With two, the gate and the app router could disagree on which host it was. - A wolt key's writes are an allow-list (sessions, messages, viewport, crons, running apps, tools). Every other write is the owner's, and a route added later is too until it is listed. WebSockets (terminals, reload) are the owner's. - The chat lodge sends the owner key on all of its calls to the lodge.
…t a time - The app, WebSocket and tool proxies strip the lodge key (header, or a bearer that is a lodge key), the paired-browser and site cookies, and refuse those cookies from a backend's Set-Cookie. An app's own bearer still passes. - Issuing, redeeming and forgetting pairing codes, and creating keys, hold an exclusive lock across processes, so a CLI mint can't bring a spent code back and a pairing can't undo --forget. - Keyed clients don't follow redirects: urllib and fetch would carry the key to the next hop.
urllib never copies an unredirected header onto a redirect's next hop, so keyed clients keep urllib.request.urlopen (and the tests that stand in for it) instead of a custom opener.
- The paired-browser cookie is parsed leniently, so a malformed cookie from another localhost server can't lock the owner out. - Its name carries the lodge's port (woltspace-lodge-7777), so two lodges on one machine keep their own pairing. - The bot core's calls to the lodge (a new session's viewport redirect) carry the key. - An empty app label is no longer exempt from the gate. - The desktop app checks /health for readiness, which answers without a key. - Docs: WOLTSPACE_LODGE_KEY belongs in one script's shell, never .env.
…port A wolt's site is served at <wolt>.sites.localhost on the lodge's port, by the Mill, a process of its own on a unix socket in <data root>/.space/run/mill/ (folder 0700). The lodge forwards site hosts to it and runs no lodge route for them, so a page a wolt wrote never runs with the lodge's origin. - The lodge decides who the visitor is and tells the Mill in one header, dropping any copy the visitor sent, and forwards only content, caching, range and provenance headers: no cookie, Authorization or lodge key. - Sites are the owner's only for now. With lodge auth off: a loopback caller that didn't come through a tunnel. With it on: /sites/open hands a paired browser a one-time grant for one site, which the site's host turns into a cookie of its own. - The Mill serves GET and HEAD from the wolt's real wolt/site folder (no symlinked root, nothing outside it), refuses service workers, and feeds the owner's live reload from the site's own origin only. - Site naming is a list each way in adds to; only the local scheme serves so far. The old /wolt/<name>/site route stays. - If the Mill is down, sites answer 502 and the lodge keeps working. - The Mill is supervised like the connectors, by the entrypoint only.
The old /wolt/<name>/site/ route redirects to /sites/open, the lodge's root no longer falls back to the current wolt's site files, a static app's files aren't served from /app/<name>/, and /sites lists the /sites/open links. With auth off, nothing changes. docs/mill.md states the trust limit: code running as the owner can still reach the socket.
- With lodge auth on, /history/ sparks and /tools/ pages, wolt-written HTML the lodge still shows on its own origin, run in an opaque origin (a sandbox without allow-same-origin), so their script can't act as the owner. - woltspace pair --forget also voids unspent pairing codes and site grants. - An open site socket rechecks its visitor every 30 s, whatever the traffic, and closes once the site session has expired or been forgotten.
- The site sandbox also allows forms, new tabs, dialogs and downloads. The site keeps its own origin, which still can't reach the lodge's. - With lodge auth on, the old site address inside one of the lodge's own frames (the wolt page's site tab, a session's viewport) shows an "open it" link to /sites/open. A frame can't keep the site's cookie from a cross-site navigation, so redirecting there only spent a grant. - /sites/open answers 404 for a wolt without a site folder. - Docs: the local rule is stricter than the old address; container lodges need lodge auth for Mill sites for now; the socket is 0666 in a 0700 folder; no-store means files are refetched.
Local app hosts now go to the Mill like sites do. The Mill proxies the owner to the app's process on 127.0.0.1, streamed both ways, WebSockets with their subprotocol, and serves static apps (no start command) from their real dist/ or folder, never their dotfiles. - The lodge forwards an app's own headers and body to the Mill, minus the lodge's and Access's credentials, and says who the visitor is. - With lodge auth on, an app opens from /apps/open with a one-time grant for that app only; a site's grant never opens an app. - A browser write must come from the app's own page. - An app gets PORT (its manifest's port when free, else any free one) and HOST=127.0.0.1. - Tunnel app hosts keep the lodge's own proxy until the switch-over.
…, fixed ports - A static app never serves a file whose real path has a dot-named part, however the request reached it (a symlink included). - The Mill sends the app the raw request target, so %3F and %23 stay escaped; the lodge's WebSocket hop does too. - Both hops close their HTTP client on any failure before a response, cancellation included. - Apps keep their manifest port and the conflict check: choosing a free port needs a reservation and a check that the app's own process holds it, which is a later step.
- Sites and apps through the tunnel (site-<wolt>.<domain>, <app>.<domain>) go to the Mill once the lodge has verified the Cloudflare Access token for the apps' Access application and the owner's email (server/access.py, from the earlier gateway work). The token goes no further. Without access settings in woltspace.json, nothing through the tunnel opens. site- is reserved for sites. - The lodge's old app proxy, its app WebSocket router and per-app quick tunnels (share, unshare, WOLTSPACE_SHARING_ENABLED) are gone. - /wolt/<name>/site/ and /app/<name>/ redirect to /sites/open and /apps/open, which send a browser that reached the lodge through the tunnel to the public addresses. The lodge's root no longer falls back to a wolt's site files. Sparks and tool pages always run opaque. - woltspace doctor: apps that listen on 0.0.0.0 or share a name, and missing or shared Access settings when there's a tunnel. Read-only. - container/migrations/v0.6.0.md: what changed and the steps.
…n tests - The lodge's routes answer only its own names (loopback and its tunnel host). Every site and app host is taken by the Mill forwarder first, so lodge auth no longer exempts app-shaped hosts, and a reserved or invalid one (site-foo.localhost, a.b.<domain>) is refused, over HTTP and WebSocket, with lodge auth on or off. - Tests for the real Access verifier, with locally signed RS256 tokens served from a stand-in certs endpoint. The owner's token for the apps' audience verifies. A wrong signature, the lodge's audience, another issuer, an expired token, an unknown or missing key id, or HS256 fail closed, and only the owner's email gets in.
…remote sites opt-in
- A write or WebSocket through the tunnel must carry Origin
https://<its own host>, checked before the request is rewritten to its
local form; a local-form, sibling, wrong-scheme, wrong-port or null
Origin, or a cross-site fetch, is refused before anything reaches the
Mill. Only the content's own Origin is carried over.
- At boot the lodge stops the per-app quick tunnels an older lodge
started, signalling a pid only while it's still cloudflared, and clears
a record only once nothing of it runs. woltspace doctor fails while one
still runs.
- Remote sites are off until the owner sets "mill": {"remote_sites":
true} in woltspace.json, having checked no app ever used a site- name,
whose browser storage and service workers would outlive it. A current
app named site-<wolt> keeps the name from the site.
…signalled - An app whose old quick tunnel won't stop, or whose recorded pid is now some other cloudflared, is left untouched at boot (no orphan cleanup, no restart), so the next boot and woltspace doctor still see it. - A pid is signalled only while its command is that app's quick tunnel (cloudflared tunnel --url http://localhost:<its port>). The lodge's own named tunnel, or anyone else's cloudflared, is never signalled. - A write through the tunnel without an Origin is refused too. - An app on the reserved site- prefix isn't served; doctor names it, and the docs no longer say it keeps its name.
- The apps, site and cloudflare skills and the wildcard and environment docs no longer describe sharing, quick tunnels, the old proxy or the site shell on site pages; they point at the Mill and the access settings. - Migration step 2 says to reuse an existing apps Access application (woltspace-apps) and where both AUD tags come from. - doctor remote-apps validates the access settings by the lodge's own rules (all four keys, team domain, distinct audiences, owner email), and finds a native lodge's tunnel URL in the data root's .env. - Code this PR left unused is gone.
- apps: served at <name>.localhost by the Mill; the starter manifest and fields use --host $HOST, never 0.0.0.0; public is ignored; static apps come from dist/ or the folder through the Mill; apps are the owner's only, with no sharing. - site: pages are served as written on the site's origin and are the owner's only; the shell and its four levels skin the built-in pages; a site needs its own index.html. - wildcard setup: without the Access step, the lodge refuses everyone.
The skill told wolts to curl the lodge port to start and list apps. With lodge auth on, a bare curl carries no key and gets 401. It now uses woltspace app start/stop/restart/logs, and a new woltspace app list that shows each app's own port. A new section says how a wolt uses an app: on 127.0.0.1:<port>, never through <app>.localhost:7777 or /app/<name>/, which are the owner's browser addresses. Sharing is marked as the owner's call.
A wolt can still share in 0.5: it can set public: true and start the app, and starting is a wolt write. So the skill no longer says sharing needs the owner key; it tells wolts never to share or set public: true unless the human asked. The viewport URL is marked as the owner's.
…c apps Inside a container lodge, 127.0.0.1 is the container. An off app needs starting before its port answers; a static app has no API. Editing a running app's database by hand needs the human's ask and the app's locking and schema.
docs/0.6.md, mill.md, lodge-auth.md, ways-in.md and the migration guide say sites stay at /wolt/<name>/site/ in 0.6, as lodge pages, and that an origin of their own is a later change. The Mill's page is about apps. The site skill is main's again (shell, About fallback, live reload), still private with no sharing; the apps skill names both addresses. The Cloudflare skill drops remote_sites.
…nside a quoted value
…would expand is saved
… them, doctor warns, the guide has a step
… which names those are
…comes from the given env, a quick tunnel no longer returns its deleted log
…e bindings dotenv sees
… to the same address
… 0.10, for its parser
Resolve two conflicts with the messaging stack: the CLI request keeps the lodge key and also passes the inbox headers, and the server imports both sets of names.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is #651 with the site move taken out, plus fixes from its review. Start with
docs/0.6.md.What changes from #651
/wolt/<name>/site/, served as on main, with the nav shell, the About fallback, live reload and query strings. Giving sites their own addresses was premature. Site isolation becomes a separate, smaller change later, and app names starting withsite-stay reserved for it.site_livereload.pyand the site half ofcontent_hosts.pyandmill.pyare gone. The Mill's file server becomesstatic_files.py, used by static apps. One-time grants and the browser cookie now belong to apps.lodge_audfor a separate lodge Access application.If-Rangeis honoured.connectwrites settings in a safe order.::1are reached.mainis merged in, with no conflicts.Everything else from #651 is unchanged: credentials kept away from apps, ways in, one-code connect, email login, explicit sharing and opt-in lodge auth.
Tests: run in a throwaway HOME and data root. The failures are the same set as
main's in that environment; the one extra came from the shell's legacyWOLT_*variables and passes without them. CI on this PR is the reference.Replaces #651, which stays open as the record until this lands.