[{"data":1,"prerenderedAt":2906},["ShallowReactive",2],{"garden-tag-en-self-hosting":3},[4,380,550,758,896,1568,2068,2314,2618,2730],{"id":5,"title":6,"body":7,"description":366,"draft":367,"extension":368,"meta":369,"navigation":370,"notice":371,"path":372,"seo":373,"stage":374,"stem":13,"tags":375,"topic":378,"__hash__":379},"garden_en\u002Facting-on-a-humans-behalf.md","Acting on a Human's Behalf",{"type":8,"value":9,"toc":355},"minimark",[10,14,23,26,45,48,53,64,67,94,107,110,114,117,124,135,143,147,154,164,170,174,177,199,210,213,217,220,231,241,255,263,266,276,296,299,303,306,324,335,338,342,348],[11,12,6],"h1",{"id":13},"acting-on-a-humans-behalf",[15,16,17,18,22],"p",{},"Put a service between a person and a backend and you've created an identity question, whether or not you notice. The person authenticated to ",[19,20,21],"em",{},"your service",". Your service now has to say something to the backend. What, exactly?",[15,24,25],{},"There are two credentials in play here, and the entire design rests on keeping them apart:",[27,28,29,38],"ol",{},[30,31,32,33,37],"li",{},"The one the ",[34,35,36],"strong",{},"caller"," presented to your service.",[30,39,40,41,44],{},"The one your ",[34,42,43],{},"service"," presents to the backend.",[15,46,47],{},"The tempting move — and the one worth naming as a mistake up front — is to make the second a copy of the first.",[49,50,52],"h2",{"id":51},"why-passing-the-token-through-breaks-things","Why passing the token through breaks things",[15,54,55,56,59,60,63],{},"If a service simply forwards the caller's token to the backend, the backend can no longer tell whether a request arrived ",[19,57,58],{},"through the service"," or ",[19,61,62],{},"directly",". Both look identical on the wire, because they are identical.",[15,65,66],{},"That sounds abstract until you list what depends on being able to tell:",[68,69,70,76,82,88],"ul",{},[30,71,72,75],{},[34,73,74],{},"Rate limits"," scoped per client stop meaning anything, because there's only one apparent client.",[30,77,78,81],{},[34,79,80],{},"Audit trails"," record the human, with no record that a service was involved at all.",[30,83,84,87],{},[34,85,86],{},"Anomaly detection"," loses its baseline: traffic from an automated component and traffic from a person at a keyboard have very different shapes, and merging them hides both.",[30,89,90,93],{},[34,91,92],{},"Revocation"," gets coarse. You can no longer cut off the service without cutting off the person.",[15,95,96,97,100,101,106],{},"Every one of those controls is keyed on the token's ",[19,98,99],{},"audience"," — who the token was minted for. Passthrough destroys that field's meaning. This isn't a policy preference; it's why the ",[102,103,105],"a",{"href":104},"\u002Fgarden\u002Fmcp-server-authorization","MCP specification forbids it outright"," rather than merely discouraging it.",[15,108,109],{},"So the service needs its own credential. That leaves exactly two shapes.",[49,111,113],{"id":112},"option-one-service-identity","Option one: service identity",[15,115,116],{},"The service holds a credential of its own. It authenticates to the backend as itself.",[15,118,119,120,123],{},"The backend sees ",[19,121,122],{},"the service",", cleanly and unambiguously. Audience checks work. Rate limits work. Revoking the service is one operation.",[15,125,126,127,130,131,134],{},"The cost is stated plainly: ",[34,128,129],{},"the human's trail ends at the service."," The backend knows \"this service did it\", not \"this service did it for Amir\". If you need to answer ",[19,132,133],{},"which person"," triggered a given backend call, you have to answer it from the service's own logs, and you have to trust those logs, and they're now a separate system from your backend's audit trail.",[15,136,137,138,142],{},"This option's real virtue is that it works everywhere. Every OAuth server does ",[139,140,141],"code",{},"client_credentials",". There is no feature to shop for.",[49,144,146],{"id":145},"option-two-delegation","Option two: delegation",[15,148,149,150,153],{},"The service exchanges the caller's token for a ",[19,151,152],{},"different"," token, minted for the backend, that carries both identities.",[15,155,156,157,163],{},"RFC 8693 (OAuth 2.0 Token Exchange) is the standard for this, and its mechanism is the ",[34,158,159,162],{},[139,160,161],{},"act"," claim",": the subject of the token is the human, and the actor is the service. Both are present in one token, and a backend that reads the claim sees \"X, acting through Y.\" Whether a given backend reads it is a separate question, taken up further down — the answer is not automatically yes.",[15,165,166,167,169],{},"The ",[139,168,161],{}," claim nests. A chain of services each add themselves, and the resulting token describes the whole path rather than flattening it.",[49,171,173],{"id":172},"delegation-is-not-impersonation","Delegation is not impersonation",[15,175,176],{},"This is the distinction most worth carrying away, because the two are frequently discussed as one thing and they have opposite properties.",[68,178,179,189],{},[30,180,181,184,185,188],{},[34,182,183],{},"Impersonation",": the service ",[19,186,187],{},"becomes"," the human. The resulting token says \"X\". The service's involvement is gone — not obscured, gone. Downstream, there is no way to recover the fact that a machine was in the path.",[30,190,191,194,195,198],{},[34,192,193],{},"Delegation",": the service acts ",[19,196,197],{},"as itself, on behalf of"," the human. The resulting token says \"X, via Y\". Both identities survive.",[15,200,201,202,205,206,209],{},"Mechanically, the difference between them is a ",[34,203,204],{},"single parameter",": whether the exchange request includes an ",[139,207,208],{},"actor_token",". Send it, and you get delegation. Omit it, and you get impersonation.",[15,211,212],{},"That's a startlingly small lever for a large semantic difference, and it's worth knowing which one a given implementation gives you by default — and, more to the point, whether it supports both at all. Several do not.",[49,214,216],{"id":215},"but-does-the-backend-read-it","But does the backend read it?",[15,218,219],{},"Putting the identity into the token and having something downstream act on it are different achievements, and the standard only guarantees the first.",[15,221,222,223,226,227,230],{},"This got measured rather than assumed: Zitadel 2.71.7 issuing tokens against Stalwart 0.16.17. The exchange itself works exactly as described above — ",[139,224,225],{},"sub"," is the human, ",[139,228,229],{},"act.sub"," is the service, and Stalwart accepts the token and acts as the human.",[15,232,233,234,236,237,240],{},"Then it stops. Stalwart never reads ",[139,235,161],{},". And a successful bearer authentication produces ",[34,238,239],{},"no authentication event at all"," — checked with a trace-level tracer — where a password login produces one. So on the backend side, a delegated request is indistinguishable from the person acting directly, and neither one is written down.",[15,242,243,244,246,247,250,251,254],{},"Worth being precise about what that is and isn't. It's not a defect, and it isn't carelessness: reading ",[139,245,161],{}," is a capability that simply isn't present, which is the ordinary state of most backends today. The RFC defines what a token may ",[19,248,249],{},"say",". It obliges no resource server to ",[19,252,253],{},"care",".",[256,257,258],"blockquote",{},[15,259,260],{},[34,261,262],{},"Delegation puts the identity in the token. Whether the backend reads it is a second question — and it often isn't asked.",[15,264,265],{},"The consequence lands directly on the choice this page is about. If you picked delegation to get an end-to-end audit trail, you've done half the work. The other half is confirming the backend uses what you sent. Where it doesn't, the trail has to be produced by your own service instead — which is precisely the cost option one was already charged for above.",[15,267,268,269,272,273,275],{},"So delegation's advantage over service identity is ",[34,270,271],{},"smaller than the earlier sections imply",", whenever the backend ignores ",[139,274,161],{},". What survives that discount is still real:",[68,277,278,284,290],{},[30,279,280,283],{},[34,281,282],{},"Short-lived tokens"," rather than a long-lived shared secret sitting in the service's configuration.",[30,285,286,289],{},[34,287,288],{},"Central revocation"," — cut the service off at the issuer instead of rotating a credential everywhere it was deployed.",[30,291,292,295],{},[34,293,294],{},"No backend credential in the service's config"," at all.",[15,297,298],{},"Those are security properties, not audit properties. If audit was the reason for reaching for delegation, check the far end before counting the win.",[49,300,302],{"id":301},"the-questions-this-actually-turns-into","The questions this actually turns into",[15,304,305],{},"Notice what happened. \"How should my service authenticate?\" resolved into two much more specific questions, both answerable before committing to anything:",[27,307,308,316],{},[30,309,310],{},[34,311,312,313,315],{},"Does my token issuer support RFC 8693 with an ",[139,314,208],{},"?",[30,317,318],{},[34,319,320,321,323],{},"Does my backend do anything with the ",[139,322,161],{}," claim once it arrives?",[15,325,326,327,330,331,254],{},"The first is a yes\u002Fno you can settle from documentation — and plenty of identity providers answer ",[19,328,329],{},"no"," while still advertising RFC 8693 support, because issuing a token for yourself and issuing a token that names two parties are different features filed under one RFC number. That's the subject of the next note: ",[102,332,334],{"href":333},"\u002Fgarden\u002Fquestions-to-ask-an-idp","Questions to Ask an IdP",[15,336,337],{},"The second is the one that's easy to skip, because the standard says nothing about it and the token looks correct either way. It can only be answered against the specific backend — and, as above, the answer is often no.",[49,339,341],{"id":340},"where-this-actually-stands","Where this actually stands",[15,343,344,347],{},[34,345,346],{},"Budding."," The model is settled and the vocabulary is standard — none of it is invented here. One end of it has since been measured rather than read: the Zitadel\u002FStalwart result above is a real test of a real pair, not a documentation claim. The deployment itself still hasn't happened, so most of the operational story is still ahead.",[15,349,350,351,354],{},"The honest summary, updated: service identity is always available and gives up the human's trail. Delegation preserves that trail ",[19,352,353],{},"in the token",", and constrains your choice of issuer considerably — but it only pays off in audit terms if the backend reads what it's sent. Pick knowing which you're paying for, and check the far end before assuming you got it.",{"title":356,"searchDepth":357,"depth":357,"links":358},"",2,[359,360,361,362,363,364,365],{"id":51,"depth":357,"text":52},{"id":112,"depth":357,"text":113},{"id":145,"depth":357,"text":146},{"id":172,"depth":357,"text":173},{"id":215,"depth":357,"text":216},{"id":301,"depth":357,"text":302},{"id":340,"depth":357,"text":341},"A service that calls a backend for a person needs two credentials — and the first must never become the second.",false,"md",{},true,null,"\u002Fgarden\u002Facting-on-a-humans-behalf",{"title":6,"description":366},"budding",[376,377],"security","identity","delegation","uKdI0phUDX6P87X_uCGwFwgfIRwYhscK0xZhZiTTZik",{"id":381,"title":382,"body":383,"description":539,"draft":367,"extension":368,"meta":540,"navigation":370,"notice":541,"path":542,"seo":543,"stage":374,"stem":387,"tags":544,"topic":548,"__hash__":549},"garden_en\u002Fclean-room-reimplementation-with-an-agent.md","Clean-Room Reimplementation With an Agent",{"type":8,"value":384,"toc":533},[385,388,395,401,404,408,423,434,438,498,502,505,513,517],[11,386,382],{"id":387},"clean-room-reimplementation-with-an-agent",[15,389,390,391,394],{},"The case: a Sieve interpreter in Rust, permissively licensed. The existing one, ",[139,392,393],{},"sieve-rs",", is licensed \"under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version\", with a commercial licence on offer. Its parent project's contribution guidelines state: \"AI-generated code is not accepted in this project.\" That closes the door on contributing fixes back the way we work, and opens the question of writing our own.",[15,396,397,398,400],{},"The agent writing it with me has, in all likelihood, seen ",[139,399,393],{}," during training. Asked directly, it reported a rough memory of the architecture: a compiler to an instruction list and an event-driven runtime that hands actions back to the caller. It could not say whether any snippet it produces is memorised rather than composed. Nobody can say that about a model from the outside either.",[15,402,403],{},"A traditional clean room keeps the implementers from ever reading the original. Here one implementer has already read it and can't unread it. So the clean room has to be rebuilt as a process that doesn't depend on ignorance.",[49,405,407],{"id":406},"what-copyright-does-and-doesnt-cover","What copyright does and doesn't cover",[15,409,410,414,415,418,419,422],{},[102,411,413],{"href":412},"\u002Fgarden\u002Fgenerated-code-has-no-owner","Generated code has no owner"," works through the legal side; the relevant piece here is § 69a Abs. 2 UrhG: ideas, principles and interfaces are not protected. An interpreter for RFC 5228 will resemble every other one ",[19,416,417],{},"in structure",", because the RFC dictates the grammar, the commands, the tests, the actions. That is fine. What must not be carried over is ",[19,420,421],{},"expression",": the particular names, comments, error strings, the shape of a non-obvious trick, any passage recognisable as a passage.",[15,424,425,426,429,430,433],{},"The RFCs themselves are safe input. The IETF Trust Legal Provisions license \"Code Components\" of RFCs, the parts between ",[139,427,428],{},"\u003CCODE BEGINS>"," and ",[139,431,432],{},"\u003CCODE ENDS>"," or otherwise marked as code, under the Revised BSD License. Example scripts and grammar in the Sieve RFCs can be used as test material and reference outright.",[49,435,437],{"id":436},"the-process","The process",[27,439,440,449,458,464,486,492],{},[30,441,442,445,446,448],{},[34,443,444],{},"Spec in, nothing else in."," The interpreter is derived from the RFCs and from our own design book, written first. The question \"how does ",[139,447,393],{}," do it\" is never asked, of the agent or of a search engine. Bugs observed in the existing implementation are described as behaviour, never as code.",[30,450,451,454,455,457],{},[34,452,453],{},"Context hygiene."," No third-party Sieve source in the workspace, ever. Not ",[139,456,393],{},", not Pigeonhole (LGPLv2.1, and its test scripts are code too), not Cyrus. Nothing pasted into the conversation. The model's training exposure is a risk I can't control; what enters its context window is one I fully can, and it's the larger of the two.",[30,459,460,463],{},[34,461,462],{},"Divergence on purpose."," Where the agent remembers how the incumbent is built, that memory has exactly one permitted use: choose differently, and write down why in the design book. Ideas are free to reuse; the point of diverging is that it makes accidental expressive overlap improbable, not that reuse would be illegal.",[30,465,466,469,470,473,474,477,478,481,482,485],{},[34,467,468],{},"A diff tool at the door."," In CI, the incumbent is cloned into a comparison directory outside the project tree and ",[139,471,472],{},"jscpd",", which lists ",[139,475,476],{},"rust"," among its formats, runs over both with ",[139,479,480],{},"--min-tokens"," in the 40 to 50 range and a ",[139,483,484],{},"--threshold"," of zero on cross-repository hits. The check is mechanical, never done by the agent reading the other codebase, because reading it to compare is the contamination we're trying to avoid. Hits go to a human; affected regions are rewritten from the spec.",[30,487,488,491],{},[34,489,490],{},"Provenance on record."," Commit trailers name the agent. The design book predates the code. The hygiene rules live in the repo's agent instructions. If a claim ever came, the defence is documented independent creation, and this is what that documentation looks like.",[30,493,494,497],{},[34,495,496],{},"Say it on the tin."," The licence file states that the codebase is agent-assisted. Downstream users deserve to know which risk profile they're adopting.",[49,499,501],{"id":500},"why-this-is-enough-and-why-it-isnt-more","Why this is enough, and why it isn't more",[15,503,504],{},"The process cannot prove that no memorised line ever slipped through. What it does is make each path narrow: the model's memory is only for diverging, the context is clean, the output is scanned against the one implementation it could plausibly echo, and the trail shows the work was done from the specification. That is more than most human reimplementations can show, given that human authors usually read the competition first and keep no record of it.",[15,506,507,508,512],{},"What remains is residual, and it is the residual that ",[102,509,511],{"href":510},"\u002Fgarden\u002Findemnity-is-not-ownership","vendor indemnity"," is designed for, on plans where it exists, which is not the plan this project runs on. So the process carries the whole weight. It should.",[49,514,516],{"id":515},"open","Open",[68,518,519,524,530],{},[30,520,166,521,523],{},[139,522,472],{}," threshold numbers are a starting guess, not a calibrated value; tune after the first false positives.",[30,525,526,527,529],{},"Whether the comparison should also include non-Rust implementations. Cross-language clones are rare in practice and ",[139,528,472],{}," compares within a format, so the current answer is no.",[30,531,532],{},"How the design book and the code should reference each other so that \"the book predates the code\" is checkable from git history alone, not just asserted.",{"title":356,"searchDepth":357,"depth":357,"links":534},[535,536,537,538],{"id":406,"depth":357,"text":407},{"id":436,"depth":357,"text":437},{"id":500,"depth":357,"text":501},{"id":515,"depth":357,"text":516},"The model has read the competitor. That makes the clean room a process, not a promise — spec in, nothing else in, a diff tool at the door.",{},"legal","\u002Fgarden\u002Fclean-room-reimplementation-with-an-agent",{"title":382,"description":539},[545,546,547],"licensing","ai","self-hosting","ai-provenance","Y0IWseTi1BpoJZwOQK8zNSvAlYyuTsY6jvjeX01cIpg",{"id":551,"title":552,"body":553,"description":751,"draft":367,"extension":368,"meta":752,"navigation":370,"notice":541,"path":412,"seo":753,"stage":754,"stem":755,"tags":756,"topic":548,"__hash__":757},"garden_en\u002Fgenerated-code-has-no-owner.md","Generated Code Has No Owner, and That's Not the Problem",{"type":8,"value":554,"toc":744},[555,558,561,567,571,574,595,598,602,605,610,613,618,631,638,643,648,655,660,663,666,670,673,678,683,690,694,697,700,703,708,714,719,723,738],[11,556,552],{"id":557},"generated-code-has-no-owner-and-thats-not-the-problem",[15,559,560],{},"The worry that started this: an agent writes a Sieve interpreter with me, and the agent's model has almost certainly seen the only other Rust implementation, which is AGPL. Could that code end up in ours?",[15,562,563,564,566],{},"Before answering, a detour that looked like it might dissolve the question: if machine output isn't copyrightable anyway, does licensing even apply? It turns out the detour answers a ",[19,565,152],{}," question, and knowing which one is the point of this note.",[49,568,570],{"id":569},"two-directions","Two directions",[15,572,573],{},"There are two copyright questions in any generated codebase, and they are independent:",[27,575,576,586],{},[30,577,578,581,582,585],{},[34,579,580],{},"Outbound."," Do ",[19,583,584],{},"we"," hold rights in what we produce? Can we license it at all?",[30,587,588,591,592,315],{},[34,589,590],{},"Inbound."," Did the output carry in expression that belongs to ",[19,593,594],{},"someone else",[15,596,597],{},"The \"machine output isn't protectable\" argument speaks to the first. It says nothing about the second.",[49,599,601],{"id":600},"outbound-machine-output-is-not-protectable","Outbound: machine output is not protectable",[15,603,604],{},"The German statute is one sentence. § 2 Abs. 2 UrhG:",[256,606,607],{},[15,608,609],{},"\"Werke im Sinne dieses Gesetzes sind nur persönliche geistige Schöpfungen.\"",[15,611,612],{},"For software specifically, § 69a Abs. 3 UrhG:",[256,614,615],{},[15,616,617],{},"\"Computerprogramme werden geschützt, wenn sie individuelle Werke in dem Sinne darstellen, daß sie das Ergebnis der eigenen geistigen Schöpfung ihres Urhebers sind.\"",[15,619,620,429,623,626,627,630],{},[19,621,622],{},"Persönlich",[19,624,625],{},"Urheber"," both presuppose a person. Output with no human creative contribution isn't a ",[19,628,629],{},"Werk"," in the statute's sense, so there is nothing to license.",[15,632,633,634,637],{},"The US position was stated at length by the Copyright Office in its report ",[19,635,636],{},"Copyright and Artificial Intelligence, Part 2: Copyrightability"," (January 2025). From its executive summary, verbatim:",[256,639,640],{},[15,641,642],{},"\"Copyright does not extend to purely AI-generated material, or material where there is insufficient human control over the expressive elements.\"",[256,644,645],{},[15,646,647],{},"\"Based on the functioning of current generally available technology, prompts do not alone provide sufficient control.\"",[15,649,650,651,654],{},"And the D.C. Circuit in ",[19,652,653],{},"Thaler v. Perlmutter",", No. 23-5233, decided 2025-03-18 (Millett, J.):",[256,656,657],{},[15,658,659],{},"\"The Creativity Machine cannot be the recognized author of a copyrighted work because the Copyright Act of 1976 requires all eligible work to be authored in the first instance by a human being.\"",[15,661,662],{},"The court stopped there on purpose: \"Given that holding, we need not address the Copyright Office's argument that the Constitution itself requires human authorship of all copyrighted material.\" Reports say the Supreme Court declined to review the case in March 2026; I have only seen the headline, not the order.",[15,664,665],{},"So: what the agent generates on its own is public domain in effect. Not \"unlicensable\" as some kind of restriction. Free for everyone, including us, because nobody can withhold it.",[49,667,669],{"id":668},"outbound-second-half-humans-still-get-their-share","Outbound, second half: humans still get their share",[15,671,672],{},"The same sources are equally clear that using a machine doesn't strip a human of authorship. The Copyright Office again:",[256,674,675],{},[15,676,677],{},"\"The use of AI tools to assist rather than stand in for human creativity does not affect the availability of copyright protection for the output.\"",[256,679,680],{},[15,681,682],{},"\"Human authors are entitled to copyright in their works of authorship that are perceptible in AI-generated outputs, as well as the creative selection, coordination, or arrangement of material in the outputs, or creative modifications of the outputs.\"",[15,684,685,686,689],{},"A project like ours is a mix. Architecture decisions, the design book, hand-edited code, the selection of what survives review: human, protectable. Boilerplate the agent produced from a one-line instruction: not. Nobody will draw that line file by file, and for a project that wants a permissive licence anyway, nobody needs to. A permissive licence still does real work on the human-authored parts and is a harmless no-op on the rest. What it ",[19,687,688],{},"should"," say is that the codebase is agent-assisted, so nobody downstream is surprised by which part is which.",[49,691,693],{"id":692},"inbound-the-part-the-detour-doesnt-touch","Inbound: the part the detour doesn't touch",[15,695,696],{},"None of the above changes the status of code that was somebody else's before the model ever saw it. If the agent reproduces a distinctive passage from an AGPL project, that passage is still that project's work, under that project's licence, inside our tree. Our own rights being thin or absent is irrelevant to their rights being intact.",[15,698,699],{},"That is the risk with teeth, and it is the only one in this picture that can actually hurt: a licence conflict we didn't choose, discoverable by anyone with a diff tool.",[15,701,702],{},"One thing narrows it. Ideas, algorithms and interfaces are outside copyright, in the statute's own words, § 69a Abs. 2 UrhG:",[256,704,705],{},[15,706,707],{},"\"Ideen und Grundsätze, die einem Element eines Computerprogramms zugrunde liegen, einschließlich der den Schnittstellen zugrundeliegenden Ideen und Grundsätze, sind nicht geschützt.\"",[15,709,710,711,713],{},"An interpreter for an RFC-defined language will resemble every other interpreter for that language in structure, because the RFC dictates the structure. That resemblance is not infringement. What would be is the ",[19,712,421],{},": names, comments, the particular shape of a non-obvious trick, a paragraph of code carried over intact.",[15,715,716,717,254],{},"So the practical question isn't \"is our code ours\". It's \"how do we keep somebody else's expression out\". That's a process question, and it has a process answer: ",[102,718,382],{"href":542},[49,720,722],{"id":721},"what-this-note-does-not-settle","What this note does not settle",[68,724,725,732,735],{},[30,726,727,728,731],{},"Whether the human contribution in ",[19,729,730],{},"our"," project would clear the bar in a dispute. Unlikely to matter for a permissively licensed codebase, but not examined.",[30,733,734],{},"The EU-level case law on \"author's own intellectual creation\". The German statute was enough for the purpose here.",[30,736,737],{},"Whether any of this would look different in a jurisdiction that isn't Germany or the US.",[15,739,740,741,254],{},"A related misreading, that the model vendor might own the output because it offers to defend you over it, gets its own note: ",[102,742,743],{"href":510},"Indemnity Is Not Ownership",{"title":356,"searchDepth":357,"depth":357,"links":745},[746,747,748,749,750],{"id":569,"depth":357,"text":570},{"id":600,"depth":357,"text":601},{"id":668,"depth":357,"text":669},{"id":692,"depth":357,"text":693},{"id":721,"depth":357,"text":722},"Purely machine-generated code is nobody's property. The risk with teeth runs the other way — what the machine may have carried in.",{},{"title":552,"description":751},"seedling","generated-code-has-no-owner",[545,546],"ysY_MwpoRorIxUJnBXsFDIJwYVXdVZSh-quvnqPhwX8",{"id":759,"title":743,"body":760,"description":891,"draft":367,"extension":368,"meta":892,"navigation":370,"notice":541,"path":510,"seo":893,"stage":754,"stem":764,"tags":894,"topic":548,"__hash__":895},"garden_en\u002Findemnity-is-not-ownership.md",{"type":8,"value":761,"toc":884},[762,765,772,776,779,784,787,792,803,807,810,815,818,825,832,836,839,853,856,860,869,873],[11,763,743],{"id":764},"indemnity-is-not-ownership",[15,766,767,768,771],{},"A reasonable inference on first hearing that Anthropic indemnifies customers for copyright claims over model output: ",[19,769,770],{},"if they'll pay for it, they must think it's theirs."," The terms say the opposite, and the two clauses sit a few sections apart.",[49,773,775],{"id":774},"the-assignment-clause","The assignment clause",[15,777,778],{},"Consumer Terms of Service, effective 2025-10-08, Section 4:",[256,780,781],{},[15,782,783],{},"\"Subject to your compliance with our Terms, we assign to you all our right, title, and interest (if any) in Outputs.\"",[15,785,786],{},"Commercial Terms of Service, effective 2025-06-17:",[256,788,789],{},[15,790,791],{},"\"Subject to Customer's compliance with these Terms, Anthropic hereby assigns to Customer its right, title and interest (if any) in and to Outputs.\"",[15,793,794,795,798,799,802],{},"Two things in one sentence. The vendor claims nothing: it ",[19,796,797],{},"assigns away",". And the parenthetical \"(if any)\" is the vendor agreeing with ",[102,800,801],{"href":412},"the copyright offices"," that there may be nothing to assign. Whatever rights could exist in an output, they end up with the user, not the vendor.",[49,804,806],{"id":805},"the-indemnity-clause-and-where-it-isnt","The indemnity clause, and where it isn't",[15,808,809],{},"The Commercial Terms contain the defence promise, quoted here as excerpted:",[256,811,812],{},[15,813,814],{},"\"Anthropic will defend Customer and its personnel, successors, and assigns from and against any Customer Claim...alleging that Customer's paid use of the Services...or Outputs generated through such authorized use violates any third-party intellectual property right.\"",[15,816,817],{},"It comes with exclusions. As summarised from the same section: claims arising from the customer's modifications to outputs, from combining outputs with non-Anthropic technology, from customer-provided inputs, from knowing misuse, from practising patented inventions found in outputs, and from trademark use.",[15,819,820,821,824],{},"The Consumer Terms have ",[34,822,823],{},"no such clause",". Their Section 11 disclaims warranties, including non-infringement, and the only indemnity flows from the user to the vendor.",[15,826,827,828,831],{},"So the promise is a ",[19,829,830],{},"contractual defence against third parties",", offered on paid commercial use, hedged by exclusions, and absent from consumer plans. It is insurance for exactly the inbound risk described in the other note: the model may reproduce material that belongs to someone else. A vendor claiming ownership wouldn't need to insure you against that; it would be the defendant.",[49,833,835],{"id":834},"why-the-two-fit-together","Why the two fit together",[15,837,838],{},"Read side by side, the clauses are coherent:",[68,840,841,847],{},[30,842,843,846],{},[34,844,845],{},"Assignment:"," \"whatever this is, it's yours.\"",[30,848,849,852],{},[34,850,851],{},"Indemnity:"," \"and if it turns out to contain someone else's, we'll stand behind you, under conditions.\"",[15,854,855],{},"Neither says \"we made this, it's ours\". The reverse inference fails because it assumes indemnity is a property claim. It's a risk allocation.",[49,857,859],{"id":858},"what-this-changes-in-practice","What this changes in practice",[15,861,862,863,866,867,254],{},"For a one-person project on a consumer plan: nothing to rely on. The defence clause isn't there, and even where it is, the exclusion for ",[19,864,865],{},"modifications"," would bite any code that has been edited after generation, which is all of it. The only real protection remains not letting foreign expression in: ",[102,868,382],{"href":542},[49,870,872],{"id":871},"not-checked","Not checked",[68,874,875,878,881],{},[30,876,877],{},"Which Anthropic terms actually govern a Claude Code session on a given subscription. The consumer\u002Fcommercial split is stated in the terms; the mapping of plans to terms was not verified.",[30,879,880],{},"The full text of the indemnity clause. The quote above is excerpted; the ellipses are in the excerpt, not mine to fill.",[30,882,883],{},"Whether other vendors' terms follow the same pattern. Only Anthropic's were read.",{"title":356,"searchDepth":357,"depth":357,"links":885},[886,887,888,889,890],{"id":774,"depth":357,"text":775},{"id":805,"depth":357,"text":806},{"id":834,"depth":357,"text":835},{"id":858,"depth":357,"text":859},{"id":871,"depth":357,"text":872},"A vendor offering to defend you over its model's output is not claiming that output. Read the clause next to it.",{},{"title":743,"description":891},[545,546],"A0boDVly1GZsKAofi1gUqympr03lH0LpSB9XX179_ZI",{"id":897,"title":898,"body":899,"description":1559,"draft":367,"extension":368,"meta":1560,"navigation":370,"notice":371,"path":1561,"seo":1562,"stage":374,"stem":1563,"tags":1564,"topic":1566,"__hash__":1567},"garden_en\u002Fkauditd-hold-queue-overflow.md","The kauditd Hold Queue Incident",{"type":8,"value":900,"toc":1539},[901,904,907,923,927,930,933,942,946,973,976,987,1002,1006,1013,1024,1035,1038,1042,1045,1056,1059,1085,1101,1104,1109,1124,1131,1135,1141,1145,1148,1224,1228,1233,1236,1240,1246,1267,1274,1280,1284,1316,1336,1341,1354,1362,1380,1395,1399,1402,1446,1466,1470,1476,1505,1508,1511,1513,1518,1535],[11,902,898],{"id":903},"the-kauditd-hold-queue-incident",[15,905,906],{},"This one took three rounds. Two of them produced fixes that looked like they worked, and neither of them was a fix. Writing it down mostly for the shape of the mistake, which is more portable than the specifics.",[15,908,909,910,914,915,918,919,922],{},"The mechanism underneath is in the neighbouring note — ",[102,911,913],{"href":912},"\u002Fgarden\u002Flinux-audit-subsystem","The Linux Audit Subsystem"," — and this story doesn't really parse without it. The one-line version: the kernel delivers audit records over ",[34,916,917],{},"two independent paths",", unicast to a registered ",[139,920,921],{},"auditd"," and multicast to passive subscribers like journald, and the second does not substitute for the first.",[49,924,926],{"id":925},"the-host","The host",[15,928,929],{},"chiba: a headless Hetzner box running NixOS on kernel 6.18.45, btrfs root, impermanence (the root filesystem is wiped every boot). It runs the git forge, Matrix, mail and webmail.",[15,931,932],{},"Its intrusion detection uses five kernel audit watches — three tamper watches and two canary baits:",[934,935,940],"pre",{"className":936,"code":938,"language":939},[937],"language-text","-w \u002Froot\u002F.ssh                            -p wa -k chiba-tamper-ssh\n-w \u002Fhome\u002Fsynapse\u002F.ssh                    -p wa -k chiba-tamper-ssh\n-w \u002Fpersist\u002Fvar\u002Flib\u002Fsops-nix\u002Fkey.txt     -p wa -k chiba-tamper-agekey\n-w \u002Froot\u002F.aws\u002Fcredentials                -p r  -k chiba-canary-bait\n-w \u002Fhome\u002Fsynapse\u002F.aws\u002Fcredentials        -p r  -k chiba-canary-bait\n","text",[139,941,938],{"__ignoreMap":356},[49,943,945],{"id":944},"the-actual-bug-stated-up-front","The actual bug, stated up front",[15,947,948,949,952,953,956,957,960,961,964,965,968,969,972],{},"In NixOS, ",[139,950,951],{},"security.audit.enable"," turns on ",[34,954,955],{},"only the kernel side"," and loads the rules. The daemon is a ",[19,958,959],{},"separate"," option: ",[139,962,963],{},"security.auditd.enable",". It was never set. ",[139,966,967],{},"auditctl -s"," showed ",[139,970,971],{},"pid 0"," — no registered consumer, the entire time.",[15,974,975],{},"The reason nobody looked there is a single sentence in this project's own design documentation, which read roughly:",[256,977,978],{},[15,979,980,983,984,254],{},[139,981,982],{},"systemd-journald-audit.socket"," is active, so audit records reach journald ",[19,985,986],{},"without the userspace auditd daemon",[15,988,989,990,993,994,997,998,1001],{},"That sentence is ",[34,991,992],{},"true",". It was read as ",[19,995,996],{},"\"therefore we don't need auditd\"",", which is ",[34,999,1000],{},"false",". Journald subscribes over multicast; the kernel still wants a unicast consumer and treats its absence as a delivery failure. That one misreading cost three rounds of diagnosis.",[49,1003,1005],{"id":1004},"round-1-the-cold-boot-burst-2026-08-21","Round 1 — the cold-boot burst (2026-08-21)",[15,1007,1008,1009,1012],{},"After a genuine cold start — the first in a while, since most deploys had been live ",[139,1010,1011],{},"switch","es rather than power cycles — every hardened systemd service loaded its seccomp-BPF filter at more or less the same moment. Each BPF program load is its own audit event. The burst blew straight through the then-current limit of 1024.",[15,1014,1015,1016,1019,1020,1023],{},"Symptoms: ",[139,1017,1018],{},"kauditd hold queue overflow",", thousands of suppressed callbacks, kauditd pegged, and the network stack unresponsive for minutes. From outside it looked like a hard crash — no ping, no SSH. But ",[139,1021,1022],{},"journalctl -b -1"," showed clean, orderly shutdowns. No panic.",[15,1025,1026,1027,1030,1031,1034],{},"Diagnosis: a simultaneous-start burst. Fix: raise ",[139,1028,1029],{},"backlogLimit",", ending at 16384 (8192 having been patched live into ",[139,1032,1033],{},"grub.cfg"," from Hetzner rescue mode as the emergency measure).",[15,1036,1037],{},"Plausible. Also wrong.",[49,1039,1041],{"id":1040},"round-2-its-not-boot-specific-2026-08-22","Round 2 — it's not boot-specific (2026-08-22)",[15,1043,1044],{},"It came back: 216 overflow lines inside a single three-hour window, with no reboot in between.",[15,1046,1047,1048,1051,1052,1055],{},"Diagnosis this time: the BPF cascade isn't unique to boot. It fires on ",[34,1049,1050],{},"every PAM login session"," that spins up a fresh ",[139,1053,1054],{},"user@\u003Cuid>.service"," — systemd builds and tears down the whole per-user instance, cgroup-BPF programs included, per login, when the user has no lingering session to stay warm between connections.",[15,1057,1058],{},"Two accounts drove it:",[68,1060,1061,1073],{},[30,1062,1063,1068,1069,1072],{},[34,1064,1065],{},[139,1066,1067],{},"synapse"," — every individual ",[139,1070,1071],{},"ssh chiba '\u003Ccmd>'"," is its own login\u002Flogout cycle. Automation that opens a fresh connection per command rather than multiplexing generates a lot of these.",[30,1074,1075,1080,1081,1084],{},[34,1076,1077],{},[139,1078,1079],{},"forgejo"," — and this is the one that scales badly. ",[19,1082,1083],{},"Every"," git-over-SSH operation rides system sshd as that one shared system user. So every contributor pushing or pulling against the forge multiplies the same PAM churn, not just the operator.",[15,1086,1087,1088,1091,1092,1094,1095,429,1098,1100],{},"Fix: ",[139,1089,1090],{},"users.users.\u003Cname>.linger = true"," for ",[139,1093,1067],{},", ",[139,1096,1097],{},"root",[139,1099,1079],{},", keeping the per-user instances resident instead of cycling per connection.",[15,1102,1103],{},"Also plausible. Also wrong — though linger stayed, because it removed real churn and is correct on its own terms.",[1105,1106,1108],"h3",{"id":1107},"the-instrumentation-that-was-the-bug-it-was-measuring","The instrumentation that was the bug it was measuring",[15,1110,1111,1112,1123],{},"Worth preserving for the sheer shape of it. The metric built in round 2, specifically to make this visible, was ",[34,1113,1114,1115,1118,1119,1122],{},"first implemented as a ",[139,1116,1117],{},"systemd.timer"," firing a ",[139,1120,1121],{},"Type=oneshot"," service every 30 seconds"," — which is to say, precisely the pattern that had just been diagnosed as the cause, running unconditionally and more often than the SSH traffic it was meant to observe.",[15,1125,1126,1127,1130],{},"It surfaced because paging continued after the linger deploy, and the overflow timestamps landed exactly on that service's start times. Rebuilt as a long-running process with a ",[139,1128,1129],{},"sleep"," loop.",[49,1132,1134],{"id":1133},"round-3-the-real-one-2026-08-24","Round 3 — the real one (2026-08-24)",[15,1136,1137,1138,1140],{},"It came back again. Linger active. ",[139,1139,1029],{}," at 16384.",[1105,1142,1144],{"id":1143},"the-evidence","The evidence",[15,1146,1147],{},"The counter shape is what cracked it:",[68,1149,1150,1164,1179,1188,1202,1218],{},[30,1151,1152,1155,1156,1159,1160,1163],{},[139,1153,1154],{},"node_audit_lost_total"," sat at ",[34,1157,1158],{},"exactly 0 for 50 hours"," after the 08-22 boot, then began climbing at ",[34,1161,1162],{},"20:13:42"," and never stopped. No reboot in between.",[30,1165,1166,1167,1170,1171,1174,1175,1178],{},"The audit event serial at onset was ",[34,1168,1169],{},"~5580",". At ~3 records per event that's ",[34,1172,1173],{},"~16.7k records"," — crossing ",[139,1176,1177],{},"backlog_limit = 16384"," right about there.",[30,1180,1181,968,1184,1187],{},[139,1182,1183],{},"\u002Fproc\u002Fslabinfo",[139,1185,1186],{},"skbuff_head_cache active = 17328"," against that same limit of 16384. The held records were directly visible as pinned kernel memory.",[30,1189,1190,1191,1194,1195,1198,1199,254],{},"After onset, ",[139,1192,1193],{},"lost"," grew by ",[34,1196,1197],{},"18–19 per 10-minute check cycle",", and one cycle generates ~17 audit records. Loss running ",[34,1200,1201],{},"1:1 with every record produced",[30,1203,1204,1207,1208,1211,1212,1215,1216,295],{},[139,1205,1206],{},"backlog"," read ",[34,1209,1210],{},"0"," the entire time — because that's the ",[19,1213,1214],{},"main"," queue, which drains fine. The hold queue isn't in ",[139,1217,967],{},[30,1219,1220,1221,1223],{},"The host's only error-level logs in 24 hours: 137 lines, all 137 ",[139,1222,1018],{},". Otherwise completely healthy — zero failed units, disk 5%, RAM 2.7G of 15G, backups green, certificates valid.",[1105,1225,1227],{"id":1226},"the-signature-worth-memorising","The signature worth memorising",[15,1229,1230],{},[34,1231,1232],{},"A counter that is exactly zero for hours, then climbs monotonically and never stops, describes a bucket filling up — not a burst.",[15,1234,1235],{},"A burst problem is jagged: spike, recover, spike, recover. This was flat, then a knee, then a straight line. That shape rules out \"too much traffic at once\" and points at \"something that should be draining isn't.\"",[1105,1237,1239],{"id":1238},"why-both-earlier-fixes-could-never-have-held","Why both earlier fixes could never have held",[15,1241,1242,1243],{},"They were ",[34,1244,1245],{},"time dials, not fixes.",[68,1247,1248,1261],{},[30,1249,1250,1251,1253,1254,1257,1258,254],{},"Raising ",[139,1252,1029],{}," makes the bucket bigger (and pins more kernel memory). It moved onset from ",[19,1255,1256],{},"minutes after boot"," to ",[19,1259,1260],{},"~50 hours after boot",[30,1262,1263,1266],{},[139,1264,1265],{},"linger"," lowered the record rate. It moved onset the same way, for the same reason.",[15,1268,1269,1270,1273],{},"And ",[34,1271,1272],{},"every reboot reset the counter",", which made both look like they had worked.",[15,1275,1276,1277],{},"That's the transferable lesson: ",[19,1278,1279],{},"if a fix only moves the moment the problem returns, it isn't a fix.",[49,1281,1283],{"id":1282},"the-fix","The fix",[934,1285,1289],{"className":1286,"code":1287,"language":1288,"meta":356,"style":356},"language-nix shiki shiki-themes github-light github-dark","security.auditd = {\n  enable = true;\n  settings.write_logs = false;\n};\n","nix",[139,1290,1291,1299,1304,1310],{"__ignoreMap":356},[1292,1293,1296],"span",{"class":1294,"line":1295},"line",1,[1292,1297,1298],{},"security.auditd = {\n",[1292,1300,1301],{"class":1294,"line":357},[1292,1302,1303],{},"  enable = true;\n",[1292,1305,1307],{"class":1294,"line":1306},3,[1292,1308,1309],{},"  settings.write_logs = false;\n",[1292,1311,1313],{"class":1294,"line":1312},4,[1292,1314,1315],{},"};\n",[15,1317,1318,1320,1321,1324,1325,1328,1329,1332,1333,1335],{},[139,1319,921],{}," registers as ",[139,1322,1323],{},"audit_pid"," on the unicast netlink socket. kauditd's send now succeeds, so ",[139,1326,1327],{},"audit_hold_queue"," ",[34,1330,1331],{},"drains"," instead of filling, the printk spam stops, and ",[139,1334,1193],{}," stands still.",[15,1337,1338,1340],{},[139,1339,1029],{}," stays at 16384. With a consumer present it guards only the main queue against genuine simultaneous-start bursts — which is what it was always meant to do.",[15,1342,1343,1345,1346,1349,1350,1353],{},[139,1344,921],{}," was also added to the central unit registry so its failure pages ",[34,1347,1348],{},"immediately",". Otherwise its absence only announces itself ~2 days later as ",[139,1351,1352],{},"AuditRecordsLost",", long after the cause.",[1105,1355,1357,1358,1361],{"id":1356},"on-write_logs-no-a-trade-off-not-a-trick","On ",[139,1359,1360],{},"write_logs = no"," — a trade-off, not a trick",[15,1363,1364,1365,1368,1369,1372,1373,1376,1377,1379],{},"Deliberate ",[34,1366,1367],{},"for this host",", and not a general recommendation. journald plus Vector is already chiba's evidence path, and ",[139,1370,1371],{},"\u002Fvar\u002Flog"," lives in ",[139,1374,1375],{},"\u002Fpersist",", so a second on-disk copy of the same records would grow every backup while covering nothing that isn't already covered. Here, ",[139,1378,921],{}," is wanted purely as the queue consumer, not as a second store.",[15,1381,1382,1383,1386,1387,1390,1391,1394],{},"If you don't already have another durable log path, you want the real ",[139,1384,1385],{},"audit.log"," with rotation. (",[139,1388,1389],{},"write_logs"," is a genuine ",[139,1392,1393],{},"auditd.conf(5)"," keyword, checked against the shipped man page for audit 4.2.1.)",[49,1396,1398],{"id":1397},"verification-after-deploy","Verification after deploy",[15,1400,1401],{},"Measured live, before → after:",[68,1403,1404,1416,1424,1431,1437],{},[30,1405,1406,1408,1409,1412,1413],{},[139,1407,967],{}," → ",[139,1410,1411],{},"pid",": ",[34,1414,1415],{},"0 → 216431",[30,1417,1418,1412,1421],{},[139,1419,1420],{},"skbuff_head_cache active",[34,1422,1423],{},"17328 → 1147",[30,1425,1426,1412,1428],{},[139,1427,1193],{},[34,1429,1430],{},"+18–19 per 10-minute cycle → frozen at 4802",[30,1432,1433,1434],{},"overflow lines: ",[34,1435,1436],{},"continuous → 0",[30,1438,1439,1412,1442,1445],{},[139,1440,1441],{},"\u002Fvar\u002Flog\u002Faudit",[34,1443,1444],{},"empty",", as intended",[15,1447,1448,1449,1451,1452,1454,1455,1458,1459,1462,1463,1465],{},"Then a 14-minute window at 60-second sampling: ",[139,1450,1193],{}," flat across all 14 samples, ",[139,1453,1206],{}," 0 throughout, ",[139,1456,1457],{},"skbuff_head_cache"," between 1147 and 1236 with no trend, zero overflow lines in every bucket — and ",[34,1460,1461],{},"319 audit records delivered to journald in that same window, of which 0 were lost."," Before the fix, ",[139,1464,1193],{}," had been growing 1:1 with every record. That's the closing proof. The Alertmanager alert cleared itself.",[49,1467,1469],{"id":1468},"the-correction-that-matters-most","The correction that matters most",[15,1471,1472,1473,254],{},"This project's own alert text claimed that during an overflow, tamper watches and canary reads had ",[34,1474,1475],{},"\"NOT recorded\"",[15,1477,1478,1481,1482,1485,1486,1489,1490,1493,1494,1497,1498,1501,1502,254],{},[34,1479,1480],{},"That was wrong."," The multicast hook runs ",[19,1483,1484],{},"before"," the failing unicast send — that's the ",[139,1487,1488],{},"if (skb_hook)"," line above the ",[139,1491,1492],{},"if (!sk)"," branch in ",[139,1495,1496],{},"kauditd_send_queue()",". journald, and therefore the whole intrusion-detection pipeline, received ",[34,1499,1500],{},"every"," record throughout. Intrusion detection was ",[34,1503,1504],{},"never blind at any point",[15,1506,1507],{},"What was lost was exclusively the copy destined for a daemon that wasn't there.",[15,1509,1510],{},"So: this was a monitoring and memory problem, not a security hole. Worth stating plainly, because the alert text made it sound like the latter for two days, and an alert that overstates its own blast radius is its own kind of bug — it was retracted at every site.",[49,1512,341],{"id":340},[15,1514,1515,1517],{},[34,1516,346],{}," Closed, deployed, and verified against live measurement rather than reasoning. The three things worth carrying forward:",[27,1519,1520,1523,1532],{},[30,1521,1522],{},"Multicast subscribers don't satisfy a kernel that wants a unicast consumer — and \"the records are showing up in the journal\" is not evidence that they aren't being lost.",[30,1524,1525,1527,1528,1531],{},[139,1526,967],{}," doesn't show the hold queue, so ",[139,1529,1530],{},"backlog: 0"," proves less than it looks like it proves.",[30,1533,1534],{},"A fix that only moves the recurrence date isn't a fix — and a reboot that resets the counter will happily let you believe it was.",[1536,1537,1538],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":356,"searchDepth":357,"depth":357,"links":1540},[1541,1542,1543,1544,1547,1552,1556,1557,1558],{"id":925,"depth":357,"text":926},{"id":944,"depth":357,"text":945},{"id":1004,"depth":357,"text":1005},{"id":1040,"depth":357,"text":1041,"children":1545},[1546],{"id":1107,"depth":1306,"text":1108},{"id":1133,"depth":357,"text":1134,"children":1548},[1549,1550,1551],{"id":1143,"depth":1306,"text":1144},{"id":1226,"depth":1306,"text":1227},{"id":1238,"depth":1306,"text":1239},{"id":1282,"depth":357,"text":1283,"children":1553},[1554],{"id":1356,"depth":1306,"text":1555},"On write_logs = no — a trade-off, not a trick",{"id":1397,"depth":357,"text":1398},{"id":1468,"depth":357,"text":1469},{"id":340,"depth":357,"text":341},"Three rounds of diagnosis, two plausible root causes that were both wrong, and the one counter shape that settled it.",{},"\u002Fgarden\u002Fkauditd-hold-queue-overflow",{"title":898,"description":1559},"kauditd-hold-queue-overflow",[1565,376,547],"infra","linux-audit","9qhAky0zs70XnEYq3vhPTe-NbfXeffYhuMN4MxEnN_M",{"id":1569,"title":913,"body":1570,"description":2062,"draft":367,"extension":368,"meta":2063,"navigation":370,"notice":371,"path":912,"seo":2064,"stage":374,"stem":2065,"tags":2066,"topic":1566,"__hash__":2067},"garden_en\u002Flinux-audit-subsystem.md",{"type":8,"value":1571,"toc":2052},[1572,1575,1582,1586,1619,1630,1634,1644,1671,1674,1681,1691,1695,1698,1726,1736,1740,1749,1806,1809,1818,1837,1922,1926,1929,1944,1947,1962,1965,1984,1991,2007,2014,2016,2024,2050],[11,1573,913],{"id":1574},"the-linux-audit-subsystem",[15,1576,1577,1578,1581],{},"Auditing on Linux is a kernel feature, not a logging library. The kernel notices things — a syscall touching a watched path, a BPF program being loaded, a PAM session opening, a systemd service starting — and emits ",[19,1579,1580],{},"records"," describing them. Everything after that is a delivery problem, and the delivery problem is where the interesting failures live.",[49,1583,1585],{"id":1584},"events-are-not-records","Events are not records",[15,1587,1588,1589,1592,1593,1595,1596,1599,1600,1603,1604,429,1607,1610,1611,1613,1614,1613,1617,254],{},"The first thing worth internalising, because every capacity calculation depends on it: one logical ",[34,1590,1591],{},"event"," is usually several ",[34,1594,1580],{},". A single watched-file access produces a ",[139,1597,1598],{},"SYSCALL"," record, a ",[139,1601,1602],{},"PROCTITLE"," record, often ",[139,1605,1606],{},"PATH",[139,1608,1609],{},"CWD"," too — all sharing one event serial number. A BPF program load produces ",[139,1612,1598],{}," + ",[139,1615,1616],{},"BPF",[139,1618,1602],{},[15,1620,1621,1622,1625,1626,1629],{},"Rule of thumb on a host like ",[102,1623,1624],{"href":1561},"chiba",": roughly ",[34,1627,1628],{},"3 records per event",". So a limit expressed in records is hit at about a third of the event count you'd naively expect, and anything that reasons about \"how many events until we hit the ceiling\" without that factor is off by 3×.",[49,1631,1633],{"id":1632},"kauditd-and-its-two-completely-separate-delivery-paths","kauditd, and its two completely separate delivery paths",[15,1635,1636,1639,1640,1643],{},[139,1637,1638],{},"kauditd"," is a kernel thread. Its job is to take records off the queue and deliver them. It has ",[34,1641,1642],{},"two"," delivery mechanisms, and they are not alternatives to each other:",[68,1645,1646,1659],{},[30,1647,1648,1651,1652,1654,1655,1658],{},[34,1649,1650],{},"Unicast, over netlink",", to a single registered userspace daemon — ",[139,1653,921],{},". The daemon registers itself with the kernel, and the kernel remembers its PID. This is the path the kernel considers ",[19,1656,1657],{},"the"," consumer.",[30,1660,1661,1664,1665,1667,1668,254],{},[34,1662,1663],{},"Multicast",", to any number of passive subscribers. On a systemd host this is where ",[139,1666,982],{}," attaches — that's how audit records end up in the journal tagged ",[139,1669,1670],{},"_TRANSPORT=audit",[15,1672,1673],{},"Here is the sentence that matters more than anything else on this page:",[256,1675,1676],{},[15,1677,1678],{},[34,1679,1680],{},"Multicast does not replace the unicast consumer.",[15,1682,1683,1684,1686,1687,1690],{},"If no process has registered as ",[139,1685,921],{},", the kernel treats delivery as ",[19,1688,1689],{},"failed"," — even while journald is happily receiving every single record over multicast. Journald being full of audit records is not evidence that the audit subsystem is healthy. It is evidence of exactly one thing: that the multicast hook is attached.",[49,1692,1694],{"id":1693},"the-three-queues","The three queues",[15,1696,1697],{},"There are three, and only one of them is visible in the obvious place:",[68,1699,1700,1711,1716],{},[30,1701,1702,1705,1706,1708,1709,254],{},[139,1703,1704],{},"audit_queue"," — the main queue. This is what ",[139,1707,967],{}," reports as ",[139,1710,1206],{},[30,1712,1713],{},[139,1714,1715],{},"audit_retry_queue",[30,1717,1718,1720,1721],{},[139,1719,1327],{}," — ",[34,1722,1723,1724,295],{},"not shown by ",[139,1725,967],{},[15,1727,1728,1729,1732,1733,1735],{},"All three are checked against the same ",[139,1730,1731],{},"audit_backlog_limit",". So ",[139,1734,1530],{}," is a statement about the main queue only. It is entirely consistent with a hold queue sitting pinned at the limit, dropping every new record.",[49,1737,1739],{"id":1738},"what-the-code-actually-does","What the code actually does",[15,1741,1742,1743,1094,1746,1748],{},"From ",[139,1744,1745],{},"kernel\u002Faudit.c",[139,1747,1496],{}," — read against v6.18, the kernel this was diagnosed on, rather than recalled:",[934,1750,1754],{"className":1751,"code":1752,"language":1753,"meta":356,"style":356},"language-c shiki shiki-themes github-light github-dark","while ((skb != skb_tail) && (skb = skb_dequeue(queue))) {\n    if (skb_hook)\n        (*skb_hook)(skb);          \u002F* multicast -> journald. ALWAYS, and first. *\u002F\n    if (!sk) {                      \u002F* no auditd registered *\u002F\n        if (err_hook)\n            (*err_hook)(skb, -ECONNREFUSED);\n        continue;\n    }\n    ...\n","c",[139,1755,1756,1761,1766,1771,1776,1782,1788,1794,1800],{"__ignoreMap":356},[1292,1757,1758],{"class":1294,"line":1295},[1292,1759,1760],{},"while ((skb != skb_tail) && (skb = skb_dequeue(queue))) {\n",[1292,1762,1763],{"class":1294,"line":357},[1292,1764,1765],{},"    if (skb_hook)\n",[1292,1767,1768],{"class":1294,"line":1306},[1292,1769,1770],{},"        (*skb_hook)(skb);          \u002F* multicast -> journald. ALWAYS, and first. *\u002F\n",[1292,1772,1773],{"class":1294,"line":1312},[1292,1774,1775],{},"    if (!sk) {                      \u002F* no auditd registered *\u002F\n",[1292,1777,1779],{"class":1294,"line":1778},5,[1292,1780,1781],{},"        if (err_hook)\n",[1292,1783,1785],{"class":1294,"line":1784},6,[1292,1786,1787],{},"            (*err_hook)(skb, -ECONNREFUSED);\n",[1292,1789,1791],{"class":1294,"line":1790},7,[1292,1792,1793],{},"        continue;\n",[1292,1795,1797],{"class":1294,"line":1796},8,[1292,1798,1799],{},"    }\n",[1292,1801,1803],{"class":1294,"line":1802},9,[1292,1804,1805],{},"    ...\n",[15,1807,1808],{},"Two things fall out of that ordering, and both are load-bearing.",[15,1810,1811,1814,1815,1817],{},[34,1812,1813],{},"The multicast hook runs first, unconditionally."," Before the ",[139,1816,1492],{}," check, before any error handling. Journald gets the record whether or not a unicast consumer exists. So a broken unicast path never blinds the journal-based pipeline.",[15,1819,1820,1832,1833,1836],{},[34,1821,1822,1823,1094,1825,1828,1829],{},"With no ",[139,1824,921],{},[139,1826,1827],{},"sk"," is NULL and every record goes to ",[139,1830,1831],{},"err_hook"," — which is ",[139,1834,1835],{},"kauditd_hold_skb()",":",[934,1838,1840],{"className":1751,"code":1839,"language":1753,"meta":356,"style":356},"static void kauditd_hold_skb(struct sk_buff *skb, int error)\n{\n    kauditd_printk_skb(skb);        \u002F* the \"callbacks suppressed\" spam *\u002F\n    if (!audit_default)\n        goto drop;\n    if (error == -EAGAIN) { ... retry queue ... }\n    if (!audit_backlog_limit ||\n        skb_queue_len(&audit_hold_queue) \u003C audit_backlog_limit) {\n        skb_queue_tail(&audit_hold_queue, skb);\n        return;\n    }\n    audit_log_lost(\"kauditd hold queue overflow\");\ndrop:\n    kfree_skb(skb);\n}\n",[139,1841,1842,1847,1852,1857,1862,1867,1872,1877,1882,1887,1893,1898,1904,1910,1916],{"__ignoreMap":356},[1292,1843,1844],{"class":1294,"line":1295},[1292,1845,1846],{},"static void kauditd_hold_skb(struct sk_buff *skb, int error)\n",[1292,1848,1849],{"class":1294,"line":357},[1292,1850,1851],{},"{\n",[1292,1853,1854],{"class":1294,"line":1306},[1292,1855,1856],{},"    kauditd_printk_skb(skb);        \u002F* the \"callbacks suppressed\" spam *\u002F\n",[1292,1858,1859],{"class":1294,"line":1312},[1292,1860,1861],{},"    if (!audit_default)\n",[1292,1863,1864],{"class":1294,"line":1778},[1292,1865,1866],{},"        goto drop;\n",[1292,1868,1869],{"class":1294,"line":1784},[1292,1870,1871],{},"    if (error == -EAGAIN) { ... retry queue ... }\n",[1292,1873,1874],{"class":1294,"line":1790},[1292,1875,1876],{},"    if (!audit_backlog_limit ||\n",[1292,1878,1879],{"class":1294,"line":1796},[1292,1880,1881],{},"        skb_queue_len(&audit_hold_queue) \u003C audit_backlog_limit) {\n",[1292,1883,1884],{"class":1294,"line":1802},[1292,1885,1886],{},"        skb_queue_tail(&audit_hold_queue, skb);\n",[1292,1888,1890],{"class":1294,"line":1889},10,[1292,1891,1892],{},"        return;\n",[1292,1894,1896],{"class":1294,"line":1895},11,[1292,1897,1799],{},[1292,1899,1901],{"class":1294,"line":1900},12,[1292,1902,1903],{},"    audit_log_lost(\"kauditd hold queue overflow\");\n",[1292,1905,1907],{"class":1294,"line":1906},13,[1292,1908,1909],{},"drop:\n",[1292,1911,1913],{"class":1294,"line":1912},14,[1292,1914,1915],{},"    kfree_skb(skb);\n",[1292,1917,1919],{"class":1294,"line":1918},15,[1292,1920,1921],{},"}\n",[49,1923,1925],{"id":1924},"the-bucket-not-the-burst","The bucket, not the burst",[15,1927,1928],{},"Read that function as a state machine and the failure mode names itself.",[15,1930,1931,1937,1938,1940,1941],{},[34,1932,1933,1934,1936],{},"Without ",[139,1935,921],{},", the hold queue grows monotonically."," Every record is held. Nothing ever drains it — draining is what the absent daemon was for. It grows until it reaches ",[139,1939,1731],{},", and from that instant ",[34,1942,1943],{},"every further record is lost, permanently, 1:1 with ordinary audit traffic.",[15,1945,1946],{},"That is a bucket filling up, not a burst overflowing. The distinction is the whole diagnostic value:",[68,1948,1949,1956],{},[30,1950,1951,1952,1955],{},"A ",[34,1953,1954],{},"burst"," problem spikes and recovers. Spike, quiet, spike, quiet.",[30,1957,1951,1958,1961],{},[34,1959,1960],{},"bucket"," problem is flat at exactly zero for hours — and then starts climbing and never stops.",[15,1963,1964],{},"If a loss counter has that second shape, no amount of raising the limit will fix it. A bigger bucket takes longer to fill. That is all it does.",[15,1966,1967,1970,1971,1973,1974,1976,1977,1980,1981,1983],{},[34,1968,1969],{},"The held records are pinned kernel memory."," They're visible in ",[139,1972,1183],{}," as ",[139,1975,1457],{},", and comparing its ",[139,1978,1979],{},"active"," count against ",[139,1982,1731],{}," is a direct read of how full the invisible queue is.",[49,1985,166,1987,1990],{"id":1986},"the-audit_default-branch",[139,1988,1989],{},"audit_default"," branch",[15,1992,1993,1995,1996,1999,2000,2003,2004,254],{},[139,1994,1989],{}," is 1 when auditing was enabled via the kernel command line (",[139,1997,1998],{},"audit=1",") — which is what the NixOS ",[139,2001,2002],{},"security.audit"," module sets. That's the branch that reaches ",[139,2005,2006],{},"skb_queue_tail(&audit_hold_queue, skb)",[15,2008,2009,2010,2013],{},"Were it 0, records would be ",[19,2011,2012],{},"dropped"," rather than held — still printk-spammed into dmesg, but not accumulating pinned memory. Worth knowing which side of that branch a given host is on, because it decides whether \"no consumer\" costs you memory or just noise.",[49,2015,341],{"id":340},[15,2017,2018,2020,2021,2023],{},[34,2019,346],{}," The delivery path is nailed down — read from the source, confirmed against a live host, and it explains a real incident end to end (that's the neighbouring note: ",[102,2022,898],{"href":1561},").",[15,2025,2026,2027,2030,2031,2034,2035,2038,2039,1094,2042,2045,2046,2049],{},"What this note deliberately does ",[19,2028,2029],{},"not"," cover: audit rule syntax beyond the ",[139,2032,2033],{},"-w path -p perms -k key"," watches that prompted it, ",[139,2036,2037],{},"auditctl"," tuning past ",[139,2040,2041],{},"backlog_limit",[139,2043,2044],{},"ausearch","\u002F",[139,2047,2048],{},"aureport",", or how any of this differs on non-systemd distributions. Those are all real and none of them were needed to explain the failure, so they're not asserted here rather than half-remembered.",[1536,2051,1538],{},{"title":356,"searchDepth":357,"depth":357,"links":2053},[2054,2055,2056,2057,2058,2059,2061],{"id":1584,"depth":357,"text":1585},{"id":1632,"depth":357,"text":1633},{"id":1693,"depth":357,"text":1694},{"id":1738,"depth":357,"text":1739},{"id":1924,"depth":357,"text":1925},{"id":1986,"depth":357,"text":2060},"The audit_default branch",{"id":340,"depth":357,"text":341},"What has to be true for an audit record to actually arrive — kauditd, its two delivery paths, and the queue nobody shows you.",{},{"title":913,"description":2062},"linux-audit-subsystem",[1565,376],"gG-WNw6YDJpV72Rx-6KBNxKf1eJClGO27WDnSeJY_7s",{"id":2069,"title":2070,"body":2071,"description":2308,"draft":367,"extension":368,"meta":2309,"navigation":370,"notice":371,"path":104,"seo":2310,"stage":374,"stem":2075,"tags":2311,"topic":2312,"__hash__":2313},"garden_en\u002Fmcp-server-authorization.md","MCP Server Authorization",{"type":8,"value":2072,"toc":2300},[2073,2076,2079,2086,2090,2093,2098,2101,2108,2112,2121,2128,2135,2139,2142,2147,2156,2171,2176,2183,2190,2195,2204,2209,2212,2225,2229,2232,2244,2250,2260,2264,2267,2287,2290,2292,2297],[11,2074,2070],{"id":2075},"mcp-server-authorization",[15,2077,2078],{},"The common reading is \"MCP servers need OAuth\", followed by bolting OAuth onto every server someone writes. The specification says something considerably more specific, and the specific version is easier to implement.",[15,2080,2081,2082,2085],{},"Everything below is from the Model Context Protocol specification, ",[34,2083,2084],{},"revision 2026-07-28",". Quotes are verbatim, and the RFC 2119 keywords are the spec's own — the difference between MUST and SHOULD is doing real work here.",[49,2087,2089],{"id":2088},"authorization-is-optional","Authorization is optional",[15,2091,2092],{},"Straight from the spec:",[256,2094,2095],{},[15,2096,2097],{},"\"Authorization is OPTIONAL for MCP implementations.\"",[15,2099,2100],{},"A server behind a reverse proxy that already authenticates, or one bound to localhost and reachable only by processes on that machine, is entitled to implement none of this. That's not a loophole; it's the design.",[15,2102,2103,2104,2107],{},"The question to ask isn't \"do I need auth?\" but \"",[34,2105,2106],{},"what's my transport, and what's my trust boundary?","\"",[49,2109,2111],{"id":2110},"over-stdio-dont","Over stdio, don't",[256,2113,2114],{},[15,2115,2116,2117,2120],{},"\"Implementations using an STDIO transport ",[34,2118,2119],{},"SHOULD NOT"," follow this specification, and instead retrieve credentials from the environment.\"",[15,2122,2123,2124,2127],{},"Worth reading twice, because it's stronger than permission to skip the work — it's a recommendation ",[19,2125,2126],{},"against"," doing it.",[15,2129,2130,2131,2134],{},"The logic is that over stdio the trust boundary is the ",[34,2132,2133],{},"process boundary",". The client launched the server as a subprocess. It already controls the server's environment, its lifetime, and its file descriptors. Layering a token exchange on top of that relationship doesn't add a boundary; it adds ceremony across a boundary that was never crossed. Credentials come from the environment, because whoever set the environment is already the trusted party.",[49,2136,2138],{"id":2137},"over-http-youre-an-oauth-21-resource-server","Over HTTP, you're an OAuth 2.1 resource server",[15,2140,2141],{},"This is where the obligations become concrete. An HTTP-transport MCP server is a resource server, and it owes four things.",[15,2143,2144],{},[34,2145,2146],{},"Publish protected resource metadata.",[256,2148,2149],{},[15,2150,2151,2152,2155],{},"\"MCP servers ",[34,2153,2154],{},"MUST"," implement OAuth 2.0 Protected Resource Metadata (RFC 9728)\"",[15,2157,2158,2159,2162,2163,2166,2167,2170],{},"In practice: serve ",[139,2160,2161],{},"\u002F.well-known\u002Foauth-protected-resource",", and include a ",[139,2164,2165],{},"resource_metadata"," pointer in the ",[139,2168,2169],{},"WWW-Authenticate"," header on a 401. That's how a client discovers which authorization server to go to without being told out of band.",[15,2172,2173],{},[34,2174,2175],{},"Validate the audience.",[256,2177,2178],{},[15,2179,2151,2180,2182],{},[34,2181,2154],{}," validate that access tokens were issued specifically for them as the intended audience\"",[15,2184,2185,2186,2189],{},"A token that is merely ",[19,2187,2188],{},"valid"," is not sufficient. It has to have been minted for this server. A server that accepts any well-signed token from a familiar issuer becomes a confused deputy for every other resource that issuer serves.",[15,2191,2192],{},[34,2193,2194],{},"Never pass the token through.",[256,2196,2197],{},[15,2198,2199,2200,2203],{},"\"The MCP server ",[34,2201,2202],{},"MUST NOT"," pass through the token it received from the MCP client\"",[15,2205,2206,2207,254],{},"Forwarding the caller's token to a downstream API is prohibited, not discouraged. If the server needs to call something on the user's behalf, it needs its own credential — which is the entire subject of ",[102,2208,6],{"href":372},[15,2210,2211],{},"Note how cleanly these last two fit together: if every resource server validates audience, and no server forwards tokens, then a token's audience always tells you truthfully where it was meant to go. Each rule is what makes the other one worth enforcing.",[15,2213,2214,2221,2222,2224],{},[34,2215,2216,2217,2220],{},"Include ",[139,2218,2219],{},"scope"," in the challenge."," This one is a SHOULD, not a MUST — the ",[139,2223,2169],{}," header should name the scope required, so a client can request the right thing on its second attempt rather than guessing.",[49,2226,2228],{"id":2227},"transport-obligations","Transport obligations",[15,2230,2231],{},"Independent of authorization, and easy to overlook because they aren't about tokens:",[256,2233,2234],{},[15,2235,2236,2237,2239,2240,2243],{},"\"Servers ",[34,2238,2154],{}," validate the ",[139,2241,2242],{},"Origin"," header on all incoming connections\"",[15,2245,2246,2247,2249],{},"An invalid ",[139,2248,2242],{}," gets a 403. This is what stops a web page the user happens to have open from driving a local MCP server through the browser — an attack that needs no credentials at all, because the browser helpfully supplies them.",[15,2251,2252,2253,2256,2257,254],{},"Additionally: servers running locally ",[34,2254,2255],{},"SHOULD"," bind only to localhost rather than all interfaces, and there is to be exactly ",[34,2258,2259],{},"one endpoint path",[49,2261,2263],{"id":2262},"the-actual-shape-of-the-decision","The actual shape of the decision",[15,2265,2266],{},"Reordered as a thing you can act on:",[68,2268,2269,2275],{},[30,2270,2271,2274],{},[34,2272,2273],{},"stdio transport"," → no authorization layer. Credentials from the environment. The spec recommends against anything more.",[30,2276,2277,2280,2281,2283,2284,2286],{},[34,2278,2279],{},"HTTP transport"," → OAuth 2.1 resource server: RFC 9728 metadata, strict audience validation, no passthrough, ",[139,2282,2219],{}," in the challenge, ",[139,2285,2242],{}," validation, localhost binding when local.",[15,2288,2289],{},"Most of the confusion in this area comes from reading advice written for the second case and applying it to the first.",[49,2291,341],{"id":340},[15,2293,2294,2296],{},[34,2295,346],{}," Sourced directly from one specification revision and quoted rather than paraphrased, so it's as reliable as that document — and it will age exactly as fast as the spec does. The revision is stamped at the top for that reason.",[15,2298,2299],{},"What this note doesn't cover: how any particular SDK implements the above, whether the client side holds up its end, or the dynamic client registration story. Those were not checked and aren't asserted.",{"title":356,"searchDepth":357,"depth":357,"links":2301},[2302,2303,2304,2305,2306,2307],{"id":2088,"depth":357,"text":2089},{"id":2110,"depth":357,"text":2111},{"id":2137,"depth":357,"text":2138},{"id":2227,"depth":357,"text":2228},{"id":2262,"depth":357,"text":2263},{"id":340,"depth":357,"text":341},"Whether an MCP server needs OAuth at all depends on its transport — and over stdio the spec says it shouldn't have any.",{},{"title":2070,"description":2308},[376,377],"mcp-auth","z-V6MLWNhR2V4fupN2eyFYJchodqruriGx8djSrlAUE",{"id":2315,"title":334,"body":2316,"description":2612,"draft":367,"extension":368,"meta":2613,"navigation":370,"notice":371,"path":333,"seo":2614,"stage":754,"stem":2320,"tags":2615,"topic":2616,"__hash__":2617},"garden_en\u002Fquestions-to-ask-an-idp.md",{"type":8,"value":2317,"toc":2600},[2318,2321,2324,2327,2344,2347,2353,2359,2363,2375,2378,2380,2383,2387,2394,2399,2405,2410,2419,2422,2425,2430,2433,2438,2452,2459,2463,2466,2471,2476,2479,2494,2499,2502,2506,2512,2515,2519,2526,2540,2543,2547,2550,2555,2558,2561,2576,2579,2581,2587,2593],[11,2319,334],{"id":2320},"questions-to-ask-an-idp",[15,2322,2323],{},"The usual framing is a shootout: line up the self-hostable identity providers, compare features, pick a winner, deploy it. That framing quietly assumes the thing worth testing here — that one system should do the whole job.",[15,2325,2326],{},"The more useful move is to split the job first:",[27,2328,2329,2334,2339],{},[30,2330,2331],{},[34,2332,2333],{},"Who authenticates humans?",[30,2335,2336],{},[34,2337,2338],{},"Who issues tokens for services?",[30,2340,2341],{},[34,2342,2343],{},"Does the backend use what those tokens say?",[15,2345,2346],{},"The first two are different problems. The first is about authentication policy, credential strength, recovery flows, and how much you trust a password. The second is about token shapes, claims, exchange, and what a backend can prove about a request's provenance.",[15,2348,2349,2352],{},[34,2350,2351],{},"They are allowed to have different answers."," Forcing both into one system means choosing a product that is inevitably stronger at one than the other, and then living with the weaker half. Once you accept two answers, each question gets to be decided on its own merits.",[15,2354,2355,2356,2358],{},"The third isn't a question about the identity provider at all — which is exactly why it gets skipped. You're shopping for an IdP, so you ask IdP questions. But an issuer that can express delegation buys nothing if the backend discards it on arrival, and that turned out to be the case here, measured rather than assumed: ",[102,2357,6],{"href":372}," has the result. It doesn't change the choice below. It does thin out one of the arguments for it.",[49,2360,2362],{"id":2361},"what-actually-forced-the-split","What actually forced the split",[15,2364,2365,2366,2368,2369,2372,2373,254],{},"The concrete requirement was ",[102,2367,378],{"href":372}," — a service acting on a person's behalf, with both identities visible to the backend. In RFC 8693 terms: token exchange ",[19,2370,2371],{},"with"," an ",[139,2374,208],{},[15,2376,2377],{},"This turns out to be an unusually good discriminator, because \"supports RFC 8693\" is a claim many products make while implementing meaningfully different subsets of it.",[49,2379,1144],{"id":1143},[15,2381,2382],{},"What follows was checked against each project's own documentation. Quotes are verbatim. Where something wasn't checked, that's stated rather than glossed.",[1105,2384,2386],{"id":2385},"kanidm","Kanidm",[15,2388,2389,2390,2393],{},"RFC 8693 support landed in ",[34,2391,2392],{},"v1.9.0 (2026-02-17)"," — but not in the delegating form. From its service-account documentation:",[256,2395,2396],{},[15,2397,2398],{},"\"Service accounts can exchange their API bearer token for OAuth2\u002FOIDC tokens (access\u002Fid\u002Frefresh) without user consent or interaction.\"",[15,2400,166,2401,2404],{},[139,2402,2403],{},"subject_token"," must be the service account's own API token. And explicitly:",[256,2406,2407],{},[15,2408,2409],{},"\"actor_token is not supported for this flow.\"",[15,2411,2412,2413,2418],{},"Read carefully, this is ",[19,2414,2415,2416],{},"effect-equivalent to ",[139,2417,141],{},": a service obtains a token for itself. That's a genuinely useful feature. It is not delegation, and no configuration turns it into delegation, because the parameter that would express delegation isn't accepted.",[1105,2420,2421],{"id":2421},"authentik",[15,2423,2424],{},"Supports both forms:",[256,2426,2427],{},[15,2428,2429],{},"\"authentik supports both impersonation and delegation as defined by RFC 8693.\"",[15,2431,2432],{},"With a version floor:",[256,2434,2435],{},[15,2436,2437],{},"\"Delegation and on-behalf-of token exchange are available in authentik 2026.8 and later.\"",[15,2439,2440,2441,1720,2444,2447,2448,2451],{},"Two packaging facts, both checked directly: nixpkgs currently carries ",[34,2442,2443],{},"2026.5.6",[19,2445,2446],{},"below"," the release where delegation arrived — and there is ",[34,2449,2450],{},"no NixOS module"," for it. So adopting it means both waiting on (or driving) a package bump and writing the service definition yourself.",[15,2453,2454,2455,2458],{},"Operationally it's a Python application plus a separate worker process, with a stated minimum of ",[34,2456,2457],{},"2 cores and 2 GB RAM",". Its own documentation describes Compose as being \"for test setups and small-scale production\".",[1105,2460,2462],{"id":2461},"zitadel","Zitadel",[15,2464,2465],{},"Supports delegation, and the documentation reads as more settled — no beta qualifier in the current docs.",[256,2467,2468],{},[15,2469,2470],{},"\"currently only a valid access token or ID token are allowed as actor token\"",[256,2472,2473],{},[15,2474,2475],{},"\"The user represented by the actor token must have the impersonation permission set, or else the request will be rejected\"",[15,2477,2478],{},"That second line matters: delegation isn't ambient, it's a permission you grant. There are four permission levels along two axes (IAM\u002FOrg × Admin\u002FEndUser), so the authority to act on someone's behalf is itself scoped rather than binary.",[15,2480,2481,2482,2485,2486,2489,2490,2493],{},"Operationally: Go and PostgreSQL, roughly ",[34,2483,2484],{},"512 MB RAM"," for the service, a NixOS module at ",[139,2487,2488],{},"services\u002Fweb-apps\u002Fzitadel.nix",", package ",[34,2491,2492],{},"2.71.7",". It federates to generic OIDC providers, and is explicit about what that means for the resulting user model:",[256,2495,2496],{},[15,2497,2498],{},"\"All user profiles are managed within ZITADEL, allowing for a unified view of user identities regardless of the IdP used for authentication.\"",[15,2500,2501],{},"It also does passkeys natively per FIDO2\u002FWebAuthn.",[1105,2503,2505],{"id":2504},"keycloak-not-evaluated","Keycloak — not evaluated",[15,2507,2508,2511],{},[34,2509,2510],{},"This was not checked."," Keycloak was excluded early on operational grounds — a JVM deployment, heavier than the rest of the estate — and its delegation capabilities were never investigated.",[15,2513,2514],{},"Stating that plainly matters. Keycloak is the most widely deployed option in this category and may well handle delegation perfectly. It isn't ranked here because no ranking was performed on it, and presenting an unexamined product alongside three examined ones would misrepresent how much work went into the comparison.",[49,2516,2518],{"id":2517},"kanidms-own-axis","Kanidm's own axis",[15,2520,2521,2522,2525],{},"It would be easy to read the above as \"Kanidm fell short.\" That misreads the situation, and its own comparison documentation makes the point: it offers stronger authentication policy, ",[34,2523,2524],{},"WebAuthn attestation"," (which the others don't provide), Unix authentication integration, and a deliberate choice to implement its own database rather than depend on an external SQL server.",[15,2527,2528,2529,2532,2533,2535,2536,2539],{},"Those are real strengths, and they all sit on the ",[19,2530,2531],{},"first"," axis — authenticating humans well. None of them are weakened by the absence of ",[139,2534,208],{},", because delegation is a question on the ",[19,2537,2538],{},"second"," axis.",[15,2541,2542],{},"Kanidm chose a design space on purpose. The requirement here simply grew outside it. That's a different finding from \"built badly\", and the distinction is the whole reason the two-question framing is worth adopting.",[49,2544,2546],{"id":2545},"the-plan","The plan",[15,2548,2549],{},"Not yet implemented — this is the decision, not a report on living with it:",[15,2551,2552],{},[34,2553,2554],{},"Keep Kanidm as the identity source for humans. Add Zitadel alongside it for token issuance and delegation, federating to Kanidm for authentication.",[15,2556,2557],{},"Humans keep authenticating where they already do. Services get an issuer that can mint tokens naming two parties. Neither system is asked to do the thing it wasn't designed for.",[15,2559,2560],{},"Two reasons this is an addition rather than a migration:",[27,2562,2563,2566],{},[30,2564,2565],{},"Real services already authenticate against Kanidm in production — the git forge and the fediverse server.",[30,2567,2568,2571,2572,254],{},[34,2569,2570],{},"The passkeys are domain-bound."," Moving would ask every user to re-enrol their hardware. That constraint deserves its own note: ",[102,2573,2575],{"href":2574},"\u002Fgarden\u002Fwebauthn-credentials-are-domain-bound","WebAuthn Credentials Are Domain-Bound",[15,2577,2578],{},"The second reason is the one that would still apply even if the first didn't.",[49,2580,341],{"id":340},[15,2582,2583,2586],{},[34,2584,2585],{},"Seedling."," The decision is made; nothing is deployed. Most of the above is documentation-checking, which is exactly the kind of evidence that survives contact with reality least well — federation in particular tends to be where the interesting problems are, and none of them have been met yet.",[15,2588,2589,2590,2592],{},"One piece has since been measured rather than read: the exchange itself works end to end against the real backend, and that backend ignores the ",[139,2591,161],{}," claim. The plan below is unchanged by it — the reasons for it were never only about audit — but one of its arguments is thinner than it looked.",[15,2594,2595,2596,2599],{},"Two known gaps, kept visible rather than quietly dropped: ",[34,2597,2598],{},"Keycloak was never evaluated",", and running two identity systems has an ongoing operational cost that hasn't been paid yet and therefore can't honestly be reported on. This note gets revisited once it has.",{"title":356,"searchDepth":357,"depth":357,"links":2601},[2602,2603,2609,2610,2611],{"id":2361,"depth":357,"text":2362},{"id":1143,"depth":357,"text":1144,"children":2604},[2605,2606,2607,2608],{"id":2385,"depth":1306,"text":2386},{"id":2421,"depth":1306,"text":2421},{"id":2461,"depth":1306,"text":2462},{"id":2504,"depth":1306,"text":2505},{"id":2517,"depth":357,"text":2518},{"id":2545,"depth":357,"text":2546},{"id":340,"depth":357,"text":341},"Which identity provider?",{},{"title":334,"description":2612},[376,377,547],"identity-provider","im6mTU-ZGplSH07m1Rrp3imlket6iGZ9rJoraVNuU_w",{"id":2619,"title":2620,"body":2621,"description":2723,"draft":367,"extension":368,"meta":2724,"navigation":370,"notice":371,"path":2725,"seo":2726,"stage":754,"stem":2625,"tags":2727,"topic":2625,"__hash__":2729},"garden_en\u002Fweb-analytics.md","Web Analytics",{"type":8,"value":2622,"toc":2717},[2623,2626,2629,2633,2639,2643,2658,2662,2673,2679,2689,2695,2705,2707],[11,2624,2620],{"id":2625},"web-analytics",[15,2627,2628],{},"I want to know which parts of this site are actually interesting, and what's driving traffic to it. That's it — not behavioral profiling, not ad targeting, just \"which pages, from where.\"",[49,2630,2632],{"id":2631},"the-constraint-self-hosted-or-it-doesnt-count","The constraint: self-hosted, or it doesn't count",[15,2634,2635,2636,2638],{},"Everything else here — self-hosted fonts, no third-party embeds, its own Forgejo\u002FGoToSocial instances — exists specifically to avoid sending visitor data to companies that have no reason to have it. Cloud-hosted analytics (including cloud Plausible) would undo that for the one thing that watches ",[19,2637,1500],{}," page load. So: self-hosted, full stop.",[49,2640,2642],{"id":2641},"privacy-friendly-is-doing-a-lot-of-marketing-work","\"Privacy-friendly\" is doing a lot of marketing work",[15,2644,2645,2646,2649,2650,2653,2654,2657],{},"Worth being precise about this rather than repeating the tagline: tools in this category (Plausible, GoatCounter, Umami) still process IP, User-Agent, referrer, and page path per visit — that's still personal data being handled, just briefly. The way they avoid cookies is usually a daily-rotating salted hash of ",[139,2647,2648],{},"IP + User-Agent + domain",", producing an ephemeral \"unique visitor\" count with no persisted identifier. That's real and meaningfully better than cross-site, cookie-based tracking — no third-party sharing, no long-lived ID, nothing correlated across other sites. But it's not \"no tracking happens.\" Self-hosting solves ",[19,2651,2652],{},"where the data goes",", not ",[19,2655,2656],{},"whether collection is happening",". Both matter; they're not the same claim.",[49,2659,2661],{"id":2660},"the-options-considered","The options considered",[15,2663,2664,2667,2668,2672],{},[34,2665,2666],{},"Plausible"," — the default answer in this space, and the one this site's own ",[102,2669,2671],{"href":2670},"\u002Fprivacy","Privacy Policy"," explicitly ruled out before this note existed. Self-hostable, but requires ClickHouse alongside Postgres. ClickHouse is a columnar OLAP database — built for aggregating huge event volumes fast, which is exactly an analytics dashboard's query shape, but it doesn't scale down: even a minimal install wants several GB of RAM as a baseline, sized for workloads far bigger than a personal site generates. On infrastructure where every service's memory is already being actively budgeted, that's disproportionate to the actual data volume involved.",[15,2674,2675,2678],{},[34,2676,2677],{},"Umami"," — dodges the ClickHouse problem (Postgres or MySQL instead), similar cookie-less design. Ruled out for a more personal reason: it's a Node\u002FTypeScript stack, and a single self-contained binary fits this project's self-hosting taste a lot better than a Node app with its own dependency tree to keep patched.",[15,2680,2681,2684,2685,2688],{},[34,2682,2683],{},"Matomo"," — the mature, full-featured option, also Postgres\u002FMySQL-backed. Worth naming as the honest counter-example to the \"privacy-friendly\" framing above: out of the box it does ",[19,2686,2687],{},"much"," more than Plausible\u002FGoatCounter\u002FUmami — session data, optional fingerprinting, heatmap\u002Fsession-replay add-ons. It only becomes privacy-light if you deliberately disable most of what it ships with. Not a default-safe choice the way the others are.",[15,2690,2691,2694],{},[34,2692,2693],{},"PostHog (self-hosted)"," — ruled out early. Full product-analytics suite, session replay available, more invasive by default posture than this needed. Wrong direction.",[15,2696,2697,2700,2701,2704],{},[34,2698,2699],{},"GoatCounter"," — landed here. Single self-contained Go binary, SQLite by default, packaged in nixpkgs (",[139,2702,2703],{},"pkgs\u002Fby-name\u002Fgo\u002Fgoatcounter\u002Fpackage.nix","), so it's a normal NixOS service module rather than a Docker Compose stack. Built by one developer with privacy stated as a design constraint, not a marketing layer — and unlike a claim in a README, this one has a track record: v2.5.0 (2023) removed raw User-Agent storage in favor of parsed browser\u002FOS only, v2.6.0 (2024) stopped storing individual pageviews by default. The direction of travel is toward storing less, not more. The dashboard UI is dated-looking (server-rendered, not a modern SPA — accessibility was the stated priority over visual polish) but still actively released: v2.7.0 shipped December 2025.",[49,2706,341],{"id":340},[15,2708,2709,2710,2712,2713,2716],{},"Still a ",[34,2711,754],{}," — the decision is made, the infrastructure side (does chiba have the headroom, what the NixOS module looks like) isn't done yet. This note exists because writing down ",[19,2714,2715],{},"why"," GoatCounter and not the others felt worth doing before the implementation details push the reasoning out of memory.",{"title":356,"searchDepth":357,"depth":357,"links":2718},[2719,2720,2721,2722],{"id":2631,"depth":357,"text":2632},{"id":2641,"depth":357,"text":2642},{"id":2660,"depth":357,"text":2661},{"id":340,"depth":357,"text":341},"Picking a self-hosted, honestly-described analytics tool for this site.",{},"\u002Fgarden\u002Fweb-analytics",{"title":2620,"description":2723},[2728,547],"privacy","aWpDv7KB1sRSXomN7yFW3HXKrVDy-RFPmjEBSqz3M70",{"id":2731,"title":2575,"body":2732,"description":2900,"draft":367,"extension":368,"meta":2901,"navigation":370,"notice":371,"path":2574,"seo":2902,"stage":374,"stem":2736,"tags":2903,"topic":2904,"__hash__":2905},"garden_en\u002Fwebauthn-credentials-are-domain-bound.md",{"type":8,"value":2733,"toc":2892},[2734,2737,2744,2747,2751,2762,2765,2769,2774,2777,2780,2808,2815,2819,2825,2828,2847,2851,2865,2872,2876,2879,2885,2887],[11,2735,2575],{"id":2736},"webauthn-credentials-are-domain-bound",[15,2738,2739,2740,2743],{},"A WebAuthn credential is bound to the ",[34,2741,2742],{},"relying-party ID"," — the domain it was registered under. That binding is part of the credential itself, not a setting alongside it.",[15,2745,2746],{},"Change the domain and every registered credential becomes worthless. Not degraded, not migratable: worthless. Every user re-registers their hardware.",[49,2748,2750],{"id":2749},"why-this-is-sharper-than-it-sounds","Why this is sharper than it sounds",[15,2752,2753,2754,2757,2758,2761],{},"The binding is deliberate, and it's the reason passkeys resist phishing. A credential registered for ",[139,2755,2756],{},"id.example.com"," will not be offered on ",[139,2759,2760],{},"id-example.com",", because the browser refuses to match them. The user cannot be talked into making that mistake, because the check happens below the layer where persuasion works.",[15,2763,2764],{},"The property that makes the credential unphishable is exactly the property that makes it immovable. You don't get one without the other.",[49,2766,2768],{"id":2767},"the-part-that-catches-people","The part that catches people",[15,2770,2771],{},[34,2772,2773],{},"This decision is made at first deploy, usually without being recognised as a decision.",[15,2775,2776],{},"You pick a hostname for the login service because you need to put it somewhere. Nothing warns you. Everything works. The choice looks like a DNS detail, filed mentally alongside the reverse-proxy config.",[15,2778,2779],{},"Then it sets. And it sets harder than almost anything else in the deployment:",[68,2781,2782,2789,2795,2801],{},[30,2783,2784,2785,2788],{},"Change your ",[34,2786,2787],{},"database","? Dump and restore.",[30,2790,2784,2791,2794],{},[34,2792,2793],{},"hosting provider","? Move the volumes.",[30,2796,2784,2797,2800],{},[34,2798,2799],{},"identity provider","? Export users, import users, reconfigure the clients.",[30,2802,2803,2804,2807],{},"Change the ",[34,2805,2806],{},"domain your credentials were registered under","? Contact every user and ask them to re-enrol their security keys, one at a time, from a login page they can no longer log into with the thing they'd normally use.",[15,2809,2810,2811,2814],{},"That last one isn't a data migration problem. There's no data to move. It's a ",[19,2812,2813],{},"coordination"," problem with every human who ever registered, and the fallback path is whatever recovery mechanism you set up — which is now load-bearing for your entire user base simultaneously.",[49,2816,2818],{"id":2817},"which-means-the-hostname-is-an-architectural-commitment","Which means the hostname is an architectural commitment",[15,2820,2821,2822],{},"Worth stating as a rule: ",[34,2823,2824],{},"the hostname you serve WebAuthn from is a long-term commitment, and it should be chosen with the same seriousness as a data model.",[15,2826,2827],{},"Practically:",[68,2829,2830,2841,2844],{},[30,2831,2832,2833,2836,2837,2840],{},"Pick a hostname you can keep independent of the software behind it. A domain named after the ",[19,2834,2835],{},"function"," survives replacing the ",[19,2838,2839],{},"product","; one named after the product does not.",[30,2842,2843],{},"Understand the relationship between the RP ID and origin before first registration, not after.",[30,2845,2846],{},"Treat \"we might move this later\" as a reason to choose more carefully now, not as a reason to defer.",[49,2848,2850],{"id":2849},"seen-in-the-wild","Seen in the wild",[15,2852,2853,2854,429,2857,2860,2861,2864],{},"This isn't theoretical caution. A concrete instance: the Kanidm deployment behind this site's services carries an explicit warning in its own module — ",[139,2855,2856],{},"domain",[139,2858,2859],{},"origin"," are pinned at first deploy to ",[139,2862,2863],{},"id.byteflavour.dev"," and must never change afterwards, because Kanidm bakes the domain into every WebAuthn credential it issues.",[15,2866,2867,2868,2871],{},"Notably, that warning had to be written ",[19,2869,2870],{},"into the deployment"," to be visible at the moment it mattered. The constraint comes from the standard, but nothing in the act of deploying surfaces it on its own.",[49,2873,2875],{"id":2874},"the-consequence-downstream","The consequence downstream",[15,2877,2878],{},"This is why replacing an identity provider is much harder than its feature list suggests. Comparing two products on capabilities implies you could swap one for the other. If passkeys are registered against the current one's domain, you cannot — at least not without an event that touches every user.",[15,2880,2881,2882,2884],{},"It's a live constraint here: it's one of the two reasons the plan in ",[102,2883,334],{"href":333}," adds a second system rather than migrating to it. The other reason — existing services already depending on it — would eventually expire as those services were reconfigured. This one wouldn't.",[49,2886,341],{"id":340},[15,2888,2889,2891],{},[34,2890,346],{}," The mechanism is standardised and settled; nothing here is expected to change. What would improve this note is the other side of it — what a domain migration actually costs when someone has to run one, how well recovery codes carry the load, and whether related-origin requests meaningfully soften any of it. None of that has been tested, so none of it is claimed.",{"title":356,"searchDepth":357,"depth":357,"links":2893},[2894,2895,2896,2897,2898,2899],{"id":2749,"depth":357,"text":2750},{"id":2767,"depth":357,"text":2768},{"id":2817,"depth":357,"text":2818},{"id":2849,"depth":357,"text":2850},{"id":2874,"depth":357,"text":2875},{"id":340,"depth":357,"text":341},"The relying-party ID gets chosen at first deploy, usually without anyone noticing, and it binds harder than the database does.",{},{"title":2575,"description":2900},[376,377],"webauthn","ioI6_BamLC_R3uUEQUBOiCbzgQhfKVf0XXDW1nh1BFw",1789414695589]