* caddyhttp: Combine repeated expression matchers in a matcher set
The Caddyfile parser groups repeated matchers of the same type and hands
all their segments to the module, but the expression matcher only read
the first segment, so every later `expression` line in a named matcher
or `not` block was dropped without an error. Read every segment and AND
the expressions, like the other matchers in a set.
* caddyhttp: Put each joined expression on its own line
A trailing // comment in one expression no longer swallows the closing
parenthesis and the expressions joined after it.
* caddyhttp: Reject repeated expression matchers instead of joining them
A second expression line in a named matcher or a not block is now a
Caddyfile error that points to combining them with && in one expression,
instead of being dropped or joined with an implied operator.
(cherry picked from commit d4165a4eb1)
* caddyhttp: Skip matchers of grouped routes once the group is satisfied
* caddyhttp: Test that a skipped grouped route keeps regexp captures
A later rewrite in the same group must not run its path_regexp matcher
and overwrite {re.x.1} from the rewrite that was used.
(cherry picked from commit 5538b947d2)
* caddyhttp: preserve expired request-body read deadlines
* caddyhttp: retain only expired drain deadlines
Track the last successfully installed read deadline and retain it after a terminal timeout only when draining is enabled and the deadline has expired. Failed deadline changes preserve the previous installed value.
Cover early and wrapped timeouts, non-draining requests, failed deadline installation and replacement, and cleanup retries.
* caddyhttp: shorten installed deadline field name
(cherry picked from commit c31d9a71ed)
* cmd: unbuffer default logger on startup failure
Restore default logger to unbuffered console logger on cmdRun error so WrapCommandFuncForCobra can output the error.
* add regression tests
(cherry picked from commit d588d5c8f0)
* reverseproxy: Only retry responses for expressions that use response data
Since #7569 every lb_retry_match set with an expression is evaluated against
upstream responses. An expression that only tests the request, such as
{http.request.method} == "POST", then matched every successful response:
the request was sent again to the next upstreams and the client got a 502
once retries ran out. Only treat expressions referencing {rp.*} placeholders
as response conditions.
* reverseproxy: Only count placeholders set from the response as response data
* reverseproxy: Read response placeholders from the checked expression
The expression matcher records the placeholder names its checked CEL
expression reads, so retry_match finds direct ph() calls and ignores
escaped placeholder text in string literals. A placeholder name only
known at evaluation time counts as response data.
* reverseproxy: Match response placeholder names exactly
Only the header placeholders are matched by prefix now, and both
upstream latency placeholders are listed. MatchExpression.Placeholders
returns a copy, so callers cannot change the provisioned matcher.
(cherry picked from commit 7c7456353e)
* caddyhttp: ignore case when matching hosts in logger_names
* caddyhttp: test case-insensitive logger_names through App.Provision
* caddyhttp: test case-insensitive logger_names in integration harness
(cherry picked from commit 7910dd94ee)
* encode: Don't write an empty encoded stream for HEAD responses
When nothing was written (e.g. a HEAD request), Close() still closed the
encoder, which wrote the empty stream trailer to the client. The server
then set Content-Length to that size (20 for gzip, 13 for zstd) instead
of omitting it as for the matching GET response.
* encode: Check resp.ContentLength in the HEAD test
* encode: Track body writes to decide whether to close the encoder
(cherry picked from commit 093fd4b7ec)
* httpcaddyfile: keep a site block's TLS policy for ECH public names
The global ech option added its own automation policy for every public
name, even when a site block already had a policy for that name. If the
site block set its own TLS options, adapting failed with "hostname
appears in more than one automation policy". If a hostless catch-all
with the same settings was also present, the site's policy was merged
away and the name got only the global settings.
Skip public names that already have a policy, as tls_automate_names
does, and only add the ECH policy when it still has subjects, since a
policy without subjects acts as a catch-all.
* httpcaddyfile: check ECH public names the way the TLS app matches them
The check for an existing site policy only matched a subject spelled
exactly like the ECH public name. A site policy for *.example.com, or
for example.com when the public name is written Example.COM, already
covers that name at runtime, but the global ECH policy was still added
and sorted ahead of it, so the site's issuer settings were overridden.
Lowercase and trim the public name as ECH provisioning does and compare
it with certmagic.MatchWildcard, which is how the TLS app picks a policy
for a name.
(cherry picked from commit c8591a1e46)
* reverseproxy: Keep own headers after 1xx responses
Since #6656 the 1xx handler clears the whole header map after writing the
informational response, which also drops headers set before proxying (for
example by the header handler). Restore those after clearing, as #5091 did.
* reverseproxy: Use maps.Copy to restore own headers
* reverseproxy: Check the 103 reaches the client in the 1xx headers test
(cherry picked from commit 519be12306)
Wrap backend dial failures as reverseproxy.DialError so non-GET requests can be retried when no connection was made.
AI-assisted implementation and test; reviewed by zemamba.
Signed-off-by: zemamba <zemamba@gmail.com>
(cherry picked from commit 221ebc4503)
* templates: Delete file headers on any template error
A template that fails to parse or execute (not only {{httpError}}) left
the Content-Length, Etag and related headers set by file_server on the
error response, so clients received a 500 that declared the file's size
but had no body.
* templates: Also delete Content-Encoding and Content-Range on errors
Precompressed and ranged file responses could leave these headers on the
replacement error response.
(cherry picked from commit 6727b2f3a9)
A handler setting a write deadline through http.ResponseController
reached the connection directly, and the next Write or Flush through
IdleTimeoutWriter replaced it with now+write_idle_timeout. Handlers
bounding a single write, like Mercure's dispatch_timeout, which drops
SSE subscribers that can't receive an update within 5s, were silently
extended to the 1 minute default. On HTTP/2, clearing between writes
also wiped the handler deadline.
IdleTimeoutWriter now implements SetWriteDeadline, keeps the handler
deadline as a ceiling for idle resets, and restores it instead of
HardDeadline alone when clearing between writes. A handler can
tighten the idle timeout but not loosen it.
(cherry picked from commit 9c44b6685e)
* caddyfile: surface OS error when an import file is inaccessible
filepath.Glob ignores file system errors, so an import path that
exists but cannot be stat'ed (e.g. permission denied on a parent
directory) was reported as 'File to import not found'. Stat the
literal path when Glob finds no matches and report the real error
unless the file genuinely does not exist.
Fixes#8161
* caddyfile: skip through-a-file import case on Windows
Windows reports a path through a regular file as not existing rather
than ENOTDIR, so os.Stat cannot distinguish it from a missing file
there. The permission-denied case from #8161 is unaffected.
(cherry picked from commit c1acb2fcba)
directoryListing built each symlink's lookup path from the escaped
request path. In a directory whose name needs escaping, such as one with
a space, the stat failed, so directory symlinks were listed as files,
file symlinks showed the link's own size, and reveal_symlinks showed no
target.
Use the decoded path, which the function already computes for the page
title. The escaped path is still used for the page path and item URLs.
(cherry picked from commit 96eadf18f6)
read_idle_timeout and write_idle_timeout default to 1 minute when zero,
and the wrappers are only installed for positive values, so a negative
value already disables them. Document it and cover it with tests, since
it is the only way to opt out, for instance behind a reverse proxy that
already protects from slow clients.
(cherry picked from commit c65c248336)
* caddyhttp: only bound HTTP/2 body reads in flight with the idle timeout
On HTTP/2, the read deadline is a timer that fails the request body
when it fires, even with no read in flight. IdleTimeoutReader left the
idle deadline armed between reads, so a handler that stopped reading
for longer than read_body_idle (1 minute by default) lost the body,
although the client was only held back by flow control. For example,
reverse_proxy uploading to an upstream that's slow to accept the body
failed with a 502.
This is the read side of #8118. Like IdleTimeoutWriter does since
read on HTTP/2, so the idle deadline only bounds reads in flight. A
client that stops sending is still cut off, since the deadline is set
before each read. HTTP/1 and HTTP/3 only check the deadline during a
read, so they're unchanged.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* caddyhttp: track a released hard read deadline as armed
From review: with HardDeadline set, releasing the idle deadline between
reads left an HTTP/2 timer armed (at HardDeadline) while deadlineSet
recorded that none was, so clearDeadlineLocked would skip clearing it.
Track which deadline is armed (none, idle, or only the hard one) instead
of a bool, and have each check ask the question it means: re-arming and
releasing only apply to an armed idle deadline, while clearing applies
to any.
The writer's deadlineSet already only means "idle deadline armed",
since nothing clears the hard deadline it leaves; document that.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* caddyhttp: clear released hard deadline at handler end, skip no-op sets
From review:
- After a partial read releases the idle deadline to HardDeadline, a
handler that returns without reading again left that timer armed past
HandlerDone, which only cleared after a terminal error. HandlerDone
now also clears a released hard deadline.
- When the idle deadline is capped to HardDeadline, the reader and
writer set the same deadline again when releasing it, and the writer
once more in HandlerDone. On HTTP/2 each is a message to the serve
loop. Both now remember the deadline they armed and skip setting it
again. Within a request, only these wrappers set the client's
deadlines.
Replaces the earlier test that faked a terminal error with the partial
read followed directly by HandlerDone.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* caddyhttp: only skip re-setting an armed hard deadline
Skipping any deadline equal to the one already armed also skipped idle
deadlines on Windows, where two calls to time.Now in the same clock tick
return the same time, so successive idle deadlines can be equal. That
broke tests counting deadline calls there. Only skip HardDeadline, the
case from review where the idle deadline is capped to it; other
deadlines are always set.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* caddyhttp: restore deadlines without trusting cached transport state
Always set the requested read and write deadlines because another handler can change them through its own response controller. Remove the armedAt shortcut and cover intervening deadline changes for reads, writes, ReadFrom, flushes, and HandlerDone.
AI-assisted with OpenAI Codex (GPT-6); tested under human direction.
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 30d3bd2a89)
[backport: test adapted to the 2.11 finalizeResponse signature, which
lacks the upstream address argument added by #7649]
(cherry picked from commit f7eeb1d65c)
Since #7017, every *fs.PathError from stat becomes fs.ErrInvalid, and
that includes ENOTDIR. So a request like /index.php/foo got 400 instead
of 404, and pass_thru no longer fell through to the next handler.
Map ENOTDIR to fs.ErrNotExist before that check. Other path errors, such
as NUL bytes, overly long names, or transient I/O failures, keep their
current handling.
(cherry picked from commit 7ba5e147ac)
* timeouts: take precedence over server-wide and enclosing idle timeouts
The timeouts handler wrapped the request body and response writer again,
on top of the server's own idle timeout wrappers. The wrapper closest to
the connection sets the deadline last, so the server-wide read_body_idle
and write_idle (on by default since 2.11.6) silently overrode the
route's settings. The same happened between nested timeouts handlers,
where the outer, less specific one won.
The server now makes its idle timeout wrappers available through the
request context, and the timeouts handler overrides their settings while
it runs, restoring them when it returns. It only wraps when there's
nothing to override, and then registers its own wrappers so that nested
handlers override those. The server-wide read_timeout and write_timeout
still cap the whole transfer. A handler that doesn't set
max_write_chunk keeps the server-wide one.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* timeouts: re-arm deadlines on override and suspend outer min-rate window
From review:
- On HTTP/2, a read or write deadline is a timer that fires even with
no read or write in flight. A deadline armed before a timeouts handler
started kept the outer timeout and could fail the body (or reset the
stream) inside a longer inner scope before the next read reset it.
Override and restore now move an armed deadline to the settings that
apply.
- Time spent under an override counted against the outer MinRate
window, so the restored deadline could already have expired. The
outer window is now suspended while overridden: its start moves
forward by the time spent overridden, and bytes transferred in the
meantime don't count towards it.
Adds an HTTP/2 read-before-inner-scope regression test and nested
minimum-rate coverage.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 8f549e6df1)
* core: Raise CAP_NET_BIND_SERVICE when binding privileged ports
When CAP_NET_BIND_SERVICE is in the permitted set but not the effective
set, raise it on a dedicated locked OS thread just for the bind, then
lower it again and let the thread exit.
This allows the binary to be granted the capability with
`setcap cap_net_bind_service=+p` instead of `=+ep`. With the effective
bit set, the kernel refuses to exec the binary at all when the
capability can't be granted (e.g. `docker run --cap-drop ALL`, or an
empty systemd CapabilityBoundingSet), even if no privileged port is
ever bound.
* core: Move bind capability helpers into internal package
(cherry picked from commit c8341d5b69)
* caddyfile: Keep an opening brace that ends the input
Format writes an opening brace lazily: the '{' case records the pending
brace and continues, and the brace is only emitted once a following
regular character is processed. When the input ends while a brace is
still pending, the read loop exits and nothing flushes it, so the brace
is dropped. 'caddy fmt' on a Caddyfile whose last line has just opened a
block therefore deletes the brace the user typed, and with --overwrite
that result is written back over the file.
Flush a pending brace after the loop ends. This also restores
idempotence for inputs such as '{ {', which formatted to '{' and then to
nothing.
* caddyfile: Flush a pending brace before another one replaces it
The '{' case overwrote the pending-brace state without emitting the
brace already waiting, so adjacent braces lost one: Format("{{") gave
just "{". Only the space-separated path flushed, and it did so with an
inline block.
Move that block into a flushOpenBrace closure and call it from the three
places a pending brace has to be emitted: the existing space-separated
path, the '{' case before it replaces the state, and the end of input.
Using it at the end of input also indents a trailing brace the same way
the other paths do.
CertMagic assembles an event's data before emit() can look at OnEvent,
because Go evaluates arguments first, so tls_get_certificate costs a map
and a ClientHello copy on every handshake even when nothing will read
them. caddyserver/certmagic#406 added Config.ShouldEmitFunc so an
embedder can answer that question first.
Export App.ShouldEmit and pass it along with OnEvent. The predicate has
to account for every observer, not just subscribed handlers: Emit writes
a debug line for each event, so an event is observable whenever debug
logging is on. Emit now asks the same method it exports, so the two
cannot drift.
Benchmarks are included; measurements are in the pull request.
With the Go 1.26 floor, golangci-lint's modernize analyzer (errorsastype)
flags the three errors.As calls that look for RequestBodyLimitError, which
turned the Lint workflow red on master after #8056. Replace them with the
generic errors.AsType; behavior is unchanged.
Assistance disclosure: this change was made by an AI agent (Claude Sonnet
5.5, model ID claude-sonnet-5-5) acting as Matthew Holt's agent and under
his approval. Build, go vet and the caddyhttp and caddyhttp/templates tests
pass locally; golangci-lint itself was not run locally.
Co-authored-by: Claude Code <noreply@anthropic.com>
Track pending HTTP server shutdowns across configuration reloads so
termination waits for responses still served by an older configuration.
Bound that wait by the terminating app's grace period and retain each
reload's grace context until its servers finish shutting down.
Cover HTTP/1.1, HTTP/2, and finite termination grace periods with
regression tests.
* httpcaddyfile: new tls_automate_names global option
Provisioning a certificate for a name that is not served requires giving
it a site block of its own:
*.example.com {
}
foo.example.com {
respond "Real site"
}
That asks the tls app for the wildcard, but it also adds a route to the
http app, so every name pointed at the server that has no site block of
its own -- bar.example.com here -- gets an empty but valid response
rather than no match at all. Wanting a certificate and wanting to serve
a name are separate things, and the Caddyfile had no way to say only the
first.
Name them in the new global option instead:
{
tls_automate_names *.example.com
}
foo.example.com {
respond "Real site"
}
The names are added to the automate certificate loader and to an
automation policy built from the global options, so a name listed here
is managed exactly as it would be from a site block, with the same
issuers; the only difference in the adapted config is that no route is
added for it. Repeating the option appends rather than replaces, so a
long list can be split over several lines.
Names that cannot get a public certificate are given the internal
issuer, the same treatment a site block gives them. Names that already
appear in the automate list are skipped, but a name that also has a site
block is left listed here as well: a site block for http:// only is not
managed by auto-HTTPS, so dropping the name because a block exists could
silently leave it without a certificate.
The option needs no http app at all, so a config consisting only of
global options now adapts to a tls app on its own -- enough to keep
certificates renewed for a mail or XMPP server, or for names served by a
layer 4 app.
Closes#7122
* httpcaddyfile: reuse an existing policy for an automated name
A name given to tls_automate_names may already have an automation policy
from its own site block. Adding a second policy for the same subject is
not just redundant: the adapter rejects overlapping subjects, so
{
email nobody@example.com
tls_automate_names foo.example.com
}
foo.example.com {
tls {
ca https://acme.example.test/directory
}
}
failed to adapt at all, with "hostname appears in more than one
automation policy, making certificate management ambiguous".
Keep such a name in the automate loader but leave its policy alone. The
site block's policy is the more specific of the two, and the loader entry
is still wanted, since a site block served only over HTTP is not managed
by auto-HTTPS.
* Pin that tls_automate_names overrides auto_https off
The names given to the option are managed whether or not auto-HTTPS is
disabled: unlike the hostless-key block above it, that code is not gated
on auto_https, because naming a subject explicitly is a stronger signal
than the general switch. Nothing enforced it, so add the adaptation
fixture @steadytao asked for.
Provision previously only checked for a single opening brace at the
start of a destination, so values like {result}suffix, {result, and
{result}} passed validation. strings.Trim(dest, "{}") then silently
stored a wrong placeholder name (e.g. result}suffix), making the
intended value unavailable.
Require the destination to be exactly one placeholder: a single
opening brace at the start, a single closing brace at the end, and a
non-empty name.
Fixes#8073
The basic auth provider expanded account usernames and passwords with
repl.ReplaceAll, which blanks any {...} the replacer does not recognize
Credentials are config data, so a username like alice-{bogus} was stored
as alice- and a password like ab{cd}ef was quietly rewritten into
different valid base64, turning a config mistake into a working
password the admin never set
Use ReplaceKnown, matching the respond headers fix in #8014 and the
header handler fix in #4880, so unknown braces survive and real
placeholders like {env.*} still expand
* internal: fix MaxSizeSubjectsListForLog off-by-one when maxToDisplay is 0
The loop appended before checking the length, so with maxToDisplay=0
one domain leaked into the output before the break check. Reorder so
the length check comes first.
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* Update internal/logs.go
Co-authored-by: Zen Dodd <mail@steadytao.com>
Signed-off-by: Mohammed Al Sahaf <mohammed@caffeinatedwonders.com>
* fix doc
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
---------
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
Signed-off-by: Mohammed Al Sahaf <mohammed@caffeinatedwonders.com>
Co-authored-by: Zen Dodd <mail@steadytao.com>
The set body is user-supplied and commonly JSON, but it was expanded with
ReplaceAll, which blanks any {...} the replacer does not recognize, so a
plain JSON body was sent empty and a body mixing JSON with a real
placeholder was mangled
Use ReplaceKnown, matching the respond body, so unrecognized braces are
left intact and real placeholders still expand. Regression test pins both
directions
Co-authored-by: hktitof <hktitof@users.noreply.github.com>