See discussion on #2015; the initial change had removed this check, and
I can't remember why I removed it or if it was accidental. Anyway, it's
back now.
Also introduce caddy.OnProcessExit which is a list of functions that
run before exiting the process cleanly; these do not count as shutdown
callbacks, so they do not return errors and must execute quickly.
- Expose the list of Caddy instances through caddy.Instances()
- Added arbitrary storage to caddy.Instance
- The cache of loaded certificates is no longer global; now scoped
per-instance, meaning upon reload (like SIGUSR1) the old cert cache
will be discarded entirely, whereas before, aggressively reloading
config that added and removed lots of sites would cause unnecessary
build-up in the cache over time.
- Key certificates in the cache by their SHA-256 hash instead of
by their names. This means certificates will not be duplicated in
memory (within each instance), making Caddy much more memory-efficient
for large-scale deployments with thousands of sites sharing certs.
- Perform name-to-certificate lookups scoped per caddytls.Config instead
of a single global lookup. This prevents certificates from stepping on
each other when they overlap in their names.
- Do not allow TLS configurations keyed by the same hostname to be
different; this now throws an error.
- Updated relevant tests, with a stark awareness that more tests are
needed.
- Change the NewContext function signature to include an *Instance.
- Strongly recommend (basically require) use of caddytls.NewConfig()
to create a new *caddytls.Config, to ensure pointers to the instance
certificate cache are initialized properly.
- Update the TLS-SNI challenge solver (even though TLS-SNI is disabled
currently on the CA side). Store temporary challenge cert in instance
cache, but do so directly by the ACME challenge name, not the hash.
Modified the getCertificate function to check the cache directly for
a name match if one isn't found otherwise. This will allow any
caddytls.Config to be able to help solve a TLS-SNI challenge, with one
extra side-effect that might actually be kind of interesting (and
useless): clients could send a certificate's hash as the SNI and
Caddy would be able to serve that certificate for the handshake.
- Do not attempt to match a "default" (random) certificate when SNI
is present but unrecognized; return no certificate so a TLS alert
happens instead.
- Store an Instance in the list of instances even while the instance
is still starting up (this allows access to the cert cache for
performing renewals at startup, etc). Will be removed from list again
if instance startup fails.
- Laid groundwork for ACMEv2 and Let's Encrypt wildcard support.
Server type plugins will need to be updated slightly to accommodate
minor adjustments to their API (like passing in an Instance). This
commit includes the changes for the HTTP server.
Certain Caddyfile configurations might error out with this change, if
they configured different TLS settings for the same hostname.
This change trades some complexity for other complexity, but ultimately
this new complexity is more correct and robust than earlier logic.
Fixes#1991Fixes#1994Fixes#1303
This adds the ask sub-directive to tls that defines the URL of a backend HTTP service to be queried during the TLS handshake to determine if an on-demand TLS certificate should be acquired for incoming hostnames. When the ask sub-directive is defined, Caddy will query the URL for permission to acquire a cert by making a HTTP GET request to the URL including the requested domain in the query string. If the backend service returns a 2xx response Caddy will acquire a cert. Any other response code (including 3xx redirects) are be considered a rejection and the certificate will not be acquired.
Renewed certificates would not be reloaded into the cache because their
names conflict with names of certificates already in the cache; this
was intentional when loading new certs to avoid confusion, but is
problematic when renewing, since the old certificate doesn't get
evicted from the cache. (Oops.)
Here, I remedy this situation by explicitly deleting the old cert from
the cache before adding the renewed one back in.
https://caddy.community/t/random-ocsp-response-errors-for-random-clients/2473?u=matt
Certificates are keyed by name in the cache, optimized for fast lookups
during TLS handshakes using SNI. A more "correct" way that is truly a
1:1 would be to cache certificates by a hash of the leaf's DER bytes,
but this involves an extra index to maintain. So instead of that, we
simply choose to prevent overlap when keying certificates by server
name. This avoids the ambiguity when updating OCSP staples, for instance.
* introduced own ChallengeProvider type, based on acme.ChallengeProvider to avoid vendoring/version mismatches in Caddy plugins; see Caddy issue #1697
* fixed up comments for ChallengeProvider
* moved ChallengeProvider to caddytls/tls.go
* Disable warning for insecure CA if located on private network.
* Add IsPrivateNetwork function
* Add tests
Signed-off-by: Jonas Östanbäck <jonas.ostanback@gmail.com>
* Add more testcases
Signed-off-by: Jonas Östanbäck <jonas.ostanback@gmail.com>
* Rename IsPrivateNetwork -> IsInternal
Signed-off-by: Jonas Östanbäck <jonas.ostanback@gmail.com>
This could have just as easily been a tls directive property in the
Caddyfile, but I figure if these challenges are being disabled, it's
because of port availability or process privileges, both of which would
affect all sites served by this process. The names of the flag are long
but descriptive.
I've never needed this but I hear of quite a few people who say they
need this ability, so here it is.
This commit removes _almost_ all instances of hard-coded ports 80 and
443 strings, and now allows the user to define what the HTTP and HTTPS
ports are by the -http-port and -https-ports flags.
(One instance of "80" is still hard-coded in tls.go because it cannot
import httpserver to get access to the HTTP port variable. I don't
suspect this will be a problem in practice, but one workaround would be
to define an exported variable in the caddytls package and let the
httpserver package set it as well as its own HTTPPort variable.)
The port numbers required by the ACME challenges HTTP-01 and TLS-SNI-01
are hard-coded into the spec as ports 80 and 443 for good reasons,
but the big question is whether they necessarily need to be the HTTP
and HTTPS ports. Although the answer is probably no, they chose those
ports for convenience and widest compatibility/deployability. So this
commit also assumes that the "HTTP port" is necessarily the same port
on which to serve the HTTP-01 challenge, and the "HTTPS port" is
necessarily the same one on which to serve the TLS-SNI-01 challenge. In
other words, changing the HTTP and HTTPS ports also changes the ports
the challenges will be served on.
If you change the HTTP and HTTPS ports, you are responsible for
configuring your system to forward ports 80 and 443 properly.
Closes#918 and closes#1293. Also related: #468.
* tls: Refactor TLS config innards with a few minor syntax changes
muststaple -> must_staple
"http2 off" -> "alpn" with list of ALPN values
* Fix typo
* Fix QUIC handler
* Inline struct field assignments
The loop which performs renewals in the background obtains a read lock
on the certificate cache map, so that it can be safely iterated. Before
this fix, it would obtain the renewals in the read lock. This has been
fine, except that the TLS-SNI challenge, when invoked after Caddy has
already started, requires adding a certificate to the cache. Doing this
requires an exclusive write lock. But it cannot obtain a write lock
because a read lock is obtained higher in the stack, while the loop
iterates. In other words, it's a deadlock.
I was able to reproduce this issue consistently locally, after jumping
through many hoops to force a renewal in a short time that bypasses
Let's Encrypt's authz caching. I was also able to verify that by queuing
renewals (like we do deletions and OCSP updates), lock contention is
relieved and the deadlock is avoided.
This only affects background renewals where the TLS-SNI(-01) challenge
are used. Users report seeing strange errors in the logs after this
happens ("tls: client offered an unsupported, maximum protocol version
of 301"), but I was not able to reproduce these locally. I was also not
able to reproduce the leak of sockets which are left in CLOSE_WAIT.
I am not sure if those are symptoms of running in production on Linux
and are related to this bug, or not.
Either way, this is an important fix. I do not yet know the ripple
effects this will have on other symptoms we've been chasing. But it
definitely resolves a deadlock during renewals.
By calling SetTLSAddress, the acme package reset the challenge provider
to the default one instead of keeping the custom one we specified before
with SetChallengeProvider. Yikes. This means that Caddy would try to
open a listener on port 443 even though we should have been handling it
with our provider, causing the challenge to fail, since usually port 443
is in use.
So this change just reorders the calls so that our provider takes
precedence.
cf. https://github.com/xenolf/lego/pull/292
