https://caddy.community/t/random-ocsp-response-errors-for-random-clients/2473?u=matt
Certificates are keyed by name in the cache, optimized for fast lookups
during TLS handshakes using SNI. A more "correct" way that is truly a
1:1 would be to cache certificates by a hash of the leaf's DER bytes,
but this involves an extra index to maintain. So instead of that, we
simply choose to prevent overlap when keying certificates by server
name. This avoids the ambiguity when updating OCSP staples, for instance.
* introduced own ChallengeProvider type, based on acme.ChallengeProvider to avoid vendoring/version mismatches in Caddy plugins; see Caddy issue #1697
* fixed up comments for ChallengeProvider
* moved ChallengeProvider to caddytls/tls.go
* Disable warning for insecure CA if located on private network.
* Add IsPrivateNetwork function
* Add tests
Signed-off-by: Jonas Östanbäck <jonas.ostanback@gmail.com>
* Add more testcases
Signed-off-by: Jonas Östanbäck <jonas.ostanback@gmail.com>
* Rename IsPrivateNetwork -> IsInternal
Signed-off-by: Jonas Östanbäck <jonas.ostanback@gmail.com>
This could have just as easily been a tls directive property in the
Caddyfile, but I figure if these challenges are being disabled, it's
because of port availability or process privileges, both of which would
affect all sites served by this process. The names of the flag are long
but descriptive.
I've never needed this but I hear of quite a few people who say they
need this ability, so here it is.
This commit removes _almost_ all instances of hard-coded ports 80 and
443 strings, and now allows the user to define what the HTTP and HTTPS
ports are by the -http-port and -https-ports flags.
(One instance of "80" is still hard-coded in tls.go because it cannot
import httpserver to get access to the HTTP port variable. I don't
suspect this will be a problem in practice, but one workaround would be
to define an exported variable in the caddytls package and let the
httpserver package set it as well as its own HTTPPort variable.)
The port numbers required by the ACME challenges HTTP-01 and TLS-SNI-01
are hard-coded into the spec as ports 80 and 443 for good reasons,
but the big question is whether they necessarily need to be the HTTP
and HTTPS ports. Although the answer is probably no, they chose those
ports for convenience and widest compatibility/deployability. So this
commit also assumes that the "HTTP port" is necessarily the same port
on which to serve the HTTP-01 challenge, and the "HTTPS port" is
necessarily the same one on which to serve the TLS-SNI-01 challenge. In
other words, changing the HTTP and HTTPS ports also changes the ports
the challenges will be served on.
If you change the HTTP and HTTPS ports, you are responsible for
configuring your system to forward ports 80 and 443 properly.
Closes#918 and closes#1293. Also related: #468.
* tls: Refactor TLS config innards with a few minor syntax changes
muststaple -> must_staple
"http2 off" -> "alpn" with list of ALPN values
* Fix typo
* Fix QUIC handler
* Inline struct field assignments
The loop which performs renewals in the background obtains a read lock
on the certificate cache map, so that it can be safely iterated. Before
this fix, it would obtain the renewals in the read lock. This has been
fine, except that the TLS-SNI challenge, when invoked after Caddy has
already started, requires adding a certificate to the cache. Doing this
requires an exclusive write lock. But it cannot obtain a write lock
because a read lock is obtained higher in the stack, while the loop
iterates. In other words, it's a deadlock.
I was able to reproduce this issue consistently locally, after jumping
through many hoops to force a renewal in a short time that bypasses
Let's Encrypt's authz caching. I was also able to verify that by queuing
renewals (like we do deletions and OCSP updates), lock contention is
relieved and the deadlock is avoided.
This only affects background renewals where the TLS-SNI(-01) challenge
are used. Users report seeing strange errors in the logs after this
happens ("tls: client offered an unsupported, maximum protocol version
of 301"), but I was not able to reproduce these locally. I was also not
able to reproduce the leak of sockets which are left in CLOSE_WAIT.
I am not sure if those are symptoms of running in production on Linux
and are related to this bug, or not.
Either way, this is an important fix. I do not yet know the ripple
effects this will have on other symptoms we've been chasing. But it
definitely resolves a deadlock during renewals.
By calling SetTLSAddress, the acme package reset the challenge provider
to the default one instead of keeping the custom one we specified before
with SetChallengeProvider. Yikes. This means that Caddy would try to
open a listener on port 443 even though we should have been handling it
with our provider, causing the challenge to fail, since usually port 443
is in use.
So this change just reorders the calls so that our provider takes
precedence.
cf. https://github.com/xenolf/lego/pull/292
We renamed caddytls.ErrStorageNotFound to caddytls.ErrNotExist to more
closely mirror the os package. We changed it to an interface wrapper
so that the custom error message can be preserved. Returning only "data
not found" was useless in debugging because we couldn't know the
concrete value of the error (like what it was trying to load).
Users can do a type assertion to determine if the error value is a "not
found" error instead of doing an equality check.
A Caddyfile using *.example.com as its site address would be subject to
this bug at renewal time, as it would use the literal "*.example.com"
value instead of the name being passed in to obtain a certificate.
This change fixes the LoadSite call so that it looks in the proper
directory for the certificate resources.
It was set by default on the caddy-internal config object, and even
checked for conflicts, but it was never actually reflected on the
tls.Config.
This will have user-visible changes: a client that prefers, say, AES-CBC
but also supports AES-GCM would have used AES-CBC befor this, and will
use AES-GCM after.
This is desirable and important behavior, because if for example the
server wanted to support 3DES, but *only if it was strictly necessary*,
it would have had no way of doing so with PreferServerCipherSuites
false, as the client preference would have won.
If another ACME client is trying to solve a challenge for a name not
being served by Caddy on the same machine where Caddy is running, the
HTTP challenge will be consumed by Caddy rather than allowing the owner
to use the Caddyfile to proxy the challenge.
With this change, we only consume requests for HTTP challenges for
hostnames that we recognize. Before doing the challenge, we add the
name to a set, and when seeing if we should proxy the challenge, we
first check the path of course to see if it is an HTTP challenge;
if it is, we then check that set to see if the hostname is in the
set. Only if it is, do we consume it.
Otherwise, the request is treated like any other, allowing the owner
to configure a proxy for such requests to another ACME client.
* Initial concept for pluggable storage (sans tests and docs)
* Add TLS storage docs, test harness, and minor clean up from code review
* Fix issue with caddymain's temporary moveStorage
* Formatting improvement on struct array literal by removing struct name
* Pluggable storage changes:
* Change storage interface to persist all site or user data in one call
* Add lock/unlock calls for renewal and cert obtaining
* Key fields on composite literals
- Server types no longer need to store their own contexts; they are
stored on the caddy.Instance, which means each context will be
properly GC'ed when the instance is stopped. Server types should use
type assertions to convert from caddy.Context to their concrete
context type when they need to use it.
- Pass the entire context into httpserver.GetConfig instead of only the
Key field.
- caddy.NewTestController now requires a server type string so it can
create a controller with the proper concrete context associated with
that server type.
Tests still need more attention so that we can test the proper creation
of startup functions, etc.
Also we change the scheme of the site's address if TLS is enabled and
no other scheme is explicitly set; this makes it appear as "https" when
we print it; otherwise it would show "http" when TLS is turned on
implicitly, and that is confusing/incorrect.
These changes span work from the last ~4 months in an effort to make
Caddy more extensible, reduce the coupling between its components, and
lay a more robust foundation of code going forward into 1.0. A bunch of
new features have been added, too, with even higher future potential.
The most significant design change is an overall inversion of
dependencies. Instead of the caddy package knowing about the server
and the notion of middleware and config, the caddy package exposes an
interface that other components plug into. This does introduce more
indirection when reading the code, but every piece is very modular and
pluggable. Even the HTTP server is pluggable.
The caddy package has been moved to the top level, and main has been
pushed into a subfolder called caddy. The actual logic of the main
file has been pushed even further into caddy/caddymain/run.go so that
custom builds of Caddy can be 'go get'able.
The HTTPS logic was surgically separated into two parts to divide the
TLS-specific code and the HTTPS-specific code. The caddytls package can
now be used by any type of server that needs TLS, not just HTTP. I also
added the ability to customize nearly every aspect of TLS at the site
level rather than all sites sharing the same TLS configuration. Not all
of this flexibility is exposed in the Caddyfile yet, but it may be in
the future. Caddy can also generate self-signed certificates in memory
for the convenience of a developer working on localhost who wants HTTPS.
And Caddy now supports the DNS challenge, assuming at least one DNS
provider is plugged in.
Dozens, if not hundreds, of other minor changes swept through the code
base as I literally started from an empty main function, copying over
functions or files as needed, then adjusting them to fit in the new
design. Most tests have been restored and adapted to the new API,
but more work is needed there.
A lot of what was "impossible" before is now possible, or can be made
possible with minimal disruption of the code. For example, it's fairly
easy to make plugins hook into another part of the code via callbacks.
Plugins can do more than just be directives; we now have plugins that
customize how the Caddyfile is loaded (useful when you need to get your
configuration from a remote store).
Site addresses no longer need be just a host and port. They can have a
path, allowing you to scope a configuration to a specific path. There is
no inheretance, however; each site configuration is distinct.
Thanks to amazing work by Lucas Clemente, this commit adds experimental
QUIC support. Turn it on using the -quic flag; your browser may have
to be configured to enable it.
Almost everything is here, but you will notice that most of the middle-
ware are missing. After those are transferred over, we'll be ready for
beta tests.
I'm very excited to get this out. Thanks for everyone's help and
patience these last few months. I hope you like it!!
