Guh, this is complicated.
Fixes#4640
This also follows up on #4398 (reverting it) which made a change that technically worked, but was incorrect. It changed the condition in `hostsFromKeysNotHTTP` from `&&` to `||`, but then the function no longer did what its name said it would do, and it would return hosts even if they were marked with `http://`, if they used a non-HTTP port. That wasn't the intent of it. The test added in there was kept though, because it is a valid usecase.
The actual fix is to check _earlier_ whether all the addresses explicitly have `http://`, and if so we can short circuit and skip considering the rest.
Huge thank-you to Tailscale (https://tailscale.com) for making this change possible!
This is a great feature for Caddy and Tailscale is a great fit for a standard implementation.
* caddytls: GetCertificate modules; Tailscale
* Caddyfile support for get_certificate
Also fix AP provisioning in case of empty subject list (persist loaded
module on struct, much like Issuers, to surive reprovisioning).
And implement start of HTTP cert getter, still WIP.
* Update modules/caddytls/automation.go
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Use tsclient package, check status for name
* Implement HTTP cert getter
And use reuse CertMagic's PEM functions for private keys.
* Remove cache option from Tailscale getter
Tailscale does its own caching and we don't need the added complexity...
for now, at least.
* Several updates
- Option to disable cert automation in auto HTTPS
- Support multiple cert managers
- Remove cache feature from cert manager modules
- Minor improvements to auto HTTPS logging
* Run go mod tidy
* Try to get certificates from Tailscale implicitly
Only for domains ending in .ts.net.
I think this is really cool!
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
If an email is specified in global options, a site called 'localhost' shouldn't be bunched together with public DNS names in the automation policies, which get the default, public-CA issuers. Fix old test that did this.
I also noticed that these two:
localhost {
}
example.com {
}
and
localhost, example.com {
}
produce slightly different TLS automation policies. The former is what the new test case covers, and we have logic that removes the empty automation policy for localhost so that auto-HTTPS can implicitly create one. (We prefer that whenever possible.) But the latter case produces two automation policies, with the second one being for localhost, with an explicit internal issuer. It's not wrong, just more explicit than it needs to be.
I'd really like to completely rewrite the code from scratch that generates automation policies, hopefully there is a simpler, more correct algorithm.
In the Caddyfile, hosts specified for HTTP sockets (either scheme is "http" or it is on the HTTP port) should not be used as subjects in TLS automation policies (APs).
* httpcaddyfile: Fix unexpectedly removed policy
When user set on_demand tls option in a catch-all (:443) policy,
we expect other policies to not have the on_demand enabled
See ex in tls_automation_policies_5.txt
Btw, we can remove policies if they are **all** empty.
* Update caddyconfig/httpcaddyfile/tlsapp.go
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
If `tls <email>` is used, we should apply that to all applicable default issuers, not drop them. This refactoring applies implicit ACME issuer settings from the tls directive to all default ACME issuers, like ZeroSSL.
We also consolidate some annoying logic and improve config validity checks.
Ref: https://caddy.community/t/error-obtaining-certificate-after-caddy-restart/11335/8
This is probably an invasive change, but existing tests continue to pass.
It seems to make sense this way. There is likely an edge case I haven't
considered.
Allows user to disable OCSP stapling (including support in the Caddyfile via the ocsp_stapling global option) or overriding responder URLs. Useful in environments where responders are not reachable due to firewalls.
This changes the signature of UnmarshalGlobalFunc but this is probably OK since it's only used by this repo as far as we know.
We need this change in order to "remember" the previous value in case a global option appears more than once, which is now a possibility with the cert_issuer option since Caddy now supports multiple issuers in the order defined by the user.
Bonus: the issuer subdirective of tls now supports one-liner for "acme" when all you need to set is the directory:
issuer acme <dir>
* caddytls: Support multiple issuers
Defaults are Let's Encrypt and ZeroSSL.
There are probably bugs.
* Commit updated integration tests, d'oh
* Update go.mod
* httpcaddyfile: Revise automation policy generation
This should fix a frustrating edge case where wildcard subjects are
used, which potentially get shadowed by more specific versions of
themselves; see the new tests for an example. This change is motivated
by an actual customer requirement.
Although all the tests pass, this logic is incredibly complex and
nuanced, and I'm worried it is not correct. But it took me about 4 days
to get this far on a solution. I did my best.
* Fix typo
We have users that have site blocks like *.*.tld with on-demand TLS
enabled. While *.*.tld does not qualify for a publicly-trusted cert due
to its wildcards, On-Demand TLS does not actually obtain a cert with
those wildcards, since it uses the actual hostname on the handshake.
This improves on that logic, but I am still not 100% satisfied with the
result since I think we need to also check if another site block is more
specific, like foo.example.tld, which might not have on-demand TLS
enabled, and make sure an automation policy gets created before the
more general policy with on-demand...
* caddytls: Add support for ZeroSSL; add Caddyfile support for issuers
Configuring issuers explicitly in a Caddyfile is not easily compatible
with existing ACME-specific parameters such as email or acme_ca which
infer the kind of issuer it creates (this is complicated now because
the ZeroSSL issuer wraps the ACME issuer)... oh well, we can revisit
that later if we need to.
New Caddyfile global option:
{
cert_issuer <name> ...
}
Or, alternatively, as a tls subdirective:
tls {
issuer <name> ...
}
For example, to use ZeroSSL with an API key:
{
cert_issuser zerossl API_KEY
}
For now, that still uses ZeroSSL's ACME endpoint; it fetches EAB
credentials for you. You can also provide the EAB credentials directly
just like any other ACME endpoint:
{
cert_issuer acme {
eab KEY_ID MAC_KEY
}
}
All these examples use the new global option (or tls subdirective). You
can still use traditional/existing options with ZeroSSL, since it's
just another ACME endpoint:
{
acme_ca https://acme.zerossl.com/v2/DV90
acme_eab KEY_ID MAC_KEY
}
That's all there is to it. You just can't mix-and-match acme_* options
with cert_issuer, because it becomes confusing/ambiguous/complicated to
merge the settings.
* Fix broken test
This test was asserting buggy behavior, oops - glad this branch both
discovers and fixes the bug at the same time!
* Fix broken test (post-merge)
* Update modules/caddytls/acmeissuer.go
Fix godoc comment
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Add support for ZeroSSL's EAB-by-email endpoint
Also transform the ACMEIssuer into ZeroSSLIssuer implicitly if set to
the ZeroSSL endpoint without EAB (the ZeroSSLIssuer is needed to
generate EAB if not already provided); this is now possible with either
an API key or an email address.
* go.mod: Use latest certmagic, acmez, and x/net
* Wrap underlying logic rather than repeating it
Oops, duh
* Form-encode email info into request body for EAB endpoint
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Adds global options for external account bindings
* Maybe other people use ctags too?
* Use nested block to configure external account
* go format files
* Restore acme_ca directive in test file
* Change Caddyfile config syntax for acme_eab
* Update test
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Panic would happen if an automation policy was specified in a singular
server block that had no hostnames in its address. Definitely an edge
case.
Fixed a bug related to checking for server blocks with a host-less key
that tried to make an automation policy. Previously if you had only two
server blocks like ":443" and another one at ":80", the one at ":443"
could not create a TLS automation policy because it thought it would
interfere with TLS automation for the block at ":80", but obviously that
key doesn't enable TLS because it is on the HTTP port. So now we are a
little smarter and count only non-HTTP-empty-hostname keys.
Also fixed a bug so that a key like "https://:1234" is sure to have TLS
enabled by giving it a TLS connection policy. (Relaxed conditions
slightly; the previous conditions were too strict, requiring there to be
a TLS conn policy already or a default SNI to be non-empty.)
Also clarified a comment thanks to feedback from @Mohammed90
- Create two default automation policies; if the TLS app is used in
isolation with the 'automate' certificate loader, it will now use
an internal issuer for internal-only names, and an ACME issuer for
all other names by default.
- If the HTTP Caddyfile adds an 'automate' loader, it now also adds an
automation policy for any names in that loader that do not qualify
for public certificates so that they will be issued internally. (It
might be nice if this wasn't necessary, but the alternative is to
either make auto-HTTPS logic way more complex by scanning the names in
the 'automate' loader, or to have an automation policy without an
issuer switch between default issuer based on the name being issued
a certificate - I think I like the latter option better, right now we
do something kind of like that but at a level above each individual
automation policies, we do that switch only when no automation
policies match, rather than when a policy without an issuer does
match.)
- Set the default LoggerName rather than a LoggerNames with an empty
host value, which is now taken literally rather than as a catch-all.
- hostsFromKeys, the function that gets a list of hosts from server
block keys, no longer returns an empty string in its resulting slice,
ever.
We now store the parsed site/server block keys with the server block,
rather than parsing the addresses every time we read them.
Also detect conflicting schemes, i.e. TLS and non-TLS cannot be served
from the same server (natively -- modules could be built for it).
Also do not add site subroutes (subroutes generated specifically from
site blocks in the Caddyfile) that are empty.
If a site block has a key like "http://localhost:2016", then the log for
that site must be mapped to "localhost:2016" and not just "localhost"
because "localhost:2016" will be the value of the Host header of requests.
But a key like "localhost:80" does not include the port since the Host
header will not include ":80" because it is a standard port.
Fixes https://caddy.community/t/v2-common-log-format-not-working/7352?u=matt
