Now attempt to staple OCSP even for certs that don't have an existing staple (issue #605). "tls off" short-circuits tls setup function. Now we call getEmail() when setting up an acme.Client that does renewals, rather than making a new account with empty email address. Check certificate expiry every 12 hours, and OCSP every hour.
This fixes a regression introduced in recent commits that enabled TLS on the default ":2015" config. This fix is possible because On-Demand TLS is no longer implicit; it must be explicitly enabled by the user by setting a maximum number of certificates to issue.
Biggest change is no longer using standard library's tls.Config.getCertificate function to get a certificate during TLS handshake. Implemented our own cache which can be changed dynamically at runtime, even during TLS handshakes. As such, restarts are no longer required after certificate renewals or OCSP updates.
We also allow loading multiple certificates and keys per host, even by specifying a directory (tls got a new 'load' command for that).
Renamed the letsencrypt package to https in a gradual effort to become more generic; and https is more fitting for what the package does now.
There are still some known bugs, e.g. reloading where a new certificate is required but port 80 isn't currently listening, will cause the challenge to fail. There's still plenty of cleanup to do and tests to write. It is especially confusing right now how we enable "on-demand" TLS during setup and keep track of that. But this change should basically work so far.
Implements "on-demand TLS" as I call it, which means obtaining TLS certificates on-the-fly during TLS handshakes if a certificate for the requested hostname is not already available. Only the first request for a new hostname will experience higher latency; subsequent requests will get the new certificates right out of memory.
Code still needs lots of cleanup but the feature is basically working.
I've built this on Go 1.6 beta 1 and made some changes to be more compatible. Namely, I removed the use of the /x/net/http2 package and let net/http enable h2 by default; updated the way h2 is disabled (if the user requires it); moved TLS_FALLBACK_SCSV to the front of the cipher suites list (all values not accepted by http2 must go after those allowed by it); removed the NextProto default of http/1.1; set the http.Server.TLSConfig value to the TLS config used by the listener (we left it nil before, but this prevents automatic enabling of h2).
It is very likely there is more to do, but at least already Caddy uses HTTP/2 when built with Go 1.6.
Before, Caddy couldn't support graceful (zero-downtime) restarts when the reloaded Caddyfile had a host in it that was elligible for a LE certificate because the port was already in use. This commit makes it possible to do zero-downtime reloads and issue certificates for new hosts that need it. Supports only http-01 challenge at this time.
OCSP stapling is improved in that it updates before the expiration time when the validity window has shifted forward. See 30c949085c. Before it only used to update when the status changed.
This commit also sets the user agent for Let's Encrypt requests with a string containing "Caddy".
By separating scheme and port at the parser, we are able to set the port appropriately and also keep the semantics of the scheme being specified by the user later on. The parser also stores an address' original input. Also, the config refactor makes it possible to partially load a config - valuable for determining which ones will need Let's Encrypt integration turned on during a restart.
Added a -grace flag to customize graceful shutdown period, fixed bugs related to closing file descriptors (and dup'ed fds), improved healthcheck signaling to parent, fixed a race condition with the graceful listener, etc. These improvements mainly provide better support for frequent reloading or unusual use cases of Start and Stop after a Restart (POSIX systems). This forum thread was valuable help in debugging: https://forum.golangbridge.org/t/bind-address-already-in-use-even-after-listener-closed/1510?u=matt
Fixed pidfile writing problem where a pidfile would be written even if child failed, also cleaned up restarts a bit and fixed a few bugs, it's more robust now in case of failures and with logging.
We had to hack some special support into the server and caddy packages for this. There are some middlewares which should only execute commands when the original parent process first starts up. For example, someone using the startup directive to start a backend service would not expect the command to be executed every time the config was reloaded or changed - only once when they first started the original caddy process.
This commit adds FirstStartup to the virtualhost config
Previously, if a listener fails to bind (for example), there was a race in caddy.go between unblocking the startup waitgroup and returning the error and putting it into errChan. Now, an error is returned directly into errChan and the closing of the startup waitgroup is defered until after that return takes place.
In testing, I've found that Windows doesn't release the socket right away even though the listener is closed, so calling caddy.Start() right after caddy.Stop() can fail. This change has server.ListenAndServe() try up to 20 times every 100ms to bind the listener, and only return an error if it doesn't succeed after 2 seconds. This might be kind of nifty for Unix, too, but there hasn't been a need for it yet.
Merged config and app packages into one called caddy. Abstracted away caddy startup functionality making it easier to embed Caddy in any Go application and use it as a library. Graceful restart (should) now ensure child starts properly. Now piping a gob bundle to child process so that the child can match up inherited listeners to server address. Much cleanup still to do.
Lots of refinement still needed and runs only on POSIX systems. Windows will not get true graceful restarts (for now), but we will opt for very, very quick forceful restarts. Also, server configs are no longer put into a map; it is critical that they stay ordered so that they can be matched with their sockets in the child process after forking.
This implementation of graceful restarts is probably not perfect, but it is a good start. Lots of details to attend to now.
Turns out having each server block share a single server.Config during initialization when the Setup functions are being called was a bad idea. Sure, startup and shutdown functions were only executed once, but they had no idea what their hostname or port was. So here we revert to the old way of doing things where Setup may be called multiple times per server block (once per host associated with the block, to be precise), but the Setup functions now know their host and port since the config belongs to exactly one virtualHost. To have something happen just once per server block, use OncePerServerBlock, a new function available on each Controller.
Even if defined for multiple hosts. Startup or shutdown callbacks registered by any directive (startup, shutdown, markdown, git, log, etc.) will only run as many times as it appears in the Caddyfile, not repeated for each host that shares that server block. Fixing this involved refactoring three packages (yeesh) and we need to restore some tests that are no longer valid (that used to verify splitting a multiServerBlock into multiple serverBlocks).
Addresses which fail to resolve are handled more gracefully in the two most common cases: the hostname doesn't resolve or the port is unknown (like "http" on a system that doesn't support that port name). If the hostname doesn't resolve, the host is served on the listener at host 0.0.0.0. If the port is unknown, we attempt to rewrite it as a number manually and try again.
