no issue
This commit is a fairly significant rework of the docker development
setup, with the goal of improving the performance of the Ghost container
and generally improving the DX of the docker development setup.
**Enumerating `node_modules` directories as volume exclusions for the
bind mount**
- The main source of performance overhead with the docker setup is the
bind mount, which mounts all the code into the container. Especially on
macOS, the bind mount introduces quite a bit of file system overhead.
Currently the bind mount includes all the `node_modules` directories,
which adds up to many thousands of files in the bind mount. This change
attempts to reduce this overhead by mounting in named volumes for each
`node_modules` directory in the monorepo, which effectively tells the
main bind mount to ignore these directories, thus reducing the
filesystem overhead.
**Removing the `yarn install` from the entrypoint script and the `yarn
setup:docker` step**
- This change has two more benefits: since all the `node_modules` are
installed into named volumes, we can now be confident that our local
`node_modules` won't conflict with the `node_modules` in the container.
Therefore, we can install `node_modules` at build time in the Dockerfile
instead of in the entrypoint script, and we can also get rid of the
`yarn setup:docker` setup command, which nukes your local `node_modules`
before building the container.
**Build optimization - enumerating all `package.json` files**
- In a typical node app's Dockerfile, you copy the `package.json` and
`yarn.lock` files into the container first, then run `yarn install`,
then copy the rest of the code in. This takes advantage of Docker's
build cache to avoid having to reinstall `node_modules` on every build
if you've only changed code files, but not any dependencies.
- This is more challenging to do with the monorepo since we have many
`package.json` files. This commit brute forces our way through that by
copying every single `package.json` file in the Dockerfile before
installing dependencies and copying the code in. The result is that
Docker builds become much faster, after the initial full build.
**Additional scripts**
- Since enumerating all the `node_modules` directories and
`package.json` files is tedious to do manually, this commit also
includes two new scripts to automate this work. Ideally we'll eventually
integrate these scripts with Slimer, so when you add a new package to
the monorepo it automatically updates the `Dockerfile` and
`compose.yml`, but for now we can manage that ourselves since it's a
fairly rare occurrence.
no issue
- I had plans to use this environment variable for the docker compose
setup, but haven't implemented anything with it yet — removing it for
now since it's not doing anything at the moment.
no issue
- The browser tests for scheduling posts were flaking/failing because
the time was set to `00:00`, which is invalid and should reset the time
to "a few seconds from now", but the test was sometimes able to
"Continue" before the validation logic could run on blur. This left the
actual scheduled time set to `00:00`, which then caused the following
assertions to fail.
- I've tried to replicate this race condition manually to confirm it's
not a bug that a user could actually reach, but I haven't been able to.
I suspect that playwright's `.fill()` and/or `.click()` method behave
slightly differently than expected, which creates this race condition.
no issue
- The `yarn dev` command accepts dash-dash-args to conditionally build &
run other apps, like portal, comments, etc.
- Passing these flags to the docker compose setup directly doesn't work,
so the only way to run these apps in docker was to either change the
command in the Dockerfile or override the command in the `compose.yml`
file
- This commit extends `yarn dev` to also check for the
`GHOST_DEV_APP_FLAGS` environment variable for these flags, which can be
forwarded into the container and make it easier to run these apps using
docker compose.
no issue
- Ghost would sometimes crash because it was booting before the database
service was ready (even if the container had already started). This
commit tells `docker compose` to only start the ghost service once the
`mysql` and `redis` services have passed their health checks
no issue
- With the `package.json` commands specifying the `COMPOSE_PROFILES`
environment variable, this `env_file` block is no longer needed.
- Worse yet, it throws an error if the `.env` file it points to does not
exist, which is really annoying.
no issue
- Hot reload for admin depends on the browser being able to reach port
4201, which was not exposed in the docker compose setup — this fixes
that so admin will hot reload when running Ghost in docker compose
ref
https://linear.app/ghost/issue/ENG-1968/get-browser-tests-working-in-docker
- When running browser tests in docker compose, the `stripe listen`
command was not outputting the API key from the environment, because
it's expecting that you've already run `stripe login` to authenticate
with stripe. It only outputs the API key in the command line in CI.
- This isn't convenient/practical to do in docker, and it's much easier
to supply the API key as an environment variable, so this changes the
logic to use the API key from the command line when running in docker
_or_ CI.
ref
https://linear.app/ghost/issue/ENG-1957/add-ghost-service-to-base-docker-compose-setup
- Currently our `compose.yml` file only runs Ghost's supporting
services, and it's expected that you'll run Ghost locally on your host
machine. This commit adds a `ghost` service to our `compose.yml` file,
so you can optionally run Ghost itself in a container using docker
compose.
- The `ghost` service is opt-in using [docker compose
profiles](https://docs.docker.com/compose/how-tos/profiles/), to
maintain the current behavior of only running supporting services as the
default.
- This commit also fixes an issue in the Dockerfile: the `WORKDIR` arg,
which is used to optionally specify an alternative working directory, is
not propagated from one build stage to the next, so it has to be
manually added as an `ARG` in each stage. This was necessary to use the
same Dockerfile for devcontainers (which require the WORKDIR to be
`/workspaces/ghost`) and docker compose, where we use `/home/ghost` in
alignment with the production image.
ref
https://linear.app/ghost/issue/ENG-1956/redis-creates-new-anonymous-volume-each-time-it-boots
- Before this commit, the redis service in our docker compose setup
would create a new anonymous volume each time it boots, with a
non-descriptive, hash-based name. Over time these volumes accumulate and
become a pain to clean up, and it's not immediately obvious what they
are used for.
- This commit adds a persistent data volume for redis data, so it will
reuse the same volume, with a more descriptive title each time it boots.
This eliminates the annoying anonymous volumes, and also gives us data
persistence across boots for the redis service (which can be easily
cleared by deleting the volume).
ref
https://linear.app/ghost/issue/ENG-1785/consolidate-dev-container-composeyml-with-the-root-level-composeyml
- We currently have two compose.yml files:
- `.devcontainer/compose.yml`: used for the Dev Container
- `compose.yml`: used for local development with docker compose
- The Dev Container compose file does need some additional
configuration, but most of this code is duplicated from the root level
compose.yml file
- This commit removes this duplication by using both compose files in
the Dev Container setup, so it will base the compose configuration on
the root level, and then extend it with the Dev Container specific
configuration
- This way any updates to the root level compose file will also be
available and reflected in the Dev Container setup without having to
make the change twice.
ref https://linear.app/ghost/issue/ENG-1782/move-docker-related-files-to-a-better-location-that-githubscripts
- The docker files are currently located in `.github/scripts`. This location doesn't make a lot of sense — you wouldn't think to look there unless you already knew they were there. This also requires you to specify the path to the `compose.yml` file whenever running a `docker compose ...` command.
- This commit moves the `compose.yml` file to the root of the repo, so you can simply run `docker compose up` and it will automatically find the file in the root, without having to specify `-f .github/scripts/docker-compose.yml`. This is a major win for convenience over the current setup.
- It also moves all the related files, including the `Dockerfile` used by the Dev Container setup and configuration files for supporting services into a new `.docker` directory, which is a more logical location, and should be easier to find.
- Also updated the current convenience commands in the `package.json` scripts block (`yarn docker:reset` and `yarn docker:down`
2024-11-19 13:15:06 -08:00
Renamed from .github/scripts/docker-compose.yml (Browse further)
