no issue
- This fails when running the Ghost container alone (i.e. for unit
tests) since there is no database available.
- The database will be initialized when running `yarn dev` for the first
time anyhow, so this isn't really necessary here.
no issue
- Doing the submodule update in the Dockerfile doesn't work if you cloned Ghost via ssh,
because the Docker container doesn't have access to your ssh keys.
no issue
This commit is a fairly significant rework of the docker development
setup, with the goal of improving the performance of the Ghost container
and generally improving the DX of the docker development setup.
**Enumerating `node_modules` directories as volume exclusions for the
bind mount**
- The main source of performance overhead with the docker setup is the
bind mount, which mounts all the code into the container. Especially on
macOS, the bind mount introduces quite a bit of file system overhead.
Currently the bind mount includes all the `node_modules` directories,
which adds up to many thousands of files in the bind mount. This change
attempts to reduce this overhead by mounting in named volumes for each
`node_modules` directory in the monorepo, which effectively tells the
main bind mount to ignore these directories, thus reducing the
filesystem overhead.
**Removing the `yarn install` from the entrypoint script and the `yarn
setup:docker` step**
- This change has two more benefits: since all the `node_modules` are
installed into named volumes, we can now be confident that our local
`node_modules` won't conflict with the `node_modules` in the container.
Therefore, we can install `node_modules` at build time in the Dockerfile
instead of in the entrypoint script, and we can also get rid of the
`yarn setup:docker` setup command, which nukes your local `node_modules`
before building the container.
**Build optimization - enumerating all `package.json` files**
- In a typical node app's Dockerfile, you copy the `package.json` and
`yarn.lock` files into the container first, then run `yarn install`,
then copy the rest of the code in. This takes advantage of Docker's
build cache to avoid having to reinstall `node_modules` on every build
if you've only changed code files, but not any dependencies.
- This is more challenging to do with the monorepo since we have many
`package.json` files. This commit brute forces our way through that by
copying every single `package.json` file in the Dockerfile before
installing dependencies and copying the code in. The result is that
Docker builds become much faster, after the initial full build.
**Additional scripts**
- Since enumerating all the `node_modules` directories and
`package.json` files is tedious to do manually, this commit also
includes two new scripts to automate this work. Ideally we'll eventually
integrate these scripts with Slimer, so when you add a new package to
the monorepo it automatically updates the `Dockerfile` and
`compose.yml`, but for now we can manage that ourselves since it's a
fairly rare occurrence.
no issue
- When the nx daemon is enabled and using docker for local development,
we periodically get an error like the following, which crashes `yarn
dev` and requires a manual restart:
```
Daemon process terminated and closed the connection
Please rerun the command, which will restart the daemon.
If you get this error again, check for any errors in the daemon process logs found in: /home/ghost/.nx/workspace-data/d/daemon.log
```
- Disabling the daemon in docker prevents these errors, and so far I
haven't noticed any performance degradation from disabling it.
no issue
- Hot reload for admin depends on the browser being able to reach port
4201, which was not exposed in the docker compose setup — this fixes
that so admin will hot reload when running Ghost in docker compose
ref
https://linear.app/ghost/issue/ENG-1959/extend-setupjs-to-modify-config-as-appropriate-for-full-docker-dev
- When switching from local development to docker, there are a few
configuration parameters that need to be updated to e.g. point to the
right database host within the docker network.
- Setting these values with environment variables doesn't work well
because the configuration passed via environment overrides the
configuration set in tests, and thus points tests to the wrong database.
- This commit adds a yarn docker:setup command to the root of the repo,
to make it easier to get started with a full docker compose based
workflow. It edits you config.local.json file to update the necessary
settings for running Ghost in docker compose.
- It also updates the clean.js script such that it will run successfully
regardless of whether it is run locally or in docker.
- Finally, this commit also adds convenience commands for developing and
running tests in docker compose
ref
https://linear.app/ghost/issue/ENG-1957/add-ghost-service-to-base-docker-compose-setup
- Currently our `compose.yml` file only runs Ghost's supporting
services, and it's expected that you'll run Ghost locally on your host
machine. This commit adds a `ghost` service to our `compose.yml` file,
so you can optionally run Ghost itself in a container using docker
compose.
- The `ghost` service is opt-in using [docker compose
profiles](https://docs.docker.com/compose/how-tos/profiles/), to
maintain the current behavior of only running supporting services as the
default.
- This commit also fixes an issue in the Dockerfile: the `WORKDIR` arg,
which is used to optionally specify an alternative working directory, is
not propagated from one build stage to the next, so it has to be
manually added as an `ARG` in each stage. This was necessary to use the
same Dockerfile for devcontainers (which require the WORKDIR to be
`/workspaces/ghost`) and docker compose, where we use `/home/ghost` in
alignment with the production image.
ref
https://linear.app/ghost/issue/ENG-1785/consolidate-dev-container-composeyml-with-the-root-level-composeyml
- We currently have two compose.yml files:
- `.devcontainer/compose.yml`: used for the Dev Container
- `compose.yml`: used for local development with docker compose
- The Dev Container compose file does need some additional
configuration, but most of this code is duplicated from the root level
compose.yml file
- This commit removes this duplication by using both compose files in
the Dev Container setup, so it will base the compose configuration on
the root level, and then extend it with the Dev Container specific
configuration
- This way any updates to the root level compose file will also be
available and reflected in the Dev Container setup without having to
make the change twice.
ref https://linear.app/ghost/issue/ENG-1782/move-docker-related-files-to-a-better-location-that-githubscripts
- The docker files are currently located in `.github/scripts`. This location doesn't make a lot of sense — you wouldn't think to look there unless you already knew they were there. This also requires you to specify the path to the `compose.yml` file whenever running a `docker compose ...` command.
- This commit moves the `compose.yml` file to the root of the repo, so you can simply run `docker compose up` and it will automatically find the file in the root, without having to specify `-f .github/scripts/docker-compose.yml`. This is a major win for convenience over the current setup.
- It also moves all the related files, including the `Dockerfile` used by the Dev Container setup and configuration files for supporting services into a new `.docker` directory, which is a more logical location, and should be easier to find.
- Also updated the current convenience commands in the `package.json` scripts block (`yarn docker:reset` and `yarn docker:down`
