Infrastructure/ghost
0
Fork 0
mirror of https://github.com/TryGhost/Ghost.git synced 2025-03-04 02:01:58 -05:00
Code Issues Activity
ghost/compose.yml

302 lines
16 KiB
YAML
Raw Normal View History

Updated docker-compose.yml to include a name for the project (#19588) no refs Added a name for the project in `docker-compose.yml` as it was using the `scripts` directory as the name Before: ```sh ❯ docker-compose ls NAME STATUS CONFIG FILES scripts running(1) /path/to/ghost/.github/scripts/docker-compose.yml ``` After: ```sh ❯ docker-compose ls NAME STATUS CONFIG FILES ghost running(1) /path/to/ghost/.github/scripts/docker-compose.yml ```
2024-02-01 09:06:44 +00:00
name: ghost
Added MySQL setup via Docker refs https://github.com/TryGhost/DevOps/issues/118 - we should standardize how we run MySQL, so there aren't massive gaps in developer experience - Docker is great for this, and it's pretty easy to set up - this adds a docker-compose.yml file with a small MySQL setup - also configures `yarn setup` to spin up the Docker container and add it to config.local.json - in the near future, we'll provide a base SQL file in .github/scripts/mysql-preload to ensure the DB is in a known state
2023-12-06 20:11:52 +01:00
services:
Added opt-in `ghost` service to docker compose (#21938) ref https://linear.app/ghost/issue/ENG-1957/add-ghost-service-to-base-docker-compose-setup - Currently our `compose.yml` file only runs Ghost's supporting services, and it's expected that you'll run Ghost locally on your host machine. This commit adds a `ghost` service to our `compose.yml` file, so you can optionally run Ghost itself in a container using docker compose. - The `ghost` service is opt-in using [docker compose profiles](https://docs.docker.com/compose/how-tos/profiles/), to maintain the current behavior of only running supporting services as the default. - This commit also fixes an issue in the Dockerfile: the `WORKDIR` arg, which is used to optionally specify an alternative working directory, is not propagated from one build stage to the next, so it has to be manually added as an `ARG` in each stage. This was necessary to use the same Dockerfile for devcontainers (which require the WORKDIR to be `/workspaces/ghost`) and docker compose, where we use `/home/ghost` in alignment with the production image.
2025-01-07 14:16:43 -08:00
ghost:
build:
context: .
dockerfile: ./.docker/Dockerfile
target: development
Moved @tryghost/update-check-service to services folder fix https://linear.app/ghost/issue/ENG-2039/move-tryghostupdate-check-service-to-services-folder - this service is already self-contained and can be moved into the services folder, which means we can also remove the package that was supporting it - minor other changes just to fix paths
2025-02-26 16:48:03 +01:00
entrypoint: [ "/home/ghost/.docker/development.entrypoint.sh" ]
command: [ "yarn", "dev" ]
Added opt-in `ghost` service to docker compose (#21938) ref https://linear.app/ghost/issue/ENG-1957/add-ghost-service-to-base-docker-compose-setup - Currently our `compose.yml` file only runs Ghost's supporting services, and it's expected that you'll run Ghost locally on your host machine. This commit adds a `ghost` service to our `compose.yml` file, so you can optionally run Ghost itself in a container using docker compose. - The `ghost` service is opt-in using [docker compose profiles](https://docs.docker.com/compose/how-tos/profiles/), to maintain the current behavior of only running supporting services as the default. - This commit also fixes an issue in the Dockerfile: the `WORKDIR` arg, which is used to optionally specify an alternative working directory, is not propagated from one build stage to the next, so it has to be manually added as an `ARG` in each stage. This was necessary to use the same Dockerfile for devcontainers (which require the WORKDIR to be `/workspaces/ghost`) and docker compose, where we use `/home/ghost` in alignment with the production image.
2025-01-07 14:16:43 -08:00
ports:
Added support for portal & other apps in containerized Ghost (#22121) no issue - The `yarn dev` command accepts dash-dash-args to conditionally build & run other apps, like portal, comments, etc. - Passing these flags to the docker compose setup directly doesn't work, so the only way to run these apps in docker was to either change the command in the Dockerfile or override the command in the `compose.yml` file - This commit extends `yarn dev` to also check for the `GHOST_DEV_APP_FLAGS` environment variable for these flags, which can be forwarded into the container and make it easier to run these apps using docker compose.
2025-02-05 18:18:58 -08:00
- "2368:2368" # Ghost
- "4200:4200" # Admin
- "4201:4201" # Admin tests
- "4175:4175" # Portal
- "4176:4176" # Portal HTTPS
- "4177:4177" # Announcement bar
- "4178:4178" # Search
- "6174:6174" # Signup form
- "7173:7173" # Comments
- "7174:7174" # Comments HTTPS
Rename compose profile to ghost (#22223) no issue - Renamed the docker compose profile for Ghost from `full` to `ghost`, because it is clearer and makes more sense this way - Also added an explicit `GHOST_DEV_IS_DOCKER` environment variable to the `compose.yml` file to make checking if running in docker less fragile and more explicit.
2025-02-18 16:45:18 -08:00
profiles: [ ghost ]
Added opt-in `ghost` service to docker compose (#21938) ref https://linear.app/ghost/issue/ENG-1957/add-ghost-service-to-base-docker-compose-setup - Currently our `compose.yml` file only runs Ghost's supporting services, and it's expected that you'll run Ghost locally on your host machine. This commit adds a `ghost` service to our `compose.yml` file, so you can optionally run Ghost itself in a container using docker compose. - The `ghost` service is opt-in using [docker compose profiles](https://docs.docker.com/compose/how-tos/profiles/), to maintain the current behavior of only running supporting services as the default. - This commit also fixes an issue in the Dockerfile: the `WORKDIR` arg, which is used to optionally specify an alternative working directory, is not propagated from one build stage to the next, so it has to be manually added as an `ARG` in each stage. This was necessary to use the same Dockerfile for devcontainers (which require the WORKDIR to be `/workspaces/ghost`) and docker compose, where we use `/home/ghost` in alignment with the production image.
2025-01-07 14:16:43 -08:00
volumes:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
# Mount the source code
Added opt-in `ghost` service to docker compose (#21938) ref https://linear.app/ghost/issue/ENG-1957/add-ghost-service-to-base-docker-compose-setup - Currently our `compose.yml` file only runs Ghost's supporting services, and it's expected that you'll run Ghost locally on your host machine. This commit adds a `ghost` service to our `compose.yml` file, so you can optionally run Ghost itself in a container using docker compose. - The `ghost` service is opt-in using [docker compose profiles](https://docs.docker.com/compose/how-tos/profiles/), to maintain the current behavior of only running supporting services as the default. - This commit also fixes an issue in the Dockerfile: the `WORKDIR` arg, which is used to optionally specify an alternative working directory, is not propagated from one build stage to the next, so it has to be manually added as an `ARG` in each stage. This was necessary to use the same Dockerfile for devcontainers (which require the WORKDIR to be `/workspaces/ghost`) and docker compose, where we use `/home/ghost` in alignment with the production image.
2025-01-07 14:16:43 -08:00
- .:/home/ghost
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
Configured ssh agent forwarding for docker compose (#22232) no issue - Currently running any `git` commands in the docker container (i.e. updating submodules) that use ssh are failing because the container can't see your ssh keys. This commit adds a volume & environment variable to enable forwarding your local ssh agent into the container. As long as you've got an ssh agent running locally with your keys enabled, this will allow you to e.g. push to a remote over ssh from inside the container. - It also mounts your local `.gitconfig` file into the container, so your git configuration (i.e. name & email address) will also work inside the container - Finally, it adds githubs ssh keys to known_hosts in the development target of the Dockerfile to avoid the prompt, which only works in interactive environments and fails in any kind of script (like `yarn main:submodules`).
2025-02-19 09:38:05 -08:00
## SSH Agent forwarding
- ${SSH_AUTH_SOCK}:/ssh-agent
## Git config
- ${HOME}/.gitconfig:/root/.gitconfig:ro
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
# Volume exclusions:
## Prevent collisions between host and container node_modules
- node_modules_ghost_root:/home/ghost/node_modules:delegated
- node_modules_ghost_activitypub:/home/ghost/ghost/activitypub/node_modules:delegated
- node_modules_ghost_adapter-cache-memory-ttl:/home/ghost/ghost/adapter-cache-memory-ttl/node_modules:delegated
- node_modules_ghost_adapter-cache-redis:/home/ghost/ghost/adapter-cache-redis/node_modules:delegated
- node_modules_ghost_adapter-manager:/home/ghost/ghost/adapter-manager/node_modules:delegated
- node_modules_ghost_admin:/home/ghost/ghost/admin/node_modules:delegated
- node_modules_ghost_announcement-bar-settings:/home/ghost/ghost/announcement-bar-settings/node_modules:delegated
- node_modules_ghost_api-framework:/home/ghost/ghost/api-framework/node_modules:delegated
- node_modules_ghost_api-version-compatibility-service:/home/ghost/ghost/api-version-compatibility-service/node_modules:delegated
- node_modules_ghost_audience-feedback:/home/ghost/ghost/audience-feedback/node_modules:delegated
- node_modules_ghost_bookshelf-repository:/home/ghost/ghost/bookshelf-repository/node_modules:delegated
- node_modules_ghost_bootstrap-socket:/home/ghost/ghost/bootstrap-socket/node_modules:delegated
- node_modules_ghost_captcha-service:/home/ghost/ghost/captcha-service/node_modules:delegated
- node_modules_ghost_constants:/home/ghost/ghost/constants/node_modules:delegated
- node_modules_ghost_core:/home/ghost/ghost/core/node_modules:delegated
- node_modules_ghost_custom-fonts:/home/ghost/ghost/custom-fonts/node_modules:delegated
- node_modules_ghost_custom-theme-settings-service:/home/ghost/ghost/custom-theme-settings-service/node_modules:delegated
- node_modules_ghost_data-generator:/home/ghost/ghost/data-generator/node_modules:delegated
- node_modules_ghost_domain-events:/home/ghost/ghost/domain-events/node_modules:delegated
- node_modules_ghost_donations:/home/ghost/ghost/donations/node_modules:delegated
- node_modules_ghost_dynamic-routing-events:/home/ghost/ghost/dynamic-routing-events/node_modules:delegated
- node_modules_ghost_email-addresses:/home/ghost/ghost/email-addresses/node_modules:delegated
- node_modules_ghost_email-analytics-provider-mailgun:/home/ghost/ghost/email-analytics-provider-mailgun/node_modules:delegated
- node_modules_ghost_email-analytics-service:/home/ghost/ghost/email-analytics-service/node_modules:delegated
- node_modules_ghost_email-content-generator:/home/ghost/ghost/email-content-generator/node_modules:delegated
- node_modules_ghost_email-events:/home/ghost/ghost/email-events/node_modules:delegated
- node_modules_ghost_email-service:/home/ghost/ghost/email-service/node_modules:delegated
- node_modules_ghost_email-suppression-list:/home/ghost/ghost/email-suppression-list/node_modules:delegated
- node_modules_ghost_express-dynamic-redirects:/home/ghost/ghost/express-dynamic-redirects/node_modules:delegated
- node_modules_ghost_external-media-inliner:/home/ghost/ghost/external-media-inliner/node_modules:delegated
- node_modules_ghost_ghost:/home/ghost/ghost/ghost/node_modules:delegated
- node_modules_ghost_html-to-plaintext:/home/ghost/ghost/html-to-plaintext/node_modules:delegated
- node_modules_ghost_i18n:/home/ghost/ghost/i18n/node_modules:delegated
- node_modules_ghost_identity-token-service:/home/ghost/ghost/identity-token-service/node_modules:delegated
- node_modules_ghost_importer-handler-content-files:/home/ghost/ghost/importer-handler-content-files/node_modules:delegated
- node_modules_ghost_importer-revue:/home/ghost/ghost/importer-revue/node_modules:delegated
- node_modules_ghost_in-memory-repository:/home/ghost/ghost/in-memory-repository/node_modules:delegated
- node_modules_ghost_job-manager:/home/ghost/ghost/job-manager/node_modules:delegated
- node_modules_ghost_link-redirects:/home/ghost/ghost/link-redirects/node_modules:delegated
- node_modules_ghost_link-replacer:/home/ghost/ghost/link-replacer/node_modules:delegated
- node_modules_ghost_magic-link:/home/ghost/ghost/magic-link/node_modules:delegated
- node_modules_ghost_mail-events:/home/ghost/ghost/mail-events/node_modules:delegated
- node_modules_ghost_mailgun-client:/home/ghost/ghost/mailgun-client/node_modules:delegated
- node_modules_ghost_member-attribution:/home/ghost/ghost/member-attribution/node_modules:delegated
- node_modules_ghost_member-events:/home/ghost/ghost/member-events/node_modules:delegated
- node_modules_ghost_members-api:/home/ghost/ghost/members-api/node_modules:delegated
- node_modules_ghost_members-csv:/home/ghost/ghost/members-csv/node_modules:delegated
- node_modules_ghost_members-events-service:/home/ghost/ghost/members-events-service/node_modules:delegated
- node_modules_ghost_members-importer:/home/ghost/ghost/members-importer/node_modules:delegated
- node_modules_ghost_members-ssr:/home/ghost/ghost/members-ssr/node_modules:delegated
- node_modules_ghost_mentions-email-report:/home/ghost/ghost/mentions-email-report/node_modules:delegated
- node_modules_ghost_metrics-server:/home/ghost/ghost/metrics-server/node_modules:delegated
- node_modules_ghost_milestones:/home/ghost/ghost/milestones/node_modules:delegated
- node_modules_ghost_minifier:/home/ghost/ghost/minifier/node_modules:delegated
- node_modules_ghost_model-to-domain-event-interceptor:/home/ghost/ghost/model-to-domain-event-interceptor/node_modules:delegated
- node_modules_ghost_mw-api-version-mismatch:/home/ghost/ghost/mw-api-version-mismatch/node_modules:delegated
- node_modules_ghost_mw-cache-control:/home/ghost/ghost/mw-cache-control/node_modules:delegated
- node_modules_ghost_mw-error-handler:/home/ghost/ghost/mw-error-handler/node_modules:delegated
- node_modules_ghost_mw-session-from-token:/home/ghost/ghost/mw-session-from-token/node_modules:delegated
- node_modules_ghost_mw-update-user-last-seen:/home/ghost/ghost/mw-update-user-last-seen/node_modules:delegated
- node_modules_ghost_mw-version-match:/home/ghost/ghost/mw-version-match/node_modules:delegated
- node_modules_ghost_mw-vhost:/home/ghost/ghost/mw-vhost/node_modules:delegated
- node_modules_ghost_nql-filter-expansions:/home/ghost/ghost/nql-filter-expansions/node_modules:delegated
- node_modules_ghost_offers:/home/ghost/ghost/offers/node_modules:delegated
- node_modules_ghost_package-json:/home/ghost/ghost/package-json/node_modules:delegated
- node_modules_ghost_payments:/home/ghost/ghost/payments/node_modules:delegated
- node_modules_ghost_post-events:/home/ghost/ghost/post-events/node_modules:delegated
- node_modules_ghost_post-revisions:/home/ghost/ghost/post-revisions/node_modules:delegated
- node_modules_ghost_posts-service:/home/ghost/ghost/posts-service/node_modules:delegated
- node_modules_ghost_prometheus-metrics:/home/ghost/ghost/prometheus-metrics/node_modules:delegated
- node_modules_ghost_recommendations:/home/ghost/ghost/recommendations/node_modules:delegated
- node_modules_ghost_referrers:/home/ghost/ghost/referrers/node_modules:delegated
- node_modules_ghost_security:/home/ghost/ghost/security/node_modules:delegated
- node_modules_ghost_session-service:/home/ghost/ghost/session-service/node_modules:delegated
- node_modules_ghost_settings-path-manager:/home/ghost/ghost/settings-path-manager/node_modules:delegated
- node_modules_ghost_slack-notifications:/home/ghost/ghost/slack-notifications/node_modules:delegated
- node_modules_ghost_staff-service:/home/ghost/ghost/staff-service/node_modules:delegated
- node_modules_ghost_stats-service:/home/ghost/ghost/stats-service/node_modules:delegated
- node_modules_ghost_stripe:/home/ghost/ghost/stripe/node_modules:delegated
- node_modules_ghost_tiers:/home/ghost/ghost/tiers/node_modules:delegated
- node_modules_ghost_version-notifications-data-service:/home/ghost/ghost/version-notifications-data-service/node_modules:delegated
- node_modules_ghost_webmentions:/home/ghost/ghost/webmentions/node_modules:delegated
- node_modules_apps_admin-x-activitypub:/home/ghost/apps/admin-x-activitypub/node_modules:delegated
- node_modules_apps_admin-x-demo:/home/ghost/apps/admin-x-demo/node_modules:delegated
- node_modules_apps_admin-x-design-system:/home/ghost/apps/admin-x-design-system/node_modules:delegated
- node_modules_apps_admin-x-framework:/home/ghost/apps/admin-x-framework/node_modules:delegated
- node_modules_apps_admin-x-settings:/home/ghost/apps/admin-x-settings/node_modules:delegated
- node_modules_apps_announcement-bar:/home/ghost/apps/announcement-bar/node_modules:delegated
- node_modules_apps_comments-ui:/home/ghost/apps/comments-ui/node_modules:delegated
- node_modules_apps_portal:/home/ghost/apps/portal/node_modules:delegated
- node_modules_apps_posts:/home/ghost/apps/posts/node_modules:delegated
- node_modules_apps_shade:/home/ghost/apps/shade/node_modules:delegated
- node_modules_apps_signup-form:/home/ghost/apps/signup-form/node_modules:delegated
- node_modules_apps_sodo-search:/home/ghost/apps/sodo-search/node_modules:delegated
Added opt-in `ghost` service to docker compose (#21938) ref https://linear.app/ghost/issue/ENG-1957/add-ghost-service-to-base-docker-compose-setup - Currently our `compose.yml` file only runs Ghost's supporting services, and it's expected that you'll run Ghost locally on your host machine. This commit adds a `ghost` service to our `compose.yml` file, so you can optionally run Ghost itself in a container using docker compose. - The `ghost` service is opt-in using [docker compose profiles](https://docs.docker.com/compose/how-tos/profiles/), to maintain the current behavior of only running supporting services as the default. - This commit also fixes an issue in the Dockerfile: the `WORKDIR` arg, which is used to optionally specify an alternative working directory, is not propagated from one build stage to the next, so it has to be manually added as an `ARG` in each stage. This was necessary to use the same Dockerfile for devcontainers (which require the WORKDIR to be `/workspaces/ghost`) and docker compose, where we use `/home/ghost` in alignment with the production image.
2025-01-07 14:16:43 -08:00
tty: true
depends_on:
Added `service_healthy` condition for mysql and redis (#22119) no issue - Ghost would sometimes crash because it was booting before the database service was ready (even if the container had already started). This commit tells `docker compose` to only start the ghost service once the `mysql` and `redis` services have passed their health checks
2025-02-05 15:45:00 -08:00
mysql:
condition: service_healthy
redis:
condition: service_healthy
Added support for portal & other apps in containerized Ghost (#22121) no issue - The `yarn dev` command accepts dash-dash-args to conditionally build & run other apps, like portal, comments, etc. - Passing these flags to the docker compose setup directly doesn't work, so the only way to run these apps in docker was to either change the command in the Dockerfile or override the command in the `compose.yml` file - This commit extends `yarn dev` to also check for the `GHOST_DEV_APP_FLAGS` environment variable for these flags, which can be forwarded into the container and make it easier to run these apps using docker compose.
2025-02-05 18:18:58 -08:00
environment:
- DEBUG=${DEBUG:-}
Configured ssh agent forwarding for docker compose (#22232) no issue - Currently running any `git` commands in the docker container (i.e. updating submodules) that use ssh are failing because the container can't see your ssh keys. This commit adds a volume & environment variable to enable forwarding your local ssh agent into the container. As long as you've got an ssh agent running locally with your keys enabled, this will allow you to e.g. push to a remote over ssh from inside the container. - It also mounts your local `.gitconfig` file into the container, so your git configuration (i.e. name & email address) will also work inside the container - Finally, it adds githubs ssh keys to known_hosts in the development target of the Dockerfile to avoid the prompt, which only works in interactive environments and fails in any kind of script (like `yarn main:submodules`).
2025-02-19 09:38:05 -08:00
- SSH_AUTH_SOCK=/ssh-agent
Rename compose profile to ghost (#22223) no issue - Renamed the docker compose profile for Ghost from `full` to `ghost`, because it is clearer and makes more sense this way - Also added an explicit `GHOST_DEV_IS_DOCKER` environment variable to the `compose.yml` file to make checking if running in docker less fragile and more explicit.
2025-02-18 16:45:18 -08:00
- GHOST_DEV_IS_DOCKER=true
Added support for portal & other apps in containerized Ghost (#22121) no issue - The `yarn dev` command accepts dash-dash-args to conditionally build & run other apps, like portal, comments, etc. - Passing these flags to the docker compose setup directly doesn't work, so the only way to run these apps in docker was to either change the command in the Dockerfile or override the command in the `compose.yml` file - This commit extends `yarn dev` to also check for the `GHOST_DEV_APP_FLAGS` environment variable for these flags, which can be forwarded into the container and make it easier to run these apps using docker compose.
2025-02-05 18:18:58 -08:00
- GHOST_DEV_APP_FLAGS=${GHOST_DEV_APP_FLAGS:-}
Fixed flaky browser tests for scheduling posts (#22122) no issue - The browser tests for scheduling posts were flaking/failing because the time was set to `00:00`, which is invalid and should reset the time to "a few seconds from now", but the test was sometimes able to "Continue" before the validation logic could run on blur. This left the actual scheduled time set to `00:00`, which then caused the following assertions to fail. - I've tried to replicate this race condition manually to confirm it's not a bug that a user could actually reach, but I haven't been able to. I suspect that playwright's `.fill()` and/or `.click()` method behave slightly differently than expected, which creates this race condition.
2025-02-05 19:56:07 -08:00
- STRIPE_SECRET_KEY=${STRIPE_SECRET_KEY:-}
- STRIPE_PUBLISHABLE_KEY=${STRIPE_PUBLISHABLE_KEY:-}
- STRIPE_ACCOUNT_ID=${STRIPE_ACCOUNT_ID:-}
Added MySQL setup via Docker refs https://github.com/TryGhost/DevOps/issues/118 - we should standardize how we run MySQL, so there aren't massive gaps in developer experience - Docker is great for this, and it's pretty easy to set up - this adds a docker-compose.yml file with a small MySQL setup - also configures `yarn setup` to spin up the Docker container and add it to config.local.json - in the near future, we'll provide a base SQL file in .github/scripts/mysql-preload to ensure the DB is in a known state
2023-12-06 20:11:52 +01:00
mysql:
image: mysql:8.0.35
container_name: ghost-mysql
Moved @tryghost/update-check-service to services folder fix https://linear.app/ghost/issue/ENG-2039/move-tryghostupdate-check-service-to-services-folder - this service is already self-contained and can be moved into the services folder, which means we can also remove the package that was supporting it - minor other changes just to fix paths
2025-02-26 16:48:03 +01:00
command: --innodb-buffer-pool-size=1G --innodb-log-buffer-size=500M --innodb-change-buffer-max-size=50 --innodb-flush-log-at-trx_commit=0 --innodb-flush-method=O_DIRECT
Added MySQL setup via Docker refs https://github.com/TryGhost/DevOps/issues/118 - we should standardize how we run MySQL, so there aren't massive gaps in developer experience - Docker is great for this, and it's pretty easy to set up - this adds a docker-compose.yml file with a small MySQL setup - also configures `yarn setup` to spin up the Docker container and add it to config.local.json - in the near future, we'll provide a base SQL file in .github/scripts/mysql-preload to ensure the DB is in a known state
2023-12-06 20:11:52 +01:00
ports:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
- 3306:3306
Added MySQL setup via Docker refs https://github.com/TryGhost/DevOps/issues/118 - we should standardize how we run MySQL, so there aren't massive gaps in developer experience - Docker is great for this, and it's pretty easy to set up - this adds a docker-compose.yml file with a small MySQL setup - also configures `yarn setup` to spin up the Docker container and add it to config.local.json - in the near future, we'll provide a base SQL file in .github/scripts/mysql-preload to ensure the DB is in a known state
2023-12-06 20:11:52 +01:00
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: ghost
restart: always
volumes:
Reorganized docker related files (#21650) ref https://linear.app/ghost/issue/ENG-1782/move-docker-related-files-to-a-better-location-that-githubscripts - The docker files are currently located in `.github/scripts`. This location doesn't make a lot of sense — you wouldn't think to look there unless you already knew they were there. This also requires you to specify the path to the `compose.yml` file whenever running a `docker compose ...` command. - This commit moves the `compose.yml` file to the root of the repo, so you can simply run `docker compose up` and it will automatically find the file in the root, without having to specify `-f .github/scripts/docker-compose.yml`. This is a major win for convenience over the current setup. - It also moves all the related files, including the `Dockerfile` used by the Dev Container setup and configuration files for supporting services into a new `.docker` directory, which is a more logical location, and should be easier to find. - Also updated the current convenience commands in the `package.json` scripts block (`yarn docker:reset` and `yarn docker:down`
2024-11-19 13:15:06 -08:00
- ./.docker/mysql-preload:/docker-entrypoint-initdb.d
Added MySQL data volume to docker compose (#20982) no issue - This allows us to run `docker-compose down` or to restart docker desktop without losing all our local databases - Added a data volume to the MySQL service in the `docker-compose.yml` file to persist the data between container restarts - The `yarn docker:reset` command will still reset all the data in the database since it uses `down -v` to remove the volumes as well
2024-09-12 09:38:24 -07:00
- mysql-data:/var/lib/mysql
Added MySQL setup via Docker refs https://github.com/TryGhost/DevOps/issues/118 - we should standardize how we run MySQL, so there aren't massive gaps in developer experience - Docker is great for this, and it's pretty easy to set up - this adds a docker-compose.yml file with a small MySQL setup - also configures `yarn setup` to spin up the Docker container and add it to config.local.json - in the near future, we'll provide a base SQL file in .github/scripts/mysql-preload to ensure the DB is in a known state
2023-12-06 20:11:52 +01:00
healthcheck:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
test: mysql -uroot -proot ghost -e 'select 1'
Added MySQL setup via Docker refs https://github.com/TryGhost/DevOps/issues/118 - we should standardize how we run MySQL, so there aren't massive gaps in developer experience - Docker is great for this, and it's pretty easy to set up - this adds a docker-compose.yml file with a small MySQL setup - also configures `yarn setup` to spin up the Docker container and add it to config.local.json - in the near future, we'll provide a base SQL file in .github/scripts/mysql-preload to ensure the DB is in a known state
2023-12-06 20:11:52 +01:00
interval: 1s
retries: 120
Added Redis via Docker (#20085) no refs Redis can be utilised for various caching purposes within Ghost. This PR adds a Redis service to the docker-compose file to allow for easier local development when Redis is required
2024-05-07 11:02:36 +01:00
redis:
image: redis:7.0
container_name: ghost-redis
ports:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
- 6379:6379
Added Redis via Docker (#20085) no refs Redis can be utilised for various caching purposes within Ghost. This PR adds a Redis service to the docker-compose file to allow for easier local development when Redis is required
2024-05-07 11:02:36 +01:00
restart: always
Added redis data volume to compose setup (#21906) ref https://linear.app/ghost/issue/ENG-1956/redis-creates-new-anonymous-volume-each-time-it-boots - Before this commit, the redis service in our docker compose setup would create a new anonymous volume each time it boots, with a non-descriptive, hash-based name. Over time these volumes accumulate and become a pain to clean up, and it's not immediately obvious what they are used for. - This commit adds a persistent data volume for redis data, so it will reuse the same volume, with a more descriptive title each time it boots. This eliminates the annoying anonymous volumes, and also gives us data persistence across boots for the redis service (which can be easily cleared by deleting the volume).
2024-12-17 16:14:21 -05:00
volumes:
- redis-data:/data
Consolidated `compose.yml` files for Dev Container (#21659) ref https://linear.app/ghost/issue/ENG-1785/consolidate-dev-container-composeyml-with-the-root-level-composeyml - We currently have two compose.yml files: - `.devcontainer/compose.yml`: used for the Dev Container - `compose.yml`: used for local development with docker compose - The Dev Container compose file does need some additional configuration, but most of this code is duplicated from the root level compose.yml file - This commit removes this duplication by using both compose files in the Dev Container setup, so it will base the compose configuration on the root level, and then extend it with the Dev Container specific configuration - This way any updates to the root level compose file will also be available and reflected in the Dev Container setup without having to make the change twice.
2024-12-17 15:57:02 -05:00
healthcheck:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
test:
- CMD
- redis-cli
- --raw
- incr
- ping
Consolidated `compose.yml` files for Dev Container (#21659) ref https://linear.app/ghost/issue/ENG-1785/consolidate-dev-container-composeyml-with-the-root-level-composeyml - We currently have two compose.yml files: - `.devcontainer/compose.yml`: used for the Dev Container - `compose.yml`: used for local development with docker compose - The Dev Container compose file does need some additional configuration, but most of this code is duplicated from the root level compose.yml file - This commit removes this duplication by using both compose files in the Dev Container setup, so it will base the compose configuration on the root level, and then extend it with the Dev Container specific configuration - This way any updates to the root level compose file will also be available and reflected in the Dev Container setup without having to make the change twice.
2024-12-17 15:57:02 -05:00
interval: 1s
retries: 120
Added prometheus and grafana services to docker compose (#21213) ref https://linear.app/tryghost/issue/ENG-1591/add-prometheus-and-grafana-services-to-docker-compose This commit adds 2 new services to the docker compose file to enable monitoring metrics from Ghost locally in real-time: 1. Prometheus - a service that scrapes Ghost's new `/metrics` endpoint introduced in this [commit](https://github.com/TryGhost/Ghost/commit/768336efad067241ca848027f4ac758148ca3e29). 2. Grafana - a service that consumes the metrics from prometheus and exposes them in a dashboard that you can view locally at `localhost:3000`. # Usage Both of these services are selectively enabled using docker compose [profiles](https://docs.docker.com/compose/how-tos/profiles/). This way, if you don't opt-in to using these monitoring tools, they won't start and consume resources on your host machine. To enable these services, enable the `monitoring` profile by either setting the `COMPOSE_PROFILES` environment variable to `monitoring`, or specifying the `--profile monitoring` CLI argument to any `docker compose ...` commands. I've found the easiest way to configure this in an 'always on' fashion is to create a `.env` file in the project's root directory and add `COMPOSE_PROFILES=monitoring` to it. As an added convenience, you can also set `COMPOSE_FILE=.github/scripts/docker-compose.yml`, which will allow you to run `docker compose ...` commands from the root directory without specifying the full path each time. # Intended for development only These services are meant for local development only, and are not configured for a production use-case. For example, the Grafana instance is configured to have _no authorization_ so you won't need a username/password to login at `localhost:3000`. Prometheus is also configured to scrape the metrics once every second, which is likely excessive for production use-cases, but may be useful for getting more granular metrics while e.g. load testing locally. # Dashboards The Grafana instance includes a default dashboard including most of the main default metrics provided by our prometheus client integration. The dashboard is defined in a JSON file at `.github/scripts/docker/grafana/dashboards/main-dashboard.json' and can be modified & committed to add new visualizations that will be available to anyone work on Ghost locally. You can also add other dashboards to the same directory for specific use-cases, which should be picked up and made available in the Grafana UI. [Read more](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/view-dashboard-json-model/) about Grafana's JSON schema for dashboards.
2024-10-03 14:43:07 -07:00
prometheus:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
profiles: [ monitoring ]
Added prometheus and grafana services to docker compose (#21213) ref https://linear.app/tryghost/issue/ENG-1591/add-prometheus-and-grafana-services-to-docker-compose This commit adds 2 new services to the docker compose file to enable monitoring metrics from Ghost locally in real-time: 1. Prometheus - a service that scrapes Ghost's new `/metrics` endpoint introduced in this [commit](https://github.com/TryGhost/Ghost/commit/768336efad067241ca848027f4ac758148ca3e29). 2. Grafana - a service that consumes the metrics from prometheus and exposes them in a dashboard that you can view locally at `localhost:3000`. # Usage Both of these services are selectively enabled using docker compose [profiles](https://docs.docker.com/compose/how-tos/profiles/). This way, if you don't opt-in to using these monitoring tools, they won't start and consume resources on your host machine. To enable these services, enable the `monitoring` profile by either setting the `COMPOSE_PROFILES` environment variable to `monitoring`, or specifying the `--profile monitoring` CLI argument to any `docker compose ...` commands. I've found the easiest way to configure this in an 'always on' fashion is to create a `.env` file in the project's root directory and add `COMPOSE_PROFILES=monitoring` to it. As an added convenience, you can also set `COMPOSE_FILE=.github/scripts/docker-compose.yml`, which will allow you to run `docker compose ...` commands from the root directory without specifying the full path each time. # Intended for development only These services are meant for local development only, and are not configured for a production use-case. For example, the Grafana instance is configured to have _no authorization_ so you won't need a username/password to login at `localhost:3000`. Prometheus is also configured to scrape the metrics once every second, which is likely excessive for production use-cases, but may be useful for getting more granular metrics while e.g. load testing locally. # Dashboards The Grafana instance includes a default dashboard including most of the main default metrics provided by our prometheus client integration. The dashboard is defined in a JSON file at `.github/scripts/docker/grafana/dashboards/main-dashboard.json' and can be modified & committed to add new visualizations that will be available to anyone work on Ghost locally. You can also add other dashboards to the same directory for specific use-cases, which should be picked up and made available in the Grafana UI. [Read more](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/view-dashboard-json-model/) about Grafana's JSON schema for dashboards.
2024-10-03 14:43:07 -07:00
image: prom/prometheus:v2.30.3
container_name: ghost-prometheus
ports:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
- 9090:9090
Added prometheus and grafana services to docker compose (#21213) ref https://linear.app/tryghost/issue/ENG-1591/add-prometheus-and-grafana-services-to-docker-compose This commit adds 2 new services to the docker compose file to enable monitoring metrics from Ghost locally in real-time: 1. Prometheus - a service that scrapes Ghost's new `/metrics` endpoint introduced in this [commit](https://github.com/TryGhost/Ghost/commit/768336efad067241ca848027f4ac758148ca3e29). 2. Grafana - a service that consumes the metrics from prometheus and exposes them in a dashboard that you can view locally at `localhost:3000`. # Usage Both of these services are selectively enabled using docker compose [profiles](https://docs.docker.com/compose/how-tos/profiles/). This way, if you don't opt-in to using these monitoring tools, they won't start and consume resources on your host machine. To enable these services, enable the `monitoring` profile by either setting the `COMPOSE_PROFILES` environment variable to `monitoring`, or specifying the `--profile monitoring` CLI argument to any `docker compose ...` commands. I've found the easiest way to configure this in an 'always on' fashion is to create a `.env` file in the project's root directory and add `COMPOSE_PROFILES=monitoring` to it. As an added convenience, you can also set `COMPOSE_FILE=.github/scripts/docker-compose.yml`, which will allow you to run `docker compose ...` commands from the root directory without specifying the full path each time. # Intended for development only These services are meant for local development only, and are not configured for a production use-case. For example, the Grafana instance is configured to have _no authorization_ so you won't need a username/password to login at `localhost:3000`. Prometheus is also configured to scrape the metrics once every second, which is likely excessive for production use-cases, but may be useful for getting more granular metrics while e.g. load testing locally. # Dashboards The Grafana instance includes a default dashboard including most of the main default metrics provided by our prometheus client integration. The dashboard is defined in a JSON file at `.github/scripts/docker/grafana/dashboards/main-dashboard.json' and can be modified & committed to add new visualizations that will be available to anyone work on Ghost locally. You can also add other dashboards to the same directory for specific use-cases, which should be picked up and made available in the Grafana UI. [Read more](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/view-dashboard-json-model/) about Grafana's JSON schema for dashboards.
2024-10-03 14:43:07 -07:00
restart: always
volumes:
Reorganized docker related files (#21650) ref https://linear.app/ghost/issue/ENG-1782/move-docker-related-files-to-a-better-location-that-githubscripts - The docker files are currently located in `.github/scripts`. This location doesn't make a lot of sense — you wouldn't think to look there unless you already knew they were there. This also requires you to specify the path to the `compose.yml` file whenever running a `docker compose ...` command. - This commit moves the `compose.yml` file to the root of the repo, so you can simply run `docker compose up` and it will automatically find the file in the root, without having to specify `-f .github/scripts/docker-compose.yml`. This is a major win for convenience over the current setup. - It also moves all the related files, including the `Dockerfile` used by the Dev Container setup and configuration files for supporting services into a new `.docker` directory, which is a more logical location, and should be easier to find. - Also updated the current convenience commands in the `package.json` scripts block (`yarn docker:reset` and `yarn docker:down`
2024-11-19 13:15:06 -08:00
- ./.docker/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
Added prometheus and grafana services to docker compose (#21213) ref https://linear.app/tryghost/issue/ENG-1591/add-prometheus-and-grafana-services-to-docker-compose This commit adds 2 new services to the docker compose file to enable monitoring metrics from Ghost locally in real-time: 1. Prometheus - a service that scrapes Ghost's new `/metrics` endpoint introduced in this [commit](https://github.com/TryGhost/Ghost/commit/768336efad067241ca848027f4ac758148ca3e29). 2. Grafana - a service that consumes the metrics from prometheus and exposes them in a dashboard that you can view locally at `localhost:3000`. # Usage Both of these services are selectively enabled using docker compose [profiles](https://docs.docker.com/compose/how-tos/profiles/). This way, if you don't opt-in to using these monitoring tools, they won't start and consume resources on your host machine. To enable these services, enable the `monitoring` profile by either setting the `COMPOSE_PROFILES` environment variable to `monitoring`, or specifying the `--profile monitoring` CLI argument to any `docker compose ...` commands. I've found the easiest way to configure this in an 'always on' fashion is to create a `.env` file in the project's root directory and add `COMPOSE_PROFILES=monitoring` to it. As an added convenience, you can also set `COMPOSE_FILE=.github/scripts/docker-compose.yml`, which will allow you to run `docker compose ...` commands from the root directory without specifying the full path each time. # Intended for development only These services are meant for local development only, and are not configured for a production use-case. For example, the Grafana instance is configured to have _no authorization_ so you won't need a username/password to login at `localhost:3000`. Prometheus is also configured to scrape the metrics once every second, which is likely excessive for production use-cases, but may be useful for getting more granular metrics while e.g. load testing locally. # Dashboards The Grafana instance includes a default dashboard including most of the main default metrics provided by our prometheus client integration. The dashboard is defined in a JSON file at `.github/scripts/docker/grafana/dashboards/main-dashboard.json' and can be modified & committed to add new visualizations that will be available to anyone work on Ghost locally. You can also add other dashboards to the same directory for specific use-cases, which should be picked up and made available in the Grafana UI. [Read more](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/view-dashboard-json-model/) about Grafana's JSON schema for dashboards.
2024-10-03 14:43:07 -07:00
grafana:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
profiles: [ monitoring ]
Added connection pool metrics to prometheus client (#21576) ref https://linear.app/ghost/issue/ENG-1592/start-monitoring-connection-pool-utilization-in-ghost - This commit adds prometheus metrics to the connection pool so we can start to track connection pool utilization, number of pending acquires, and also adds some basic SQL query summary metrics like queries per minute and query duration percentiles. - The connection pool has now been theorized to be a main constraint of Ghost for some time, but it's been challenging to get actual visibility into the state of the connection pool. With this change, we should be able to directly observe, monitor and alert on the connection pool. - Updated grafana version to fix a bug in the query editor that was fixed in 8.3, even though this is a couple versions ahead of production
2024-11-07 23:01:34 -08:00
image: grafana/grafana:8.3.0
Added prometheus and grafana services to docker compose (#21213) ref https://linear.app/tryghost/issue/ENG-1591/add-prometheus-and-grafana-services-to-docker-compose This commit adds 2 new services to the docker compose file to enable monitoring metrics from Ghost locally in real-time: 1. Prometheus - a service that scrapes Ghost's new `/metrics` endpoint introduced in this [commit](https://github.com/TryGhost/Ghost/commit/768336efad067241ca848027f4ac758148ca3e29). 2. Grafana - a service that consumes the metrics from prometheus and exposes them in a dashboard that you can view locally at `localhost:3000`. # Usage Both of these services are selectively enabled using docker compose [profiles](https://docs.docker.com/compose/how-tos/profiles/). This way, if you don't opt-in to using these monitoring tools, they won't start and consume resources on your host machine. To enable these services, enable the `monitoring` profile by either setting the `COMPOSE_PROFILES` environment variable to `monitoring`, or specifying the `--profile monitoring` CLI argument to any `docker compose ...` commands. I've found the easiest way to configure this in an 'always on' fashion is to create a `.env` file in the project's root directory and add `COMPOSE_PROFILES=monitoring` to it. As an added convenience, you can also set `COMPOSE_FILE=.github/scripts/docker-compose.yml`, which will allow you to run `docker compose ...` commands from the root directory without specifying the full path each time. # Intended for development only These services are meant for local development only, and are not configured for a production use-case. For example, the Grafana instance is configured to have _no authorization_ so you won't need a username/password to login at `localhost:3000`. Prometheus is also configured to scrape the metrics once every second, which is likely excessive for production use-cases, but may be useful for getting more granular metrics while e.g. load testing locally. # Dashboards The Grafana instance includes a default dashboard including most of the main default metrics provided by our prometheus client integration. The dashboard is defined in a JSON file at `.github/scripts/docker/grafana/dashboards/main-dashboard.json' and can be modified & committed to add new visualizations that will be available to anyone work on Ghost locally. You can also add other dashboards to the same directory for specific use-cases, which should be picked up and made available in the Grafana UI. [Read more](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/view-dashboard-json-model/) about Grafana's JSON schema for dashboards.
2024-10-03 14:43:07 -07:00
container_name: ghost-grafana
ports:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
- 3000:3000
Added prometheus and grafana services to docker compose (#21213) ref https://linear.app/tryghost/issue/ENG-1591/add-prometheus-and-grafana-services-to-docker-compose This commit adds 2 new services to the docker compose file to enable monitoring metrics from Ghost locally in real-time: 1. Prometheus - a service that scrapes Ghost's new `/metrics` endpoint introduced in this [commit](https://github.com/TryGhost/Ghost/commit/768336efad067241ca848027f4ac758148ca3e29). 2. Grafana - a service that consumes the metrics from prometheus and exposes them in a dashboard that you can view locally at `localhost:3000`. # Usage Both of these services are selectively enabled using docker compose [profiles](https://docs.docker.com/compose/how-tos/profiles/). This way, if you don't opt-in to using these monitoring tools, they won't start and consume resources on your host machine. To enable these services, enable the `monitoring` profile by either setting the `COMPOSE_PROFILES` environment variable to `monitoring`, or specifying the `--profile monitoring` CLI argument to any `docker compose ...` commands. I've found the easiest way to configure this in an 'always on' fashion is to create a `.env` file in the project's root directory and add `COMPOSE_PROFILES=monitoring` to it. As an added convenience, you can also set `COMPOSE_FILE=.github/scripts/docker-compose.yml`, which will allow you to run `docker compose ...` commands from the root directory without specifying the full path each time. # Intended for development only These services are meant for local development only, and are not configured for a production use-case. For example, the Grafana instance is configured to have _no authorization_ so you won't need a username/password to login at `localhost:3000`. Prometheus is also configured to scrape the metrics once every second, which is likely excessive for production use-cases, but may be useful for getting more granular metrics while e.g. load testing locally. # Dashboards The Grafana instance includes a default dashboard including most of the main default metrics provided by our prometheus client integration. The dashboard is defined in a JSON file at `.github/scripts/docker/grafana/dashboards/main-dashboard.json' and can be modified & committed to add new visualizations that will be available to anyone work on Ghost locally. You can also add other dashboards to the same directory for specific use-cases, which should be picked up and made available in the Grafana UI. [Read more](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/view-dashboard-json-model/) about Grafana's JSON schema for dashboards.
2024-10-03 14:43:07 -07:00
restart: always
environment:
- GF_AUTH_ANONYMOUS_ENABLED=true
Added connection pool metrics to prometheus client (#21576) ref https://linear.app/ghost/issue/ENG-1592/start-monitoring-connection-pool-utilization-in-ghost - This commit adds prometheus metrics to the connection pool so we can start to track connection pool utilization, number of pending acquires, and also adds some basic SQL query summary metrics like queries per minute and query duration percentiles. - The connection pool has now been theorized to be a main constraint of Ghost for some time, but it's been challenging to get actual visibility into the state of the connection pool. With this change, we should be able to directly observe, monitor and alert on the connection pool. - Updated grafana version to fix a bug in the query editor that was fixed in 8.3, even though this is a couple versions ahead of production
2024-11-07 23:01:34 -08:00
- GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
Added prometheus and grafana services to docker compose (#21213) ref https://linear.app/tryghost/issue/ENG-1591/add-prometheus-and-grafana-services-to-docker-compose This commit adds 2 new services to the docker compose file to enable monitoring metrics from Ghost locally in real-time: 1. Prometheus - a service that scrapes Ghost's new `/metrics` endpoint introduced in this [commit](https://github.com/TryGhost/Ghost/commit/768336efad067241ca848027f4ac758148ca3e29). 2. Grafana - a service that consumes the metrics from prometheus and exposes them in a dashboard that you can view locally at `localhost:3000`. # Usage Both of these services are selectively enabled using docker compose [profiles](https://docs.docker.com/compose/how-tos/profiles/). This way, if you don't opt-in to using these monitoring tools, they won't start and consume resources on your host machine. To enable these services, enable the `monitoring` profile by either setting the `COMPOSE_PROFILES` environment variable to `monitoring`, or specifying the `--profile monitoring` CLI argument to any `docker compose ...` commands. I've found the easiest way to configure this in an 'always on' fashion is to create a `.env` file in the project's root directory and add `COMPOSE_PROFILES=monitoring` to it. As an added convenience, you can also set `COMPOSE_FILE=.github/scripts/docker-compose.yml`, which will allow you to run `docker compose ...` commands from the root directory without specifying the full path each time. # Intended for development only These services are meant for local development only, and are not configured for a production use-case. For example, the Grafana instance is configured to have _no authorization_ so you won't need a username/password to login at `localhost:3000`. Prometheus is also configured to scrape the metrics once every second, which is likely excessive for production use-cases, but may be useful for getting more granular metrics while e.g. load testing locally. # Dashboards The Grafana instance includes a default dashboard including most of the main default metrics provided by our prometheus client integration. The dashboard is defined in a JSON file at `.github/scripts/docker/grafana/dashboards/main-dashboard.json' and can be modified & committed to add new visualizations that will be available to anyone work on Ghost locally. You can also add other dashboards to the same directory for specific use-cases, which should be picked up and made available in the Grafana UI. [Read more](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/view-dashboard-json-model/) about Grafana's JSON schema for dashboards.
2024-10-03 14:43:07 -07:00
volumes:
Reorganized docker related files (#21650) ref https://linear.app/ghost/issue/ENG-1782/move-docker-related-files-to-a-better-location-that-githubscripts - The docker files are currently located in `.github/scripts`. This location doesn't make a lot of sense — you wouldn't think to look there unless you already knew they were there. This also requires you to specify the path to the `compose.yml` file whenever running a `docker compose ...` command. - This commit moves the `compose.yml` file to the root of the repo, so you can simply run `docker compose up` and it will automatically find the file in the root, without having to specify `-f .github/scripts/docker-compose.yml`. This is a major win for convenience over the current setup. - It also moves all the related files, including the `Dockerfile` used by the Dev Container setup and configuration files for supporting services into a new `.docker` directory, which is a more logical location, and should be easier to find. - Also updated the current convenience commands in the `package.json` scripts block (`yarn docker:reset` and `yarn docker:down`
2024-11-19 13:15:06 -08:00
- ./.docker/grafana/datasources:/etc/grafana/provisioning/datasources
- ./.docker/grafana/dashboard.yml:/etc/grafana/provisioning/dashboards/main.yaml
- ./.docker/grafana/dashboards:/var/lib/grafana/dashboards
Added ability to push prometheus metrics to a pushgateway (#21526) ref https://linear.app/ghost/issue/ENG-1746/enable-ghost-to-push-metrics-to-a-pushgateway - We'd like to use prometheus to expose metrics from Ghost, but the "standard" approach of having prometheus scrape the `/metrics` endpoint adds some complexity and additional challenges on Pro. - A suggested simpler alternative is to use a pushgateway, to have Ghost _push_ metrics to prometheus, rather than have prometheus scrape the running instances. - This PR introduces this functionality behind a configuration. - It also includes a refactor to the current metrics-server implementation so all the related code for prometheus is colocated, and the configuration is a bit more organized. `@tryghost/metrics-server` has been renamed to `@tryghost/prometheus-metrics`, and it now includes the metrics server and prometheus-client code itself (including the pushgateway code) - To enable the prometheus client alone, `prometheus:enabled` must be true. This will _not_ enable the metrics server or the pushgateway — it will essentially collect the metrics, but not do anything with them. - To enable the metrics server, set `prometheus:metrics_server:enabled` to true. You can also configure the host and port that the metrics server should export the `/metrics` endpoint on in the `prometheus:metrics_server` block. - To enable the pushgateway, set `prometheus:pushgateway:enabled` to true. You can also configure the pushgateway's `url`, the `interval` it should push metrics in (in milliseconds) and the `jobName` in the `prometheus:pushgateway` block.
2024-11-05 11:50:39 -08:00
pushgateway:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
profiles: [ monitoring ]
Added ability to push prometheus metrics to a pushgateway (#21526) ref https://linear.app/ghost/issue/ENG-1746/enable-ghost-to-push-metrics-to-a-pushgateway - We'd like to use prometheus to expose metrics from Ghost, but the "standard" approach of having prometheus scrape the `/metrics` endpoint adds some complexity and additional challenges on Pro. - A suggested simpler alternative is to use a pushgateway, to have Ghost _push_ metrics to prometheus, rather than have prometheus scrape the running instances. - This PR introduces this functionality behind a configuration. - It also includes a refactor to the current metrics-server implementation so all the related code for prometheus is colocated, and the configuration is a bit more organized. `@tryghost/metrics-server` has been renamed to `@tryghost/prometheus-metrics`, and it now includes the metrics server and prometheus-client code itself (including the pushgateway code) - To enable the prometheus client alone, `prometheus:enabled` must be true. This will _not_ enable the metrics server or the pushgateway — it will essentially collect the metrics, but not do anything with them. - To enable the metrics server, set `prometheus:metrics_server:enabled` to true. You can also configure the host and port that the metrics server should export the `/metrics` endpoint on in the `prometheus:metrics_server` block. - To enable the pushgateway, set `prometheus:pushgateway:enabled` to true. You can also configure the pushgateway's `url`, the `interval` it should push metrics in (in milliseconds) and the `jobName` in the `prometheus:pushgateway` block.
2024-11-05 11:50:39 -08:00
image: prom/pushgateway:v1.6.0
container_name: ghost-pushgateway
ports:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
- 9091:9091
Added MySQL data volume to docker compose (#20982) no issue - This allows us to run `docker-compose down` or to restart docker desktop without losing all our local databases - Added a data volume to the MySQL service in the `docker-compose.yml` file to persist the data between container restarts - The `yarn docker:reset` command will still reset all the data in the database since it uses `down -v` to remove the volumes as well
2024-09-12 09:38:24 -07:00
volumes:
Improved docker development setup (#22177) no issue This commit is a fairly significant rework of the docker development setup, with the goal of improving the performance of the Ghost container and generally improving the DX of the docker development setup. **Enumerating `node_modules` directories as volume exclusions for the bind mount** - The main source of performance overhead with the docker setup is the bind mount, which mounts all the code into the container. Especially on macOS, the bind mount introduces quite a bit of file system overhead. Currently the bind mount includes all the `node_modules` directories, which adds up to many thousands of files in the bind mount. This change attempts to reduce this overhead by mounting in named volumes for each `node_modules` directory in the monorepo, which effectively tells the main bind mount to ignore these directories, thus reducing the filesystem overhead. **Removing the `yarn install` from the entrypoint script and the `yarn setup:docker` step** - This change has two more benefits: since all the `node_modules` are installed into named volumes, we can now be confident that our local `node_modules` won't conflict with the `node_modules` in the container. Therefore, we can install `node_modules` at build time in the Dockerfile instead of in the entrypoint script, and we can also get rid of the `yarn setup:docker` setup command, which nukes your local `node_modules` before building the container. **Build optimization - enumerating all `package.json` files** - In a typical node app's Dockerfile, you copy the `package.json` and `yarn.lock` files into the container first, then run `yarn install`, then copy the rest of the code in. This takes advantage of Docker's build cache to avoid having to reinstall `node_modules` on every build if you've only changed code files, but not any dependencies. - This is more challenging to do with the monorepo since we have many `package.json` files. This commit brute forces our way through that by copying every single `package.json` file in the Dockerfile before installing dependencies and copying the code in. The result is that Docker builds become much faster, after the initial full build. **Additional scripts** - Since enumerating all the `node_modules` directories and `package.json` files is tedious to do manually, this commit also includes two new scripts to automate this work. Ideally we'll eventually integrate these scripts with Slimer, so when you add a new package to the monorepo it automatically updates the `Dockerfile` and `compose.yml`, but for now we can manage that ourselves since it's a fairly rare occurrence.
2025-02-18 15:47:39 -08:00
mysql-data: {}
redis-data: {}
node_modules_ghost_root: {}
node_modules_ghost_activitypub: {}
node_modules_ghost_adapter-cache-memory-ttl: {}
node_modules_ghost_adapter-cache-redis: {}
node_modules_ghost_adapter-manager: {}
node_modules_ghost_admin: {}
node_modules_ghost_announcement-bar-settings: {}
node_modules_ghost_api-framework: {}
node_modules_ghost_api-version-compatibility-service: {}
node_modules_ghost_audience-feedback: {}
node_modules_ghost_bookshelf-repository: {}
node_modules_ghost_bootstrap-socket: {}
node_modules_ghost_captcha-service: {}
node_modules_ghost_constants: {}
node_modules_ghost_core: {}
node_modules_ghost_custom-fonts: {}
node_modules_ghost_custom-theme-settings-service: {}
node_modules_ghost_data-generator: {}
node_modules_ghost_domain-events: {}
node_modules_ghost_donations: {}
node_modules_ghost_dynamic-routing-events: {}
node_modules_ghost_email-addresses: {}
node_modules_ghost_email-analytics-provider-mailgun: {}
node_modules_ghost_email-analytics-service: {}
node_modules_ghost_email-content-generator: {}
node_modules_ghost_email-events: {}
node_modules_ghost_email-service: {}
node_modules_ghost_email-suppression-list: {}
node_modules_ghost_express-dynamic-redirects: {}
node_modules_ghost_external-media-inliner: {}
node_modules_ghost_ghost: {}
node_modules_ghost_html-to-plaintext: {}
node_modules_ghost_i18n: {}
node_modules_ghost_identity-token-service: {}
node_modules_ghost_importer-handler-content-files: {}
node_modules_ghost_importer-revue: {}
node_modules_ghost_in-memory-repository: {}
node_modules_ghost_job-manager: {}
node_modules_ghost_link-redirects: {}
node_modules_ghost_link-replacer: {}
node_modules_ghost_magic-link: {}
node_modules_ghost_mail-events: {}
node_modules_ghost_mailgun-client: {}
node_modules_ghost_member-attribution: {}
node_modules_ghost_member-events: {}
node_modules_ghost_members-api: {}
node_modules_ghost_members-csv: {}
node_modules_ghost_members-events-service: {}
node_modules_ghost_members-importer: {}
node_modules_ghost_members-ssr: {}
node_modules_ghost_mentions-email-report: {}
node_modules_ghost_metrics-server: {}
node_modules_ghost_milestones: {}
node_modules_ghost_minifier: {}
node_modules_ghost_model-to-domain-event-interceptor: {}
node_modules_ghost_mw-api-version-mismatch: {}
node_modules_ghost_mw-cache-control: {}
node_modules_ghost_mw-error-handler: {}
node_modules_ghost_mw-session-from-token: {}
node_modules_ghost_mw-update-user-last-seen: {}
node_modules_ghost_mw-version-match: {}
node_modules_ghost_mw-vhost: {}
node_modules_ghost_nql-filter-expansions: {}
node_modules_ghost_offers: {}
node_modules_ghost_package-json: {}
node_modules_ghost_payments: {}
node_modules_ghost_post-events: {}
node_modules_ghost_post-revisions: {}
node_modules_ghost_posts-service: {}
node_modules_ghost_prometheus-metrics: {}
node_modules_ghost_recommendations: {}
node_modules_ghost_referrers: {}
node_modules_ghost_security: {}
node_modules_ghost_session-service: {}
node_modules_ghost_settings-path-manager: {}
node_modules_ghost_slack-notifications: {}
node_modules_ghost_staff-service: {}
node_modules_ghost_stats-service: {}
node_modules_ghost_stripe: {}
node_modules_ghost_tiers: {}
node_modules_ghost_version-notifications-data-service: {}
node_modules_ghost_webmentions: {}
node_modules_apps_admin-x-activitypub: {}
node_modules_apps_admin-x-demo: {}
node_modules_apps_admin-x-design-system: {}
node_modules_apps_admin-x-framework: {}
node_modules_apps_admin-x-settings: {}
node_modules_apps_announcement-bar: {}
node_modules_apps_comments-ui: {}
node_modules_apps_portal: {}
node_modules_apps_posts: {}
node_modules_apps_shade: {}
node_modules_apps_signup-form: {}
node_modules_apps_sodo-search: {}
Reference in a new issue Copy permalink