no issue
Problem:
- Running `yarn dev` with `--stripe` runs the `stripe listen ...`
command in a subprocess to forward webhook events to your local instance
of Ghost in development.
- Currently, you have to run `stripe login` manually before this will
work. If you fail to do so, the `yarn dev` script simply hangs
indefinitely without any logging to indicate why.
- Running `stripe login` manually is also a non-starter in Docker since
you'd have to do this for each new container instance, which isn't
practical.
Solution:
- Luckily the `stripe listen ...` command accepts an `--api-key`
argument, which expects your stripe secret key. This change updates the
`dev.js` script to read your stripe API key from the `STRIPE_SECRET_KEY`
environment variable if it's present. As long as `STRIPE_SECRET_KEY`
exists, you won't need to run `stripe login` anymore.
- It also adds a timeout of 5 seconds to the initial `stripe listen`
command and will exit the process if this step fails, indicating with a
clear log message that you need to either set the `STRIPE_SECRET_KEY`
variable or run `stripe login`
- I also added a bunch of debug statements to the dev script which were
useful in debugging this, and may prove useful for similar issues in the
future, so I'll leave them in there.
- the original implementation of this was wrong in that it would
calculate the init time from a static point in the boot process until
"now", which would be fine for the boot process
- however, if someone uploads a routes.yaml file much later, it would
still call `onFinished` and compare itself to the static time in the
boot process, resulting in huge "url service init" metrics
- to fix that, we can just collect the timestamp from `_onQueueStarted`, calculate
the diff in `_onQueueEnded` and then do logging/metrics
- as a result, we can then remove the `onFinished` function, as it's no
longer necessary
no issue
- Currently running any `git` commands in the docker container (i.e.
updating submodules) that use ssh are failing because the container
can't see your ssh keys. This commit adds a volume & environment
variable to enable forwarding your local ssh agent into the container.
As long as you've got an ssh agent running locally with your keys
enabled, this will allow you to e.g. push to a remote over ssh from
inside the container.
- It also mounts your local `.gitconfig` file into the container, so
your git configuration (i.e. name & email address) will also work inside
the container
- Finally, it adds githubs ssh keys to known_hosts in the development
target of the Dockerfile to avoid the prompt, which only works in
interactive environments and fails in any kind of script (like `yarn
main:submodules`).
ref https://linear.app/ghost/issue/ENG-2026/
This is being removed as the JobQueueManager is being removed. By
removing this, we can allow the queue to deplete itself before stripping
out that functionality.
- no implementation changes, just improving the JSDoc and comments to
make this more readable
- also switches some assertions in the tests to use the spy instance to
fix a typing issue
Ref https://linear.app/ghost/issue/PLG-337/create-cta-card-web-design
- The minimal layout version now changes to a column layout on mobile
- The sponsor label color now inherits the theme color so that it works
on any background color
- The spacing is adjusted for mobile
- due to how our test framework works, we end up running the boot
process many times
- each boot causes the Stripe service to re-add this handler
- that means we end up with like 100 handlers for `settings.edited`
- whilst it would be ideal if we could reset the event listener each
boot, the simplest thing for now is to just remove and re-add it
- I've pulled `stripeSettingsChanged` out of the init function because
`removeListener` only works on the specific instance of that function,
so it won't clean it up if it's a local function
- due to our testing framework, we run the boot over and over, but this
means we constantly add new xmlrpc listeners
- from profiling, this accounts for ~3% of the total test time, which is
minimal, but given it's an easy fix, this commit performs that
- ideally our test framework would reset the events between runs, or
keep them completely separate, but that's a while away yet without a
big refactor
ref AP-748
- The current structure of the AcitivyPub app makes it hard to scale and
extend, and it also lacks basic design heuristics. This PR adds a new header
and right sidebar for clear orientation and navigation.
no issue
- There is still some flaky behavior with the docker setup that seems to
be caused by the container not shutting down or cleaning up properly.
Sometimes `yarn dev` just hangs, or sometimes it only runs some of the
commands required. This seems to only happen when running `docker
compose up` after having previously run it and stopped it.
- Passing the `--force-recreate` option to docker compose recreates all
the containers, which seems to avoid this problem without incurring a
noticeable performance penalty.
no issue
- Port 4173 and 41730 are used to load a locally running version of
lexical into Ghost, and these ports were added to the Dockerfile and
compose.yml along with the ports for portal, admin, etc. However,
lexical runs _outside_ the container, so we don't really need to expose
these ports.
no issue
- This commit includes a few minor optimizations to the Dockerfile
- Removes `/var/lib/apt/lists` and runs `apt clean` to remove the apt
cache after each `apt install ...` command. This reduces the resulting
image size by about 100 MB
- Also moved the playwright install to after the `package.json` and
`yarn.lock` files are copied in, so we don't have to specifically pin
the version in the Dockerfile — it will instead just look at the version
specified in the `package.json` files.
no issue
- When running Ghost in docker compose, you need to set the database
connection, redis connection, and the server host to values that are
suitable for the environment.
- Previously this was done by automatically editing your
`config.local.json` file to update these parameters in the container's
entrypoint script. This made it somewhat annoying to switch from Docker
to local development because you have to remember to update your
`config.local.json` manually.
- This change adds a new static configuration file for running in docker
compose. The values set in this configuration file will override any of
the same keys in the `config.local.json` file, while still allowing the
use of `config.local.json` for any _other_ configuration keys. The
upshot of this is that you won't need to modify your `config.local.json`
file when switching back and forth between local and docker dev — you
can keep your local database, redis, etc configuration in
`config.local.json`, and they will simply be ignored when running in
docker.
- This also removes the script that updates your `config.local.json`
file from the entrypoint, since we no longer need to perform this step.
no issue
- Renamed the docker compose profile for Ghost from `full` to `ghost`,
because it is clearer and makes more sense this way
- Also added an explicit `GHOST_DEV_IS_DOCKER` environment variable to
the `compose.yml` file to make checking if running in docker less
fragile and more explicit.
no issue
- This fails when running the Ghost container alone (i.e. for unit
tests) since there is no database available.
- The database will be initialized when running `yarn dev` for the first
time anyhow, so this isn't really necessary here.
no issue
- Doing the submodule update in the Dockerfile doesn't work if you cloned Ghost via ssh,
because the Docker container doesn't have access to your ssh keys.
no issue
This commit is a fairly significant rework of the docker development
setup, with the goal of improving the performance of the Ghost container
and generally improving the DX of the docker development setup.
**Enumerating `node_modules` directories as volume exclusions for the
bind mount**
- The main source of performance overhead with the docker setup is the
bind mount, which mounts all the code into the container. Especially on
macOS, the bind mount introduces quite a bit of file system overhead.
Currently the bind mount includes all the `node_modules` directories,
which adds up to many thousands of files in the bind mount. This change
attempts to reduce this overhead by mounting in named volumes for each
`node_modules` directory in the monorepo, which effectively tells the
main bind mount to ignore these directories, thus reducing the
filesystem overhead.
**Removing the `yarn install` from the entrypoint script and the `yarn
setup:docker` step**
- This change has two more benefits: since all the `node_modules` are
installed into named volumes, we can now be confident that our local
`node_modules` won't conflict with the `node_modules` in the container.
Therefore, we can install `node_modules` at build time in the Dockerfile
instead of in the entrypoint script, and we can also get rid of the
`yarn setup:docker` setup command, which nukes your local `node_modules`
before building the container.
**Build optimization - enumerating all `package.json` files**
- In a typical node app's Dockerfile, you copy the `package.json` and
`yarn.lock` files into the container first, then run `yarn install`,
then copy the rest of the code in. This takes advantage of Docker's
build cache to avoid having to reinstall `node_modules` on every build
if you've only changed code files, but not any dependencies.
- This is more challenging to do with the monorepo since we have many
`package.json` files. This commit brute forces our way through that by
copying every single `package.json` file in the Dockerfile before
installing dependencies and copying the code in. The result is that
Docker builds become much faster, after the initial full build.
**Additional scripts**
- Since enumerating all the `node_modules` directories and
`package.json` files is tedious to do manually, this commit also
includes two new scripts to automate this work. Ideally we'll eventually
integrate these scripts with Slimer, so when you add a new package to
the monorepo it automatically updates the `Dockerfile` and
`compose.yml`, but for now we can manage that ourselves since it's a
fairly rare occurrence.
no issue
- No real code changes
- Added README.md to stripe service as a starting point to document some of the more complex stripe related flows
- Added typedefs and docstrings to most of the Stripe service code for improved type safety and clarity
I have completed the translation of the Slovak language for the missing
phrases and edited some of them for grammatical correctness, word order
and context.
---------
Co-authored-by: Peter Gonda <pipo@pipoline.com>
Co-authored-by: Cathy Sarisky <42299862+cathysarisky@users.noreply.github.com>
- this is slightly faster than the sync methods and brings it inline
with that we have in the E2E framework, so it'll be easier to switch
down the line
- this test needs to pass some time to make it work, and the current
method awaits a setTimeout, which means we're waiting in the real
world for it to finish
- we can just replace this with sinon to fake the clock jumping a second
ahead of time
- this saves 2s in tests
- I've discovered that 20% of our E2E test time is spent waiting around
because we're throttling the requests to the Stripe API
- this makes sense in production because we have a requests/second limit
but for testing it just makes things slower
- to avoid this, this commit increases the throttling thresholds if we
detect we're in the testing environment
- the 10_000 limits are somewhat arbitrary, I tried Infinity but
LeakyBucket doesn't like that number, and this achieves what I want
- just some gentle refactors to move them to the new E2E test suite
- includes the new snapshots with the necessary matchers in place to
ensure the tests aren't hardcoded with specific IDs
fix https://linear.app/ghost/issue/ENG-1803/remove-url-cache-code
- this feature is experimental and was designed in order to speed up our
URL service init by storing a cache of the object
- however, it was never really finished and we've had a few bugs with it
- to avoid further issues, this commit removes it
- along the way, I've discovered that our tests REQUIRE the URL cache in
order to pass, which I thought was weird. Turns out it's because they
incorrectly set cache values into the URL service that are outdated, which
meant our snapshots are wrong
- this is why several of the snapshots have been updated, because URLs
have changed
- sadly this commit touches so many files, but this feature was really
spread around the codebase
- this technically removes test util API support for a split
backend/frontend but that wasn't properly finished. I'd like to get
that working, but first we need to unpick the mess we've got ourselves
into
ref https://linear.app/ghost/issue/ENG-2014/add-node-22-support-to-koenig
- we're working on adding support for Node 22 and part of that involves
bumping the Koenig packages because they have version ranges we need
updating in Ghost
- this commit bumps the packages so they have support for Node 22
ref BAE-397
Moved the hCaptcha component outside of the page level, since it was
complicating the logic within the pages with no good reason.
The hCaptcha component is now attached to the pop-up modal. Since it's
invisible, this doesn't impact layout anyway, but means that any action
can trigger Captcha to run, and use the result within that same action.
This simplifies the flow by having the action itself confirm that
Captcha is enabled, then grabbing the token by running either a
challenge (for self-hosters) or using their enterprise heuristics system
(for Ghost Pro).
This also fixes issues where sites with multiple tiers wouldn't work
with Captcha, since the page had changed and the hCaptcha component was
unloaded.
ref AP-744
- NewPostModal component is only used in Feed
- For that reason, this should be placed with the Feed component itself
in view directory
- Inbox and Feed components are still in a single file at the moment as
the separation isn't in the context of file structure updates
- Once componets are separated into smaller ones, Feed directory will be
populated with the corresponding files
ref AP-744
- moved Notifications, Profile, and Search components to views directory
as they're all page components
- updated the imports with path aliases
ref AP-744
- according to the new file structure, moved the Inbox component to
views directory as it's a page
- added path aliases to minimize the changes for imports
ref
https://linear.app/ghost/issue/ENG-2002/remove-linkclicktracking-config-in-lieu-of
- This was introduced as a way to disable `linkClickTracking` as a last
resort for sites that weren't handling email link redirects well, but
since we've introduced a more comprehensive config in
`hostSettings:settingsOverrides` that allows us to accomplish the same
thing.
- Now this `linkClickTracking` config isn't in use or really doing
anything, so this commit cleans it up.
