refs https://github.com/TryGhost/Toolbox/issues/152
- This refactor fixes the rest of authentication test suite and outlines the state setup steps that are absolutely needed for tests that rely on any db state. It's still counterintuitive why the fixture initialization has to be called and in that specific order, so next will be refactoring in this area to simplify the initi code
refs https://github.com/TryGhost/Toolbox/issues/152
- Passing around plain options object tends to become quite unreadable long term. While these new utils are being shaped up it's still easy to change interface and introduce new parameters with time as needed.
refs https://github.com/TryGhost/Toolbox/issues/152
refs 5bea089dfe
refs 16ad5f73c4
- The refactor is heavily inspired by the referenced commit from @ErisDS
- This refactor is meant to serve as a template for further refactors and unification of the abstraction over "request". Should allow us to be more agnostic towards the library that's powering the request thing. For example, mock-express style of request handling could substitute or get substututed easily if all tests are behind similar "get(Authenticated)Agent" interface
refs https://github.com/TryGhost/Toolbox/issues/152
- This is a continuation of an experiment to switch over to serverless boot in regression tests. Just a proofe of concept that authentication scenarios would also work and the expectations don't collapse unexpectedly.
- Nothing too crazy so far, easy and straight forward substitution of the config "url" with an express app passed to the supertest agent
refs https://github.com/TryGhost/Toolbox/issues/152
- Being able to set up test suites without blocking a port opens a door to new ways to run tests - for example this has been one of the blockers for running mocha tests in parallel
- Additional benefit is lighter statrup, which reducec the test execution time slightly. Doesn't seem like much but these things stack up!
no issue
- we've seen some instances of SQLite saying "database disk image is malformed"
- I think this happens because we copy the file whilst we are still connected to
the old DB
- this commit destroys the connection before copying the clean file to the live
DB file
refs https://github.com/TryGhost/Toolbox/issues/136
- we nuke and reinitialize the DB many times between tests
- this forms a good portion of the time taken and we shouldn't be spending
so much time on just resetting the DB back to a known state
- this commit switches out the knex-migrator reset + init calls for SQLite to
a function which keeps a clean copy of the DB and copies the file back
when we "reset"
- for MySQL, existing functionality is kept
- this massively speeds up tests because it saves ~700ms+ for every reset
- whilst this change seems to work, it's just the start so there's a
lot more refactoring needed. this change is currently gated to CI until
it's deemed safe/sane enough to run on local machines without blowing
up
refs https://github.com/TryGhost/Toolbox/issues/152
- Have skimmed through the test suites in hopes to find some quick performance wins to bring the runtime speed closer to the one in "main". Haven't been successful to identify major wins, cleaned up a couple of small bits.
- We'll have to live with a tradeoff between maintainability/unified boot VS cost of mainitaining a fake boot process. Imo extra couple seconds of runtime is worth it.
refs 3c7a8dead4
- The tests needed adjustments with the native boot mechanism.
- The number of returned posts changed in the test resutls because duing native boot we also insert fixtures which add to the number of initial posts
- The vhost regression suite is still failing and I had no strength to figure out why. The redirect it fails with makes no sense, the clue here is that the test doesn't fail when running in isolation, so probably has to do with some leftover overrides from the previous test cases.
refs 3c7a8dead4
- The boot process has been using an asyc method to load the routes file, which is the case now for these tests since the switch to raw boot method instead of mimicking it manually
refs 3c7a8dead4
- Simplifies the state initialization code significantly and reuses native boot mechanism instead of mimicking it (it was a headache to maintain with all the internal services moving around)
refs ec28478435
- the method signature changed in
9fa8800b9d
and the usage wasn't updated in Ghost
- this commit updates the tests to reflect this internal change
refs https://github.com/TryGhost/Team/issues/960
- Character like "%%" or "%80" would crash our current url escaping behavior. We consider they aren't valid URLs as the percentages haven't been properly escaped.
refs https://github.com/TryGhost/Team/issues/1200
- The leading/trailing whitespaces are trimmed by `new URL()` but are considered invalid in metascraper. Trimming solves this edge case.
refs: TryGhost/Toolbox#147
* Replaces all references to isIgnitionError with isGhostError
* Switches use of GhostError to InternalServerError - as GhostError is no longer public
There are places where InternalServerError is not the valid error, and new errors should be added to the @tryghost/errors package to ensure that we can use semantically correct errors in those cases.
refs https://github.com/TryGhost/Team/issues/1237#issuecomment-981770688
- API key names for external services now follow a standard pattern
- top-level key of the service name
- public/private and read/write perms inside the name, eg. `publicReadOnlyApiKey`
- updated test to match expected API key name
closes https://github.com/TryGhost/Toolbox/issues/148
- These regression tests introduce very little additional value in exchange for an expensive time to run them. Because we mostly care about stable support for the latest stable API version, the older API version test for "internal" API can go away
- In case there are bugs found we do care about in the v2/v3 APIs we can always revert some of these tests.
https://github.com/TryGhost/Toolbox/issues/140
- This test was bloating the regression/site suite and was using hacks (calling the Admin API) to create a custom redirects state
- It suits way better in e2e frontend test suite with less hacky approach to start the Ghsot instance - using custom routes file path to initialize the instance
https://github.com/TryGhost/Toolbox/issues/140
- Allows to fully start an instance with a custom routes.yaml file without a need to do workarounds - e.g. we used to call an internal Admin API from a regression test suite, which is not a good practice
- Providing "routesFilePath" to startGhost method copies that file to the default settings location and start the instance with that configuration. No need to do API calls or check if the routing service "isFinished"
refs https://github.com/TryGhost/Toolbox/issues/140
- Same/similar RSS tests were present in regression and e2e tests suites. It made sense to move missing cases from regression to e2e. This saves us time bootstraping db state in multiple places and keeps all test cases regarding single feature in same place
refs https://github.com/TryGhost/Toolbox/issues/139
- With ec2aed5ce8 the DynamiRedirectsManager has reached 100% test coverage and most of the tests present in the removed regression suite have been ported to unit tests
- No need to keep slow tests around! :)
refs https://github.com/TryGhost/Toolbox/issues/139
- The regression test suite for redirects functionality for way too big. And each restart was causing massive overhead. It's enough to have a single exhaustive test using multiple input files
- The tests testing API endpoints should've been e2e tests to start with
- The rest is covered in the unit tests for redirects api service
refs https://github.com/TryGhost/Toolbox/issues/139
- Having tight coupling with backup file path calculation for redirects makes it extremely hard to test. In addition, having it injected will make it easier to swap this dependency to the mechanism similar to one used for routes files
refs 91efa4605c
- Referenced commit introduced a double json-stringification to uploaded redirects.json files.
- The endpoint has no stability index of any sort and is meant to be dropped in Ghost v5. It's best to rework the redirects to the yaml format as descirbe here - https://ghost.org/docs/tutorials/implementing-redirects/#file-structure
- moving this middleware because we're about to add a second piece of middleware
- it's easier to see what we have when each middleware is in its own file rather than in one big middleware.js file
no issue
- this commit adds a counter for the number of boots we do in tests
- which therefore allows us to calculate the average boot time we
experience
- only useful for debugging test performance
refs https://github.com/TryGhost/Toolbox/issues/139
- The v2 and v3 redirects APIs are unofficial and should not be used by anyone in production. There's no good reason to maintain expensive to run test suites for old unofficial APIs.
- The test cases in canary suite covers the functionality of redirects enough to be sure they work as expeted
refs https://github.com/TryGhost/Toolbox/issues/138
- These tests still rely on the frontend to be present. Needs further investigation to remove "frontend: true" flag - it slows down test runs!
refs https://github.com/TryGhost/Toolbox/issues/138
- Final batch of the refactor to async/await syntax. Doing these refactors before modifying "testUtils.startGhost" everywhere to boot only with the backend
refs https://github.com/TryGhost/Toolbox/issues/138
- This is a continuation of a bigger refactor to use async/await syntax before migrating "startGhost" methods to only use backend boot
- Removed a little bit of dead code (like admin user creation) which should speed up test execution too!
- Refactored user variables to be declared closer to their usecases instead of being high up in a global scope - variables shoul not live that far apart from the code that uses them
refs https://github.com/TryGhost/Toolbox/issues/138
- First batch of the refactor to async/await syntax. Next one will cover the rest. Doing these refactors before modifying "testUtils.startGhost" everywhere to boot only with the backend
refs https://github.com/TryGhost/Toolbox/issues/138
- There is no good reason to keep this extra variable around just call "stop" in couple very specific cases. Even for those cases, there's `testUtils.stopGhost` method which achieves the same without additional variable to track.
