0
Fork 0
mirror of https://github.com/withastro/astro.git synced 2024-12-16 21:46:22 -05:00
Commit graph

50 commits

Author SHA1 Message Date
Ben Holmes
befbda7fa3 [ci] format 2024-05-03 15:09:39 +00:00
Ben Holmes
44bafa989a
[db] Fix duplicate calls to recreate tables on startup (#10919)
* fix: move recreateTables() to integration hooks

* feat: recreate and seed at load, not in virtual runtime

* feat: eager build db on startup and seed file change

* fix: respect database_file in dbUrl

* chore: remove duplicate recreateTables call

* chore: remove now self-explanatory comments

* fix: remove invalidateModule call for eager loading

* feat: respect seed package paths

* fix: remove duplicate recreateTables() call

* refactor: move recreateTables() to vite-plugin-db

* refactor: move queries.ts from runtime/ to core/

* fix: update test import to core/queries

* refactor: move executeSeedFile to vite-plugin-db

* refactor: extract seeding and recreating to helper fns

* chore: changeset

* chore: revert connectToStudio refactor

* wip: log db url

* fix(test): normalize astro_database_file flag for windows

* Revert "wip: log db url"

This reverts commit 558e2de67a09a611377929b625127c649b8504d6.

* Revert "Revert "wip: log db url""

This reverts commit ffd004e00dff485b7bc5ddde0278dde6ff058b9e.

* fix: correctly resolve relative paths with unit test

* chore: remove unused dbDirPath

Co-authored-by: Chris Swithinbank <swithinbank@gmail.com>

* chore: remove unused import

Co-authored-by: Chris Swithinbank <swithinbank@gmail.com>

* chore: remove unused type

Co-authored-by: Chris Swithinbank <swithinbank@gmail.com>

* fix: remove bad import

* [db] Load seed files with vite dev server (#10941)

* feat: load seed files with full vite dev server

* chore: remove unused export

---------

Co-authored-by: Chris Swithinbank <swithinbank@gmail.com>
2024-05-03 11:08:50 -04:00
Emanuele Stoppa
347bdfe550 [ci] format 2024-04-17 08:38:53 +00:00
Matthew Phillips
2d2ff58bb9
Fix db seeding when srcDir is root (#10720) 2024-04-09 09:11:32 -04:00
Ben Holmes
f395ebcc08
refactor: rename "config" file to "virtual" (#10638)
* refactor: rename "config" file to "virtual"

* chore: changeset

* fix(test): config import
2024-04-02 18:13:18 -04:00
Matthew Phillips
5f5176b244 [ci] format 2024-04-02 20:08:38 +00:00
Matthew Phillips
b1eda3dc5c
Fix building static sites with Astro DB (#10655)
* Fix building static sites with Astro DB

* Adding a changeset

* Try a different port range
2024-04-02 16:07:18 -04:00
Matthew Phillips
ca29764912 [ci] format 2024-04-01 15:41:03 +00:00
Matthew Phillips
157392ee44
Make ASTRO_DATABASE_FILE work with file paths (#10631)
* Make ASTRO_DATABASE_FILE work with file paths

* Use pathToFileURL
2024-04-01 11:39:49 -04:00
Ben Holmes
ed1031ba29
db: Rework index config with generated index names (#10589)
* feat: add indexes array config with name gen

* fix: add _idx suffix, remove name from output

* feat(test): new index config

* chore: remove unused type

* chore: changeset

* chore: add sort() for consistent names

* feat(test): consistent column ordering

* feat(test): ensure no queries when migrating legacy to new
2024-03-28 14:09:09 -04:00
Matthew Phillips
62a1d6df69
For --force-reset use conditional drop table query (#10587) 2024-03-27 16:30:59 -04:00
Ben Holmes
d43bcbeec3 [ci] format 2024-03-27 19:52:24 +00:00
Ben Holmes
713abb2998
chore(db): Rename all collection usage to tables (#10460)
* chore: rename `collection` field to `table`

* chore: remove deprecated ResolvedCollectionConfig type (only used by studio)

* chore: collection -> table in migration-queries

* chore: update tests

* chore: last renames

* chore: bump migration version

* chore: remove deprecated collection field

* chore: droptablequeries

* chore(test): collection -> tables

* chore: revert collection -> table change on migration file

* chore: revert migration version change

* chore: changeset
2024-03-27 15:51:26 -04:00
Matthew Phillips
016b4b7b13 [ci] format 2024-03-27 19:21:30 +00:00
Matthew Phillips
f5df12cfeb
Provide guidance when --remote is omitted (#10579)
* Provide guidance when --remote is omitted

* Only restrict to server mode

* Use an AstroError

* Update code
2024-03-27 15:20:11 -04:00
Ben Holmes
b5a80405b9
Fix db.run() type signature in production (#10566)
* deps: bump drizzle to latest patch

* chore: changeset

* feat(test): db.run output
2024-03-26 13:43:04 -04:00
Matthew Phillips
44dacedb3a [ci] format 2024-03-22 15:53:54 +00:00
Matthew Phillips
6576f5d458
Require that ASTRO_STUDIO_REMOTE_DB_URL is defined at runtime (#10533)
* Require that ASTRO_STUDIO_REMOTE_DB_URL is defined at runtime

* Add changeset

* Fix build
2024-03-22 11:52:38 -04:00
Ben Holmes
7ae1392d66 [ci] format 2024-03-22 12:29:14 +00:00
Ben Holmes
95e3cc84e8
db: dev --remote test fixture (#10527)
* feat: scaffold basic proxy

* feat: setupDbServer util

* feat: ASTRO_INTERNAL_TEST_REMOTE bypass

* feat: basic --remote test

* chore: add port incrementer
2024-03-22 08:28:12 -04:00
Matthew Phillips
297571eb98 [ci] format 2024-03-20 18:43:56 +00:00
Matthew Phillips
980020c5e0
Ensure --force-reset drops previous tables (#10506)
* Ensure --force-reset drops previous tables

* Remove unused import

* Do suggestions
2024-03-20 14:42:41 -04:00
Ben Holmes
98b982bdfc [ci] format 2024-03-20 11:29:14 +00:00
Ben Holmes
f0fc78c873
db: expose isDbError() utility (#10498)
* feat: expose isDbError

* test: foreign key constraint error detection

* fix(test); use isDbError

* chore: changeset
2024-03-20 07:27:48 -04:00
James Garbutt
2ebcf94d0a
chore(db): Add missing github-slugger dependency & tests (#10405)
* chore: add missing github-slugger dependency

* test: add column-queries data loss tests

* chore: remove unused vars

* test: assert length rather than message content
2024-03-19 17:10:33 +00:00
Ben Holmes
156c094c3e [ci] format 2024-03-18 11:56:49 +00:00
Ben Holmes
219c49473f
fix type error in seed stub on initial setup (#10457)
* fix: move db type to base types for seed stub

* chore: changeset

* fix(test): bad NOW import
2024-03-18 07:55:39 -04:00
Arsh
4e1a9c5299 [ci] format 2024-03-13 13:50:39 +00:00
Arsh
96c8bca19a
fix(db): validate column type before column schema (#10409)
* fix(db): validate column type before column schema

* add changeset

* Add test for text foreign keys

---------

Co-authored-by: Matthew Phillips <matthew@skypack.dev>
2024-03-13 09:49:28 -04:00
Emanuele Stoppa
dfd146f591
Revert "[ci] format" (#10417)
* Revert "[ci] format"

This reverts commit 1863727215.

* pin dependency

* update lock
2024-03-13 09:55:24 +00:00
Houston (Bot)
1863727215 [ci] format 2024-03-13 05:53:52 +00:00
Ben Holmes
41ca94e513
chore: remove drop table check for added table (#10396)
* chore: remove drop table check for added table

* chore: remove drop table from test

* chore: changeset
2024-03-12 07:30:45 -04:00
Darius
f76dcb769f
export DB types (#10374)
* export DB types

* refactor: move schemas to separate file

* chore: changeset

* chore: add typesVersions

---------

Co-authored-by: bholmesdev <hey@bholmes.dev>
2024-03-11 18:06:52 -04:00
Chris Swithinbank
38abae47b5
Fix db for projects without a seed file or with integrations (#10385)
* Add fixture and tests for integration with no user db config

* Create database tables when integration seed files load

* Defer running seed function until all modules are loaded

* Update package name

* Add test & fixture for a project with no seed file

* `recreateTables()` from `seedLocal()` (and move it out of main runtime module)

* Fix typo after rebase

* FREEZE DON’T MOVE

* Move `seedLocal` export back to runtime

* Simplify seed file normalization

* Clean up test files

* Add build tests for no-seed and integration-only fixtures

* Add changeset
2024-03-11 20:07:53 +01:00
Chris Swithinbank
8cceab587d
Use static imports for integration seed files in astro:db (#10381)
* Use static imports for integration seed files in `astro:db`

* Add changeset

* Add build test for integrations fixture

* chore: comment on top-level seed imports

* fix: random db id for tests

* fix: set db id from build before

* wip: remove reset on env variable

* wip: move random db id env to top of test file

* refactor: remove unlink from db startup

* chore: remove random db id completely??

---------

Co-authored-by: bholmesdev <hey@bholmes.dev>
2024-03-11 09:51:15 -04:00
Chris Swithinbank
3f27e09628
Rename defineDB to defineDb (#10364)
* Rename `defineDB` to `defineDb`

* Add changeset

* Improve changeset
2024-03-07 19:21:03 -06:00
Ben Holmes
988aad6705
Add batch support (#10361)
* deps: drizzle preview

* feat: db.batch and method run handling

* refactor: use db.batch in test fixture

* deps: bump to drizzle 0.29.5

* chore: changeset

* fix: unpin drizzle version

* fix: db execute should uh... execute
2024-03-07 14:49:46 -05:00
Chris Swithinbank
52fba64cb2 [ci] format 2024-03-07 19:20:45 +00:00
Chris Swithinbank
2e4958c8a7
Add integrations API for db config/seed files (#10321)
* Add integrations API for adding db config/seed files

* Fix seeding when user seed file is present

* Add basic test and fixture for integrations API

* Freeze that lockfile

* Test to see if this is a Windows fix

* Don’t import.meta.glob integration seed files

* Make integration seed files export a default function

* style: rejiggle

* Fix temporary file conflicts

* Remove changes to Astro’s core types, type utility method instead

* Use `astro:db` instead of `@astrojs/db`

* Revert unnecessarily cautious temporary path name

This reverts commit ef2156e41b.

* Add changeset

* Fix entrypoints and `asDrizzleTable` usage in changeset

* Getting Nate in on the co-author action

Co-authored-by: Nate Moore <natemoo-re@users.noreply.github.com>

* Fix user seed file in integrations fixture

* Update `seedLocal()` after merge

* Provide empty `seedFiles` array in `db execute`

---------

Co-authored-by: Nate Moore <natemoo-re@users.noreply.github.com>
2024-03-07 20:19:17 +01:00
Ben Holmes
06fe94e29d
Add --remote flag for remote connection (#10352)
* feat: check for --remote

* chore: remove bad ticketing example cols

* fix: get seed file working with build

* Revert "fix: get seed file working with build"

This reverts commit 92830a106164b0997c820a3e0bf2a582018084a0.

* fix: seed from build instead of runtime

* refactor: move recreateTables out of runtime

* Revert "refactor: move recreateTables out of runtime"

This reverts commit d01a802ad7915fabc4c4ac35b2d907eae0538d95.

* fix: in-memory db for test fixture

* chore: changeset

* refactor: generate random db name instead

* refactor: use yargs-parser for flag

* chore: remove in-memory db logi

* refactor: rename random id flag for clarity

* feat: support --remote in dev

* feat: support --remote on shell

* refactor: inline db client

* feat: support --remote on db execute

* chore: stray console log

* chore: remove recreateTables from runtime

* chore: update seeding for new signature

* chore: remove unused error imports
2024-03-07 13:38:43 -05:00
Chris Swithinbank
bad9b583a2
Update seed to use export default function() instead of top-level await (#10334)
* Update fixtures seed files to export default function

* Call default export when running seed files

* Add changeset
2024-03-07 13:04:16 -05:00
Matthew Phillips
652bd497dc [ci] format 2024-03-07 16:38:45 +00:00
Matthew Phillips
9f422e9bd3
db: rename experimentalVersion to version (#10348)
* db: rename experimentalVersion to version

* Fix tests

* Update .changeset/fluffy-bobcats-arrive.md

Co-authored-by: Emanuele Stoppa <my.burning@gmail.com>

---------

Co-authored-by: Emanuele Stoppa <my.burning@gmail.com>
2024-03-07 11:37:36 -05:00
Matthew Phillips
a2e9b2b936
Fixes loading .ts configs in astro:db (#10342) 2024-03-06 09:50:02 -05:00
Fred K. Schott
24bc169070
add back data loss confirmation handling (#10330) 2024-03-05 14:44:19 -05:00
Fred K. Schott
51393a729b [ci] format 2024-03-04 10:51:57 +00:00
Fred K. Schott
93ec9e264a
add new migrations system (#10312) 2024-03-04 02:50:58 -08:00
Ben Holmes
25fe5bd040 [ci] format 2024-03-02 00:31:00 +00:00
Ben Holmes
3488be9b59
finalize WIP API (#10280)
* feat: no more readable / writable

* fix: table typegen

* wip: move data seeding

* chore: add scripts to basics

* feat: data() -> seed file

* refactor: ensure precedence of file name

* feat: db execute command

* fix: test imports

* chore: remove old readable error tests

* feat: support local db with `db execute`

* refactor: remove integrations from test for now

* chore: stray comment

* chore: remove `table` config object

* feat: `db.batch`!

* refactor: move migrations/ inside db/

* fix: move ticketing-example to seed file

* fix: disable foreign keys when recreating tables

* refactor: standardize migrations dir

* feat: move to db/config.ts

* feat: file watching for db/config.ts dependencies

* feat: remove unsafeDisableStudio

* chroe: remove bad import

* feat: parse config.ts from cli

* chore: remove async from localDatabaseClient

* fix: update recipes config and seed

* chore: update unit tests

* chore: update tests to dev server

* refactor: collectionToTable -> asDrizzleTable

* chore: tidy up collection -> table error states

* refactor: regexp -> endsWith

* feat: pretty error inserting into table

* refactor: try/catch -> catch()

* feat: expose utils for integration seed files

* fix: add config import to db client modules

* fix: just use generic "seeding database" error

* chore: remove unused link args

* fix: migration queries im,port

* chore: remove irrelevant glob/ example

* feat: format migration file path

* feat: support all config file names

* chore: remove db.batch() for now

* chore: remove `db` object

* core: remove unused integration file

* chore: changeset

* fix: foreign key empty error message

* chore: remove old TODO

* fix: bad context reference

* refactor: seedDev -> seedLocal

* wip: throw some console logs at github

* wip: avoid seeding astro:db imported by seed file

* wip: use anything in db/

* refactor: only seed when loaded within srcDir

* refactor: avoid resolution when not seeding

* chore: remove logs

* refactor: seed within create local db client

* refactor: use normalizePath

* wip: logs

* wip: logs

* refactor: early return

* chore: more logs

* refactor: no batch

* fix: use beforeAll

* refactor: move all tests to base block

* wip: log dev server starting

* chore: remove logs

* wip: demo ready

* chore: remove duplicate recreateTables() call

* Revert "wip: demo ready"

This reverts commit 37585ce5cb.

* refactor: beforeEach to isolate dev servers

* chore: remove useBundledDbUrl

* refactor: naming and seed scope

* chore: remove stray console logs

* wip: fix windows file import

* wip: try fileURLToPath

* Revert "wip: try fileURLToPath"

This reverts commit 46fd65d61a.

* Revert "wip: fix windows file import"

This reverts commit 1a669ea646.

* refactor: dir -> directory

* refactor: move execute file to cli

* refactor: remove seed.dev convention

* wip: attempt fileURLToPath

* wip: debug the file exists

* fix: use mjs??

* chore: remove duplicate seedLocal

* chore: remove log check

* refactor: use in memory db for tests

* chore: clean up test comment

* fix: avoid file writes for db setup on in memory db

* chore: bump db changeset to minor

---------

Co-authored-by: Nate Moore <nate@astro.build>
2024-03-01 19:29:55 -05:00
Matthew Phillips
31a9f8469c
Introduce the db integration (prerelease) (#10201)
* Initial DB migration code

* chore: update package.json

* chore: update lockfile

* feat: add db as top-level config value

* Small package change

* Add a very basic test

* deps: remove unused

* chore: astro/db scripts, exports, deps

* chore: set tsconfig to monorepo defaults

* feat: MVP to use libsql in dev

* fix: test fixture paths

* fix: test file paths

* chore: remove CLI tests

* fix: add astro-scripts to db

* fix: package.json merge

* fix: load seed file separately for build

* feat: create db on filesystem for build

* fix: ul test. It passes now!

* Squashed commit of the following:

commit acdddd728c56f25e42975db7f367ab8a998e8c41
Author: Princesseuh <3019731+Princesseuh@users.noreply.github.com>
Date:   Wed Jan 10 14:06:16 2024 -0500

    fix: proper type augment for the config

commit b41ca9aacf291d1e5f0a27b6d6339ce4fc608ec3
Author: Nate Moore <nate@astro.build>
Date:   Tue Jan 9 14:33:42 2024 -0600

    wip: type augmentation

* feat: data() fn with basic type safety

* wip: update from seed file to data()

* fix: bad collectionSchema data type

* refactor: move dev to use filesystem to run seed at right time

* chore: remove seed file logic

* docs: add basic README

* CLI sync command

* Runtime checking of writable

* docs: add join example

* Implement defineWritableCollection

* feat: use studio connection for studio builds

* fix: add defineWritableCollection export

* refactor: use getTableName() util

* feat(db): add support for pass-through `astro db` command

* chore: add error map

* fix: add drizzle import

* refactor: studio -> db cli

* feat: add ticketing example

* fix: bad types in astro config

* fix: remove invalid `data()` on writable collection

* fix: vite warning on load-astro-config

* wip: add seeding for readable collections (nonstable ids!)

* merge migration work into branch

* cleanup migration commands

* migrate seed data to new db push command

* add migrations table to db

* fix remote db bugs

* fix: warn writable collections when studio false

* chore: delete README contents (now on Notion)

* chore: remove blank README

* chore: add dev dependency on db

* Removed unused deps

* 0.1.0

* Add config-augment.d.ts to published artifacts"

* Fixes merge issues with main

* fix: support promise response from data()

* feat: basic glob fixture

* Add a main field

* Give a help message when no db command is provided

* feat: `db push --token` for GitHub CI secrets

* fix getPackage for db package

* 0.1.2

* wip: build a table type

* chore: update lockfile

* chore: temporarily remove `Table` type

* feat: better Table object for type inference

* format

* add primaryKey support

* improve snapshot parsing support

* cleanup primary key support, db push

* add simple db shell

* cleanup old copy paste code

* feat: scrappy global data() prototype

* feat(test): recipes example

* fix: use Extract to narrow keyof to strings

* 0.1.3

* Create a runtime version of createRemoteDatabaseClient

* 0.1.4

* Grab the dbUrl from the environment

* 0.1.5

* Expose the database to the build output

* 0.1.6

* 0.1.7

* 0.1.15

* wip: data() -> set() concept

* fix: just infer insert keys for now

* refactor: rewrite to injected set() function

* deps: chokidar, drizzle

* feat: glob support with { db, table } signature

* chore: move basics to new data set

* refactor: set -> seed

* feat: expose Model.table

* refactor: clean up types

* feat: migrations now working!

* upgrade @libsql/client

* format

* expose relevant types

* 0.1.16

* feat: config design

* feat: add indexes from collectionToTable

* feat: add indexes to setupDbTables

* fix: remove unique constraint on recipeId

* Use an import statement to grab the database file

* 0.1.17

* Remove unused import

* Rename to ?fileurl

* 0.1.18

* feat: index migrations

* move migration logic to turso, add back sync support

* feat: add queries unit tests and fix related bugs

* refactor: move field queries to field-queries.test

* feat: index query tests

* refactor: reorganize the rats nest of files

* Make the DB_URL be root relative

* Upgrade to latest libsql

* 0.1.19

* 0.1.20

* Make work in webcontainer

* 0.1.22

* Remove content database from the static build

* 0.1.23

* chore: remove `optional: true` from pk

* chore: disable console linting for CLI

* fix: remove `id` column from Table type

* chore: remove `AstroId` type

* fix(ex): add `id` col to ticketing

* 0.2.0

* 0.2.1

* add keywords

* 0.2.2

* feat: API shape

* feat: FINALLY collection and name attached

* refactor: move to arrow function signature

* fix: foreignKeys references signature

* chore: unused imports

* feat: add foreignkeys to create table

* chore: lint

* chore: enable foreign keys in local mode only

* refactor: objShallowEqual -> deep diff

* fix: correct `hasDefault` inference

* fix: correct type Config reference

* fix: respect primaryKey from hasDefault

* fix: mark data keys as optional until we have type inference

* improve conflict and dataloss handling

- moved prompts to db push
- moved prompt logic out of lower-level functions
- improved logic overall
- improved user-facing prompt messages

* improve error messaging around studio config missing

* make it more clear when remove vs. local db is in use

* fix bug in prompt logic

* feat: better field.x() types

* feat: better seed() types

* chore: remove `as any` on seed values

* feat: good enough return type on seed :)

* feat: defineData()

* fix: add back promptResponse injection

* fix: use schema.parse to resolve dates

* fix: correctly respect primary key on INSERT INTO

* add short-lived db tokens

* add help output

* add better token error logging

* fix studio tests

* add shortcut link command from studio web ui

* Add support for SQL defaults

You can now use sql`CURRENT_TIMESTAMP`, `NOW`, and a couple of other
helpers, to set defaults.

* chore: todo

* feat: ignore `optional` and `default` when pk is present

* refactor: type `false` instead of type `never`

* feat: prevent `optional` on text pk

* fix db URL import for windows

* fix: add back textField multiline

* fix: remove explicit AUTOINCREMENT on int pk

* feat(db-cli): clean up CLI logging, support --json flag for `astro db verify`, extract shared logic to a utility

* prepare to run seed on all db push commands

* chore: expose setMeta for unit testing

* feat(test): reference add and remove tests

* feat: add references checks to migratiosn

* feat: remove useForeignKey checks

* feat: add pragma when pushing migrations

* feat(test): foreignKeys

* fix: transform collection config to be JSON serializable

* refactor: _setMeta -> preprocess for `table`

* refactor: reference tests

* chore: remove console log

* fix: handle serialized SQL object correctly

* refactor: store raw sql instead

* seed on every push

* Move field schema only a `schema` object

* Fix references test

* 0.3.0

* add default URLs to db package

* 0.3.1

* Fix input types

* fix: primaryKey type check

* 0.3.2

* fix: respect default in table types

* fix: avoid dropping tables on production seed

* fix: escape name on drop table

* feat: allow verify to mock migration file

* Handle unauthorized linking

* Fix verbiage of unauthorized link warning

* Add some color to the unauthorized message

* 0.3.3

* Improve the unauthorized error output

* 0.3.4

* fix: better error message

* Seed the Themes in build too

* Push skipped test

* chore: remove dead isJsonSerializable check

* fix: use `dateType` for dates (oops)

* refactor: clarify date coerce comment

* refactor: remove unused coerce

* chore: unskip date test

* feat: seed -> seedReturning

* refactor: throw when seeding writable in prod

* Add unsafeWritable option

* refactor: use FieldsConfig for Table generic

* chore: lint

* fix: use z.input for AstroConfigWithDB type

* fix: add defaults for boolean config options

* Support new CLI command structure

* Allow writable in the tests

* fix: handle defaults for safe type changes

* refactor: avoid selecting ['schema'] on input types

* 0.3.5

* Rename field->column, collection->table

* Rename collections->tables

* rename to defineReadableTable

* deps: upgrade ticketing-example

* fix: stray console.log

* deps: bump preact again

* chore: preact->react

* fix: parse params.event as number

* fix: correct event references

* Allow integrations to define schema

* fix: file-url plugin failure on repeated generateBundle() runs

* update url

* Cleanup

* Linting

* fix windows file permission issue

When runnng `astro dev`, the watcher would close trying to delete the `content.db` file due to a file permission error. This change makes the local DB client a disposable to allow cleanup after usage.

* Formatting

* "fix" Symbol.dispose usage

---------

Co-authored-by: Nate Moore <nate@astro.build>
Co-authored-by: bholmesdev <hey@bholmes.dev>
Co-authored-by: Fred K. Schott <fkschott@gmail.com>
Co-authored-by: itsMapleLeaf <19603573+itsMapleLeaf@users.noreply.github.com>
2024-02-22 14:50:44 -05:00