0
Fork 0
mirror of https://github.com/project-zot/zot.git synced 2024-12-30 22:34:13 -05:00
Commit graph

308 commits

Author SHA1 Message Date
Andrei Aaron
864cd00b9e
fix: Allow GET requests on repositories not found in metadb (#2351)
The issue was reported on Slack.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-03-26 18:38:58 +02:00
Ramkumar Chinchani
fdb401273c
fix: ignore metadb errors if tag not found (#2301) 2024-03-13 07:28:08 +02:00
Ramkumar Chinchani
18235ca254
fix(oras)!: remove ORAS artifact references support (#2294)
* fix(oras)!: remove ORAS artifact references support

ORAS artifacts/references predated OCI dist-spec 1.1.0 which now has the
same functionality and likely to see wider adoption.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* test: update to released official images

So that they are unlikely to be deleted.
*-rc images may be cleaned up over time.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

---------

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-03-06 12:16:42 -08:00
Andrei Aaron
4e5db84cb1
chore: update image-spec and dist spec to 1.1.0 (#2255)
BREAKING CHANGE: the dist spec version in the config files needs to be bumped to 1.1.0
in order for the config verification to pass without warnings.

Also fix 1 dependabot alert for helm.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-20 13:27:21 +02:00
Andrei Aaron
6c953d6400
test: fix flacky coverage in cookiestore cleanup tests (#2257)
Refactor and add more coverage to test flacky coverage in case sessions
which are already deleted are flagged as expired/for deletion.

See coverage drop in pkg/api/cookiestore.go:
8e68255946/indirect-changes

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-19 14:43:30 -08:00
peusebiu
8e68255946
fix(sync): added bearer client for sync (#2222)
fixed ping function taking too much time

closes: #2213 #2212

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-02-14 09:18:10 -08:00
Andrei Aaron
60dc8569ec
build(go): switch to go 1.21 (#2049)
Also update to the latest swaggo version, as the previous one did not work with go 1.21

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-07 10:54:28 -08:00
Andrei Aaron
6a83dd47c0
fix(scheduler): the session cleanup generator is reset too often (#2220)
This causes the "fair" scheduler to run it too often in the detriment of other generators.
The intention was to run it every 2 hours but the measurement unit for 7200 was not specified.

Add more logs, including showing a generator name, in order to troubleshoot this kind of issues easier in the future.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-01 09:15:53 -08:00
Andrei Aaron
ce4924f841
refactor: rename go module from zotregistry.io/zot to zotregistry.dev/zot (#2187)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-31 20:34:07 -08:00
LaurentiuNiculae
1785688b7c
feat(ldap): hot reloading ldap credentials on change (#2167)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-01-25 09:15:22 -08:00
peusebiu
e9ab520905
fix(bearer): fixed /v2/ route not implementing token spec (#2176)
We use chartmuseum lib for handling bearer requests, which is not
implementing the token spec, mainly it expects "scope" parameter
to be given on every request, even for /v2/ route which doesn't represent
a resource.

Handle this /v2/ route inside our code.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-01-22 09:15:27 -08:00
Andrei Aaron
8467a80a50
refactor: update tests to use the newer API for creating test images (#2168)
- update cve tests
- update scrub tests
- update tests for parsing storage and loading into meta DB
- update controller tests

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-17 10:20:07 -08:00
peusebiu
ee9bbb0bf2
feat(log): print traceback when panics occur (#2166)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-01-16 09:08:14 -08:00
Ramkumar Chinchani
d685adb029
fix: npe if ldap query doesn't return attributes (#2151)
We cannot assume the LDAP server will have group attributes programmed
everytime. So handle it accordingly.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-01-12 14:08:35 -08:00
Ramkumar Chinchani
1c756b4db9
fix: enable panic backtraces (#2150)
We expect panics in the server/datapath to be few and far between.
So the backtraces are more valuable now.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-01-11 13:56:50 -08:00
peusebiu
a46e10269a
fix(shutdown): fix crash when shutting down before server and task scheduler have started. (#2148)
init shutdown routine after controller.Init()
check for nil values before stopping http server and task scheduler.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-01-05 17:50:48 -08:00
peusebiu
dbb1c3519f
feat(ui): let UI delete manifests if current user has permissions to do so (#2132)
- added a new field 'IsDeletable' for graphql ImageSummary struct.
- apply cors on DeleteManifest route

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-12-13 09:06:08 -08:00
peusebiu
7642e5af98
fix(scheduler): fix data race (#2085)
* fix(scheduler): data race when pushing new tasks

the problem here is that scheduler can be closed in two ways:
- canceling the context given as argument to scheduler.RunScheduler()
- running scheduler.Shutdown()

because of this shutdown can trigger a data race between calling scheduler.inShutdown()
and actually pushing tasks into the pool workers

solved that by keeping a quit channel and listening on both quit channel and ctx.Done()
and closing the worker chan and scheduler afterwards.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

* refactor(scheduler): refactor into a single shutdown

before this we could stop scheduler either by closing the context
provided to RunScheduler(ctx) or by running Shutdown().

simplify things by getting rid of the external context in RunScheduler().
keep an internal context in the scheduler itself and pass it down to all tasks.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

---------

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-12-11 10:00:34 -08:00
LaurentiuNiculae
79e14027ee
refactor(test): add lint rule for messages starting with the component (#2045)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-08 10:05:02 +02:00
Andrei Aaron
f321fa91fe
feat(pagination): make sure the URL to in the link header is inside angle brackets (#2116)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-06 09:54:47 -08:00
Alexei Dodon
2e733b3f4f
feat(metrics): add scheduler related metrics (#2076)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-12-04 14:13:50 -08:00
peusebiu
6222dae1f0
feat(scheduler): gracefully shutdown (#1951)
wait for workers to finish before exiting

should fix tests reporting they couldn't remove rootDir because it's being
written by tasks

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-11-24 00:40:10 -08:00
LaurentiuNiculae
92837c2bcb
refactor(log): replace panics with log fatal or log panic functions (#1723)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-24 00:38:36 -08:00
LaurentiuNiculae
4fb1e756c4
feat(startup): update logic for metadb update on startup, skip unmodified repos (#2024)
- MetaDB stores the time of the last update of a repo
- During startup we check if the layout has been updated after the last recorded change in the db
- If this is the case, the repo is parsed and updated in the DB otherwise it's skipped

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-16 10:39:27 -08:00
LaurentiuNiculae
272eb7cc43
feat(ldap): add option to load ldap from file (#1778)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-14 16:21:36 -08:00
peusebiu
7f52f58e3c
fix(routes): fix cors headers for api keys and logout route (#1984)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-11-07 10:11:40 -08:00
peusebiu
9074f8483b
feat(retention): added image retention policies (#1866)
feat(metaDB): add more image statistics info

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-11-01 09:16:18 -07:00
Alexei Dodon
a79d79a03a
fix: more accurate storage metrics after zot restart (#1972)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-11-01 09:09:21 -07:00
LaurentiuNiculae
56ad9e6707
refactor(metadb): improve UX by speeding up metadb serialize/deserialize (#1842)
Use protocol buffers and update the metadb interface to better suit our search needs

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-30 13:06:04 -07:00
Alexei Dodon
d2fbd273ba
fix: tests refactoring (#1950)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-26 11:20:39 +03:00
Ramkumar Chinchani
1675f30d4a
ci: update golangci-lint version (#1834)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-20 17:27:04 +03:00
Andrei Aaron
7ce5a74598
feat: use the "zot" namespace for the authentication url (#1947)
Some other minor fixes for swaggo comments (indentation and a bad description)

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-10-20 15:30:56 +03:00
Alexei Dodon
a345ba0823
fix: metrics should be protected behind authZ (#1895)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-20 10:33:26 +03:00
peusebiu
7f6534a52d
fix(sessions): periodically cleanup expired sessions (#1939)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-16 20:03:42 -07:00
Andreea Lupu
fc2380b57b
fix: add support for uploaded index when signing using notation (#1882)
ci(notation): update to latest notation version
fix(sync): add layers info when syncing signatures

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-10-12 18:45:20 -07:00
peusebiu
a91c0c5cfe
fix(authn): create sessions only if UI header value is supplied (#1919)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-12 16:37:55 +03:00
peusebiu
d1fcab421a
fix(authn): apply fail delay only if credentials/sessions are supplied (#1920)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-12 16:32:36 +03:00
Alexei Dodon
044ea85279
fix: running tests locally fails (#1879)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-05 14:34:50 +03:00
Alexei Dodon
2fd7bfc37a
fix: metrics endpoint must be secured behind authN (#1864)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-02 16:37:21 +03:00
Alexei Dodon
75085dcff5
fix: errors returned by zot should match the dist-spec errors (#1868)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-29 13:45:19 -07:00
Andrei Aaron
ba6f347d8d
refactor(pkg/test): split logic in pkg/test/common.go into multiple packages (#1861)
Which could be imported independently. See more details:
1. "zotregistry.io/zot/pkg/test/common" - currently used as
   tcommon "zotregistry.io/zot/pkg/test/common" - inside pkg/test
   test "zotregistry.io/zot/pkg/test/common" - in tests
   . "zotregistry.io/zot/pkg/test/common" - in tests
Decouple zb from code in test/pkg in order to keep the size small.

2. "zotregistry.io/zot/pkg/test/image-utils" - curently used as
   . "zotregistry.io/zot/pkg/test/image-utils"

3. "zotregistry.io/zot/pkg/test/deprecated" -  curently used as
   "zotregistry.io/zot/pkg/test/deprecated"
This one will bre replaced gradually by image-utils in the future.

4. "zotregistry.io/zot/pkg/test/signature" - (cosign + notation) use as
   "zotregistry.io/zot/pkg/test/signature"

5. "zotregistry.io/zot/pkg/test/auth" - (bearer + oidc)  curently used as
   authutils "zotregistry.io/zot/pkg/test/auth"

 6. "zotregistry.io/zot/pkg/test/oci-utils" -  curently used as
   ociutils "zotregistry.io/zot/pkg/test/oci-utils"

Some unused functions were removed, some were replaced, and in
a few cases specific funtions were moved to the files they were used in.

Added an interface for the StoreController, this reduces the number of imports
of the entire image store, decreasing binary size for tests.
If the zb code was still coupled with pkg/test, this would have reflected in zb size.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-27 11:34:48 -07:00
Andreea Lupu
92e382ce39
refactor(scrub): replace umoci logic in scrub implementation (#1845)
- implement scrub also for S3 storage by replacing umoci
- change scrub implementation for ImageIndex
- take the `Subject` into consideration when running scrub
- remove test code relying on the umoci library. Since we started
relying on images in test/data, and we create our own images using
go code we can obtain digests by other means. (cherry picked from commit 489d4e2d23c1b4e48799283f8281024bbef6123f)

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-09-26 11:02:11 -07:00
peusebiu
1df743f173
fix(gc): sync repodb when gc'ing manifests (#1819)
fix(gc): fix cleaning deduped blobs because they have the modTime of
the original blobs, fixed by updating the modTime when hard linking
the blobs.
fix(gc): failing to parse rootDir at zot startup when using s3 storage
because there are no files under rootDir and we can not create empty dirs
on s3, fixed by creating an empty file under rootDir.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-22 11:51:20 -07:00
Andrei Aaron
7c78f80a96
feat(cve): implement CVE scanning as background tasks (#1833)
1. Move existing CVE DB download generator/task login under the cve package
2. Add a new CVE scanner task generator and task type to run in the background, as well as tests for it
3. Move the CVE cache in its own package
4. Add a CVE scanner methods to check if an entry is present in the cache, and to retreive the results
5. Modify the FilterTags MetaDB method to not exit on first error
This is needed in order to pass all tags to the generator,
instead of the generator stopping at the first set of invalid data
6. Integrate the new scanning task generator with the existing zot code.
7. Fix an issue where the CVE scan results for multiarch images was not cached
8. Rewrite some of the older CVE tests to use the new image-utils test package
9. Use the CVE scanner as attribute of the controller instead of CveInfo.
Remove functionality of CVE DB update from CveInfo, it is now responsible,
as the name states, only for providing CVE information.
10. The logic to get maximum severity and cve count for image sumaries now uses only the scanner cache.
11. Removed the GetCVESummaryForImage method from CveInfo as it was only used in tests

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-22 11:49:17 -07:00
Andrei Aaron
a11fe2d195
feat(pprof): add profiling route handler to debug runtime (#1818)
(cherry picked from commit 56ddb70f624e7070ad0d3531d498675f9f82c664)

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Alex Stan <alexandrustan96@yahoo.ro>
2023-09-18 14:05:41 -07:00
LaurentiuNiculae
8e18917b07
refactor(test): move image utils for tests in a separate module (#1789)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-09-15 09:53:15 -07:00
Alexei Dodon
14206dd6f3
refactor: Review metrics endpoints (#1770)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-15 14:49:34 +03:00
peusebiu
3d8d47d601
fix(config): fix config reloader panic (#1806)
reloading config from one without extensions
to one with extensions caused a panic

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-14 12:34:18 +03:00
Alexei Dodon
48bf7f69f8
refactor: Reduce zb binary size (#1783)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-13 10:28:14 +03:00
Andrei Aaron
24e37eb68b
fix(api): Fix 'last' query param for <repo>/tags/list to work without param 'n' (#1777)
Also fix additional issues:
- sorting of tags on calls without pagination parameters ('n' or 'last')
- if 'n' is 0 we should return an empty list and not error

Added tests accordingly

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-08 19:46:17 +03:00