0
Fork 0
mirror of https://github.com/project-zot/zot.git synced 2024-12-16 21:56:37 -05:00
Commit graph

631 commits

Author SHA1 Message Date
peusebiu
6f00e843a0
fix(sync): sync generator now backs off on errors (#2272)
handle unsupported features like oci artifacts.

closes: #2238

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-03-04 09:44:11 -08:00
peusebiu
740eae8f26
fix(sync): better cleaning sync's download dir (#2273)
added cleanup in the case of copy.Image() failures.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-02-29 09:09:21 -08:00
Ramkumar Chinchani
565eca2609
chore: fix dependabot alerts (#2268)
https://github.com/project-zot/zot/pull/2258
https://github.com/project-zot/zot/pull/2259
https://github.com/project-zot/zot/pull/2260
https://github.com/project-zot/zot/pull/2261
https://github.com/project-zot/zot/pull/2262
https://github.com/project-zot/zot/pull/2263
https://github.com/project-zot/zot/pull/2264
https://github.com/project-zot/zot/pull/2265
https://github.com/project-zot/zot/pull/2266
https://github.com/project-zot/zot/pull/2267

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-02-20 21:51:40 +02:00
Andrei Aaron
4e5db84cb1
chore: update image-spec and dist spec to 1.1.0 (#2255)
BREAKING CHANGE: the dist spec version in the config files needs to be bumped to 1.1.0
in order for the config verification to pass without warnings.

Also fix 1 dependabot alert for helm.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-20 13:27:21 +02:00
Andrei Aaron
6c953d6400
test: fix flacky coverage in cookiestore cleanup tests (#2257)
Refactor and add more coverage to test flacky coverage in case sessions
which are already deleted are flagged as expired/for deletion.

See coverage drop in pkg/api/cookiestore.go:
8e68255946/indirect-changes

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-19 14:43:30 -08:00
Andrei Aaron
2d2e005449
fix(npe): handle case where os.Stat returns different error types in DirExists (#2253)
See https://github.com/project-zot/zot/actions/runs/7905369535/job/21577848110

Also add tests to fix some of the coverage fluctuations.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-18 08:00:00 +02:00
Vishwas R
0aa6bf0fff
feat: include PackagePath data in CVEs for image queries (#2241)
Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
2024-02-15 13:19:49 -08:00
peusebiu
8e68255946
fix(sync): added bearer client for sync (#2222)
fixed ping function taking too much time

closes: #2213 #2212

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-02-14 09:18:10 -08:00
Andrei Aaron
d0eb043be5
feat: Get the image LastUpdated timestamp from annotations (#2240)
Fallback to Created field and the History entries in the image config
only if the annotation "org.opencontainers.image.created" is not available

closes #2210

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-14 09:14:24 -08:00
Andreea Lupu
55acce6923
feat(graphql): filter CVEs by severity (#2246)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2024-02-14 09:11:57 -08:00
LaurentiuNiculae
de90abd5dc
style(metadb): use type aliases for metadb types to be easier to read (#2043)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-02-14 09:08:08 -08:00
peusebiu
5b83937d40
fix(tests): fixed inconsistent sync test (#2237)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-02-08 18:28:59 +02:00
Andrei Aaron
60dc8569ec
build(go): switch to go 1.21 (#2049)
Also update to the latest swaggo version, as the previous one did not work with go 1.21

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-07 10:54:28 -08:00
Andrei Aaron
6a83dd47c0
fix(scheduler): the session cleanup generator is reset too often (#2220)
This causes the "fair" scheduler to run it too often in the detriment of other generators.
The intention was to run it every 2 hours but the measurement unit for 7200 was not specified.

Add more logs, including showing a generator name, in order to troubleshoot this kind of issues easier in the future.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-01 09:15:53 -08:00
Andrei Aaron
ce4924f841
refactor: rename go module from zotregistry.io/zot to zotregistry.dev/zot (#2187)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-31 20:34:07 -08:00
Andrei Aaron
a2b923b6fd
feat(search): search for a specific tag cross-repo (#2211)
Syntax to search for `<tag_name>` accross all repos is `:<tag_name>`

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-30 09:12:41 -08:00
Andreea Lupu
ddba1b7baf
fix(scrub): hold locks per image not per repo while executing scrub (#2180)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2024-01-25 11:12:21 -08:00
LaurentiuNiculae
1785688b7c
feat(ldap): hot reloading ldap credentials on change (#2167)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-01-25 09:15:22 -08:00
Andrei Aaron
8215766720
fix: the scheduler is now fair (#2158)
Generators are now ordered by rank in the priority queue.

The rank computation formula is:
- 100/(1+generated_task_count) for high priority tasks
- 10/(1+generated_task_count) for medium priority tasks
- 1/(1+generated_task_count) for low priority tasks

Note the ranks are used when comparing generators both with the same priority and with different priority.
So now we are:
- giving an opportunity to all generators with the same priority to take turns generating tasks
- giving roughly 1 low priority and 10 medium priority tasks the opportunity to run for every 100 high priority tasks running.

After a generator generates a task, the generators are reordered in the priority queue based on rank.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-25 09:05:47 -08:00
peusebiu
e9ab520905
fix(bearer): fixed /v2/ route not implementing token spec (#2176)
We use chartmuseum lib for handling bearer requests, which is not
implementing the token spec, mainly it expects "scope" parameter
to be given on every request, even for /v2/ route which doesn't represent
a resource.

Handle this /v2/ route inside our code.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-01-22 09:15:27 -08:00
Andrei Aaron
ed6be0580e
refactor: replace deprecated APIs for creating images in the search tests (#2173)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-22 09:10:34 -08:00
LaurentiuNiculae
3f97f878fd
feat(cve): add option to exclude string from cve search (#2163)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-01-19 12:59:42 -08:00
Andrei Aaron
8467a80a50
refactor: update tests to use the newer API for creating test images (#2168)
- update cve tests
- update scrub tests
- update tests for parsing storage and loading into meta DB
- update controller tests

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-17 10:20:07 -08:00
peusebiu
ee9bbb0bf2
feat(log): print traceback when panics occur (#2166)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-01-16 09:08:14 -08:00
peusebiu
d1bf713573
fix: excessive memory usage (#2164)
instead of reading entire files before calculating their digests
stream them by using their Reader method.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-01-16 09:04:36 -08:00
peusebiu
0bed25dddd
feat(config): handle config files with no explicit extension (#2147)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-01-15 13:10:56 +02:00
Ramkumar Chinchani
d685adb029
fix: npe if ldap query doesn't return attributes (#2151)
We cannot assume the LDAP server will have group attributes programmed
everytime. So handle it accordingly.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-01-12 14:08:35 -08:00
Ramkumar Chinchani
1c756b4db9
fix: enable panic backtraces (#2150)
We expect panics in the server/datapath to be few and far between.
So the backtraces are more valuable now.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-01-11 13:56:50 -08:00
Andrei Aaron
2a6bf66cb2
fix: high CPU utilization by scheduler while idle (#2156)
resolves #2155

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-11 09:30:16 -08:00
peusebiu
a46e10269a
fix(shutdown): fix crash when shutting down before server and task scheduler have started. (#2148)
init shutdown routine after controller.Init()
check for nil values before stopping http server and task scheduler.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-01-05 17:50:48 -08:00
Raul Kele
59f41ac17d
fix(ui): Add users route to extension_ui.go (#2141)
Add new ui route "/user" to ui router

Signed-off-by: Raul-Cristian Kele <raulkeleblk@gmail.com>
2023-12-17 21:56:06 +02:00
Andrei Aaron
cff74578be
fix(apikey): show api key configuration in mgmt API (#2138)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-14 09:54:51 -08:00
Andrei Aaron
18aa975ae2
feat(CVE): add CVE severity counters to returned images and CVE list calls (#2131)
For CLI output is similar to:

CRITICAL 0, HIGH 1, MEDIUM 1, LOW 0, UNKNOWN 0, TOTAL 2

ID                SEVERITY  TITLE
CVE-2023-0464     HIGH      openssl: Denial of service by excessive resou...
CVE-2023-0465     MEDIUM    openssl: Invalid certificate policies in leaf...

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-13 09:16:31 -08:00
peusebiu
dbb1c3519f
feat(ui): let UI delete manifests if current user has permissions to do so (#2132)
- added a new field 'IsDeletable' for graphql ImageSummary struct.
- apply cors on DeleteManifest route

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-12-13 09:06:08 -08:00
peusebiu
7642e5af98
fix(scheduler): fix data race (#2085)
* fix(scheduler): data race when pushing new tasks

the problem here is that scheduler can be closed in two ways:
- canceling the context given as argument to scheduler.RunScheduler()
- running scheduler.Shutdown()

because of this shutdown can trigger a data race between calling scheduler.inShutdown()
and actually pushing tasks into the pool workers

solved that by keeping a quit channel and listening on both quit channel and ctx.Done()
and closing the worker chan and scheduler afterwards.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

* refactor(scheduler): refactor into a single shutdown

before this we could stop scheduler either by closing the context
provided to RunScheduler(ctx) or by running Shutdown().

simplify things by getting rid of the external context in RunScheduler().
keep an internal context in the scheduler itself and pass it down to all tasks.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

---------

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-12-11 10:00:34 -08:00
Andrei Aaron
d71a1f494e
ci(nightly): fix nightly after log message refactor (#2121)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-11 15:39:32 +02:00
Andrei Aaron
ec7af4979f
fix(proto): the size of the repo should be int64, since that is the same type used for the manifest/config/index/digest sizes it sums up. (#2120)
Using int32 may result in negative size values when returned by the graphql API

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-08 11:38:30 -08:00
LaurentiuNiculae
79e14027ee
refactor(test): add lint rule for messages starting with the component (#2045)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-08 10:05:02 +02:00
Andrei Aaron
f321fa91fe
feat(pagination): make sure the URL to in the link header is inside angle brackets (#2116)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-06 09:54:47 -08:00
Andreea Lupu
e3bd9a8fa8
fix(log): trimmed error level logs (#2115)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-12-05 12:01:01 +02:00
Alexei Dodon
2e733b3f4f
feat(metrics): add scheduler related metrics (#2076)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-12-04 14:13:50 -08:00
Ramkumar Chinchani
8bac653dd2
chore: fix dependabot alerts (#2113)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-12-04 11:28:01 -08:00
LaurentiuNiculae
90d27ff2ac
feat(cve): expand search domain to cve description and package info (#2086)
* feat(cve): add reference url for cve

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(cve): expand search domain to cve description and package info

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

---------

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-29 20:59:00 +02:00
Andreea Lupu
e59d8da454
fix(metadb): set LastUpdated field also for indexes (#2088)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-11-29 15:15:39 +02:00
peusebiu
3c8da6e6fc
Sync s3 (#2073)
* feat(sync): local tmp store

Signed-off-by: a <a@tuxpa.in>

* fix(sync): various fixes for s3+remote storage feature

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

---------

Signed-off-by: a <a@tuxpa.in>
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Co-authored-by: a <a@tuxpa.in>
2023-11-28 12:08:15 -08:00
LaurentiuNiculae
0de2210686
feat(metadb): add support for querying for images by a blob digest (#2077)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-27 08:52:52 -08:00
peusebiu
6222dae1f0
feat(scheduler): gracefully shutdown (#1951)
wait for workers to finish before exiting

should fix tests reporting they couldn't remove rootDir because it's being
written by tasks

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-11-24 00:40:10 -08:00
LaurentiuNiculae
92837c2bcb
refactor(log): replace panics with log fatal or log panic functions (#1723)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-24 00:38:36 -08:00
LaurentiuNiculae
83f287d1f6
feat(cli): add command to interogate the server version and other details (#1709)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-22 19:25:21 +02:00
Ramkumar Chinchani
8e7b2d2047
fix(metrics): one-time tasks should not be starved (#2053)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-11-17 08:58:15 +02:00