0
Fork 0
mirror of https://github.com/project-zot/zot.git synced 2025-01-13 22:50:38 -05:00
Commit graph

232 commits

Author SHA1 Message Date
Ramkumar Chinchani
72c6e8afb3
chore: fix dependabot alerts (#2810)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-12-06 09:04:46 -08:00
Andrei Aaron
a76bfd4283
chore: update Trivy and Trivy dependencies (#2763)
The Trivy library now supports multiple locations from where to download the DBs.
The zot code has been updated to properly call the updated library functions.
If at some point we would want to support multiple Trivy DBs in zot, we could look into it more.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-11-07 09:03:37 -08:00
Andrei Aaron
da923ae232
chore: update go tests to use our hosted trivy-db and trivy-java-db images (#2754)
There are 2 remaining exceptions that I am aware of:
1. The tests under test/blackbox/cve.bats
2. One of the cli tests checking the server attempts download of the databases
from the default url

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-11-01 14:14:52 -07:00
Ramkumar Chinchani
cb2af94b0b
feat: add support for docker images (#2714)
* feat: add support for docker images

Issue #724

A new config section under "HTTP" called "Compat" is added which
currently takes a list of possible compatible legacy media-types.

https://github.com/opencontainers/image-spec/blob/main/media-types.md#compatibility-matrix

Only "docker2s2" (Docker Manifest V2 Schema V2) is currently supported.

Garbage collection also needs to be made aware of non-OCI compatible
layer types.
feat: add cve support for non-OCI compatible layer types

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* 

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* test: add more docker compat tests

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* feat: add additional validation checks for non-OCI images

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* ci: make "full" images docker-compatible

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

---------

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-10-31 09:44:04 +02:00
Andrei Aaron
da6bd56a21
fix: issues with nested index processing in CVE and metaDB code (#2732)
Also fix an issue with searching tags, which should work with case insensitive searches.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-10-21 09:57:43 -07:00
Ramkumar Chinchani
c89be3ad31
chore: fix dependabot alerts (#2709)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-10-07 16:20:37 -07:00
peusebiu
e6624a29a5
feat(graphql): Add LastPullTimestamp and PushTimestamp in ImageSummar… (#2699)
feat(graphql): Add LastPullTimestamp and PushTimestamp in ImageSummary resposne

Signed-off-by: Eusebiu Petu <petu.eusebiu@gmail.com>
2024-10-03 12:27:03 -07:00
Andrei Aaron
8553712613
chore: upgrade trivy to v0.55.2 and update the logic of waiting for zot to start in some jobs (#2685)
chore: upgrade trivy to v0.55.2, also update the logic of waiting for zot to start in some jobs

Seems like there's an increate in the time zot requires to start before servicing requests.
From my GitHub observations it is better check using curl instead of relying on hardcoded 5s or 10s values.

The logic in .github/workflows/cluster.yaml seems to be old and out of date.
Even on main right now there is only 1 our of 3 zots actualy running.
The other 2 are actually erroring: Error: operation timeout: boltdb file is already in use, path '/tmp/zot/cache.db'
This is unrelated to this PR, I am seeing the same issue in the olders workflow runs still showing the logs

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-09-30 10:37:53 -07:00
Ramkumar Chinchani
9cf6b0205d
chore: fix dependabot alerts (#2681)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-09-27 09:05:14 +03:00
Jan-Otto Kröpke
f618b1d4ef
ci(deps): upgrade golangci-lint (#2556)
* ci(deps): upgrade golangci-lint

Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>

* build(deps): removed disabled linters

Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>

* build(deps): go run github.com/daixiang0/gci@latest write .

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* build(deps): go run golang.org/x/tools/cmd/goimports@latest -l -w .

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* build(deps): go run github.com/bombsimon/wsl/v4/cmd...@latest -strict-append -test=true -fix ./...

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* build(deps): go run github.com/catenacyber/perfsprint@latest -fix ./...

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* build(deps): replace gomnd by mnd

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* build(deps): make gqlgen

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* build: Revert "build(deps): go run github.com/daixiang0/gci@latest write ."

This reverts commit 5bf8c42e1f.

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* build(deps): go run github.com/daixiang0/gci@latest write -s 'standard' -s default -s 'prefix(zotregistry.dev/zot)' .

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* build(deps): make gqlgen

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: wsl issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: check-log issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: gci issues

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

* fix: tests

Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>

---------

Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>
2024-07-29 10:32:51 -07:00
Ramkumar Chinchani
fb2edcc269
chore: fix dependabot alerts (#2486)
https://github.com/project-zot/zot/pull/2475
https://github.com/project-zot/zot/pull/2477
https://github.com/project-zot/zot/pull/2478
https://github.com/project-zot/zot/pull/2479
https://github.com/project-zot/zot/pull/2480
https://github.com/project-zot/zot/pull/2481
https://github.com/project-zot/zot/pull/2482
https://github.com/project-zot/zot/pull/2483
https://github.com/project-zot/zot/pull/2484
https://github.com/project-zot/zot/pull/2485

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-06-17 20:31:01 +03:00
peusebiu
e023936e8e
fix(ui): fix image details view (#2470)
when a UI client tries to view image details
for an image with multiple tags pointing to the same digest
we make a query to dynamodb having duplicate keys (same digest)
resulting in an error and the client is redirect back to image
overview.

closes: #2464

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-06-14 09:22:48 -07:00
Ramkumar Chinchani
1594852428
chore: fix dependabot alerts (#2446)
* chore: fix dependabot alerts

https://github.com/project-zot/zot/pull/2435
https://github.com/project-zot/zot/pull/2436
https://github.com/project-zot/zot/pull/2437
https://github.com/project-zot/zot/pull/2438
https://github.com/project-zot/zot/pull/2439
https://github.com/project-zot/zot/pull/2440
https://github.com/project-zot/zot/pull/2441
https://github.com/project-zot/zot/pull/2442
https://github.com/project-zot/zot/pull/2443
https://github.com/project-zot/zot/pull/2444
https://github.com/project-zot/zot/pull/2445
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* fix(zli): _schema query in zli code should not use empty parens

Fix also some tests
See https://github.com/vektah/gqlparser/issues/292 and https://github.com/vektah/gqlparser/pull/293

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

---------

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Andrei Aaron <aaaron@luxoft.com>
2024-06-04 13:54:30 +03:00
Andrei Aaron
2bb46b0562
chore: fix dependabot alerts (#2431)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-05-27 14:37:27 -07:00
Ramkumar Chinchani
186855b5f8
fix: additional input validation for CVE graphQL query (#2408)
It is possible to ask for a very large limit size which can exhaust
memory.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-04-24 09:23:17 +03:00
Ramkumar Chinchani
28e9aabecf
chore: fix dependabot alerts (#2331)
https://github.com/project-zot/zot/pull/2324
https://github.com/project-zot/zot/pull/2325
https://github.com/project-zot/zot/pull/2326
https://github.com/project-zot/zot/pull/2327
https://github.com/project-zot/zot/pull/2328
https://github.com/project-zot/zot/pull/2329

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-03-20 07:37:29 +02:00
LaurentiuNiculae
5039128723
feat(cve): cli cve diff (#2242)
* feat(gql): add new query for diff of cves for 2 images

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(cli): add cli for cve diff

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

---------

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-03-06 10:40:29 +02:00
peusebiu
6f00e843a0
fix(sync): sync generator now backs off on errors (#2272)
handle unsupported features like oci artifacts.

closes: #2238

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2024-03-04 09:44:11 -08:00
Ramkumar Chinchani
565eca2609
chore: fix dependabot alerts (#2268)
https://github.com/project-zot/zot/pull/2258
https://github.com/project-zot/zot/pull/2259
https://github.com/project-zot/zot/pull/2260
https://github.com/project-zot/zot/pull/2261
https://github.com/project-zot/zot/pull/2262
https://github.com/project-zot/zot/pull/2263
https://github.com/project-zot/zot/pull/2264
https://github.com/project-zot/zot/pull/2265
https://github.com/project-zot/zot/pull/2266
https://github.com/project-zot/zot/pull/2267

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2024-02-20 21:51:40 +02:00
Vishwas R
0aa6bf0fff
feat: include PackagePath data in CVEs for image queries (#2241)
Signed-off-by: Vishwas Rajashekar <vrajashe@cisco.com>
2024-02-15 13:19:49 -08:00
Andrei Aaron
d0eb043be5
feat: Get the image LastUpdated timestamp from annotations (#2240)
Fallback to Created field and the History entries in the image config
only if the annotation "org.opencontainers.image.created" is not available

closes #2210

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-14 09:14:24 -08:00
Andreea Lupu
55acce6923
feat(graphql): filter CVEs by severity (#2246)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2024-02-14 09:11:57 -08:00
LaurentiuNiculae
de90abd5dc
style(metadb): use type aliases for metadb types to be easier to read (#2043)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-02-14 09:08:08 -08:00
Andrei Aaron
6a83dd47c0
fix(scheduler): the session cleanup generator is reset too often (#2220)
This causes the "fair" scheduler to run it too often in the detriment of other generators.
The intention was to run it every 2 hours but the measurement unit for 7200 was not specified.

Add more logs, including showing a generator name, in order to troubleshoot this kind of issues easier in the future.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-02-01 09:15:53 -08:00
Andrei Aaron
ce4924f841
refactor: rename go module from zotregistry.io/zot to zotregistry.dev/zot (#2187)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-31 20:34:07 -08:00
Andrei Aaron
a2b923b6fd
feat(search): search for a specific tag cross-repo (#2211)
Syntax to search for `<tag_name>` accross all repos is `:<tag_name>`

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-30 09:12:41 -08:00
Andrei Aaron
ed6be0580e
refactor: replace deprecated APIs for creating images in the search tests (#2173)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-22 09:10:34 -08:00
LaurentiuNiculae
3f97f878fd
feat(cve): add option to exclude string from cve search (#2163)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-01-19 12:59:42 -08:00
Andrei Aaron
8467a80a50
refactor: update tests to use the newer API for creating test images (#2168)
- update cve tests
- update scrub tests
- update tests for parsing storage and loading into meta DB
- update controller tests

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-17 10:20:07 -08:00
Andrei Aaron
18aa975ae2
feat(CVE): add CVE severity counters to returned images and CVE list calls (#2131)
For CLI output is similar to:

CRITICAL 0, HIGH 1, MEDIUM 1, LOW 0, UNKNOWN 0, TOTAL 2

ID                SEVERITY  TITLE
CVE-2023-0464     HIGH      openssl: Denial of service by excessive resou...
CVE-2023-0465     MEDIUM    openssl: Invalid certificate policies in leaf...

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-13 09:16:31 -08:00
peusebiu
dbb1c3519f
feat(ui): let UI delete manifests if current user has permissions to do so (#2132)
- added a new field 'IsDeletable' for graphql ImageSummary struct.
- apply cors on DeleteManifest route

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-12-13 09:06:08 -08:00
peusebiu
7642e5af98
fix(scheduler): fix data race (#2085)
* fix(scheduler): data race when pushing new tasks

the problem here is that scheduler can be closed in two ways:
- canceling the context given as argument to scheduler.RunScheduler()
- running scheduler.Shutdown()

because of this shutdown can trigger a data race between calling scheduler.inShutdown()
and actually pushing tasks into the pool workers

solved that by keeping a quit channel and listening on both quit channel and ctx.Done()
and closing the worker chan and scheduler afterwards.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

* refactor(scheduler): refactor into a single shutdown

before this we could stop scheduler either by closing the context
provided to RunScheduler(ctx) or by running Shutdown().

simplify things by getting rid of the external context in RunScheduler().
keep an internal context in the scheduler itself and pass it down to all tasks.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

---------

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-12-11 10:00:34 -08:00
LaurentiuNiculae
79e14027ee
refactor(test): add lint rule for messages starting with the component (#2045)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-08 10:05:02 +02:00
Alexei Dodon
2e733b3f4f
feat(metrics): add scheduler related metrics (#2076)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-12-04 14:13:50 -08:00
Ramkumar Chinchani
8bac653dd2
chore: fix dependabot alerts (#2113)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-12-04 11:28:01 -08:00
LaurentiuNiculae
90d27ff2ac
feat(cve): expand search domain to cve description and package info (#2086)
* feat(cve): add reference url for cve

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(cve): expand search domain to cve description and package info

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

---------

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-29 20:59:00 +02:00
LaurentiuNiculae
0de2210686
feat(metadb): add support for querying for images by a blob digest (#2077)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-27 08:52:52 -08:00
peusebiu
6222dae1f0
feat(scheduler): gracefully shutdown (#1951)
wait for workers to finish before exiting

should fix tests reporting they couldn't remove rootDir because it's being
written by tasks

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-11-24 00:40:10 -08:00
LaurentiuNiculae
4fb1e756c4
feat(startup): update logic for metadb update on startup, skip unmodified repos (#2024)
- MetaDB stores the time of the last update of a repo
- During startup we check if the layout has been updated after the last recorded change in the db
- If this is the case, the repo is parsed and updated in the DB otherwise it's skipped

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-16 10:39:27 -08:00
Andrei Aaron
38f10af8cf
docs: update graphql examples to match current implementation (#2038)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-11-13 09:58:15 -08:00
LaurentiuNiculae
2db6e86fb5
fix(cov): coverage boltdb+dynamo (#2018)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-08 13:37:52 -08:00
LaurentiuNiculae
c9cc5b9acb
test(meta): add push-pull-read tests for metadb (#2022)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-08 13:35:51 -08:00
Andreea Lupu
d5065513f5
feat: add support for oci1.1 cosign signatures(using referrers) (#1963)
- Cosign supports 2 types of signature formats:

	1. Using tag -> each new signature of the same manifest is
	added as a new layer of the signature manifest having that
	specific tag("{alghoritm}-{digest_of_signed_manifest}.sig")

	2. Using referrers -> each new signature of the same manifest is
	added as a new manifest

- For adding these cosign signature to metadb, we reserved index 0 of the
list of cosign signatures for tag-based signatures. When a new tag-based
signature is added for the same manifest, the element on first position
in its list of cosign signatures(in metadb) will be updated/overwritten.
When a new cosign signature(using referrers) will be added for the same
manifest this new signature will be appended to the list of cosign
signatures.

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-11-06 14:09:39 -08:00
peusebiu
9074f8483b
feat(retention): added image retention policies (#1866)
feat(metaDB): add more image statistics info

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-11-01 09:16:18 -07:00
Ramkumar Chinchani
3e6053e1db
chore: fix dependabot alerts (#1986)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-30 14:47:11 -07:00
LaurentiuNiculae
56ad9e6707
refactor(metadb): improve UX by speeding up metadb serialize/deserialize (#1842)
Use protocol buffers and update the metadb interface to better suit our search needs

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-30 13:06:04 -07:00
Alexei Dodon
d2fbd273ba
fix: tests refactoring (#1950)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-26 11:20:39 +03:00
Andreea Lupu
fc2380b57b
fix: add support for uploaded index when signing using notation (#1882)
ci(notation): update to latest notation version
fix(sync): add layers info when syncing signatures

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-10-12 18:45:20 -07:00
Andrei Aaron
ee25985c3e
chore(modules): update trivy to the tip of main (#1901)
Includes ce89d08345

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-10-09 15:12:25 -07:00
Ramkumar Chinchani
ed775914df
chore: fix dependabot alerts (#1911)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-09 11:35:42 -07:00