0
Fork 0
mirror of https://github.com/project-zot/zot.git synced 2024-12-16 21:56:37 -05:00
Commit graph

668 commits

Author SHA1 Message Date
Andrei Aaron
33b55efc92
fix(ci): do not build zot container image for the annotation tests (#1110)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-01-13 22:50:20 +02:00
Ramkumar Chinchani
ca1d4beb2a
ci: also allow manual workflow triggers (#1109)
* ci: also allow manual workflow triggers

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* ci: add a github actions checker

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-01-13 00:49:42 -08:00
Andrei Aaron
9f0b833293
fix(ci): use oras login before copying trivy-db
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-01-12 18:44:45 +00:00
Andrei Aaron
787a75e218
fix(ci): fix path to trivy-db copy under project-zot (#1108)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-01-12 20:01:43 +02:00
Andrei Aaron
6f5b7afd39
ci(trivy): copy trivydb oci artefact to project-zot repo (#1106)
This will replace calls made directly to ghcr.io/aquasecurity/trivy-db
Which are hitting the rate limiter

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-01-12 09:54:53 -08:00
Lisca Ana-Roberta
261fc2fb38
fix: added error message for missing CVEs (#1085)
Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com>
2023-01-11 15:02:29 -08:00
Ravi Chamarthy
052ad8b12e
fix: add stacker to .gitignore (#1091)
Signed-off-by: Ravi Chamarthy <ravchama@cisco.com>
2023-01-10 20:56:54 -08:00
Ramkumar Chinchani
6ab27fcdcd
chore(deps): fix dependabot alerts (#1098)
https://github.com/project-zot/zot/pull/1094
https://github.com/project-zot/zot/pull/1095
https://github.com/project-zot/zot/pull/1096
https://github.com/project-zot/zot/pull/1097

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-01-10 16:17:19 -08:00
Ramkumar Chinchani
3a6c3552cd
docs: fix CNCF related documentation (#1099)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-01-10 15:52:11 -08:00
Lisca Ana-Roberta
f264f26908
fix: error message when CVE search is disabled (#1100)
Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com>
2023-01-10 10:45:15 -08:00
LaurentiuNiculae
f408df0dac
feat(repodb): Implement RepoDB for image specific information using boltdb/dynamodb (#979)
* feat(repodb): implement a DB for image specific information using boltdb

(cherry picked from commit e3cb60b856)

Some other fixes/improvements on top (Andrei)

Global search: The last updated attribute on repo level is now computed correctly.
Global search: Fix and enhance tests: validate more fields, and fix CVE verification logic
RepoListWithNewestImage: The vendors and platforms at repo level are no longer containing duplicate entries
CVE: scan OCIUncompressedLayer instead of skiping them (used in tests)
bug(repodb): do no try to increment download counters for signatures

Signed-off-by: Andrei Aaron <andaaron@cisco.com>

Add filtering to global search API (Laurentiu)

(cherry picked from commit a87976d635ea876fe8ced532e8adb7c3bb24098f)

Original work by Laurentiu Niculae <niculae.laurentiu1@gmail.com>

Fix pagination bug

 - when limit was bigger than the repo count result contained empty results
 - now correctly returns only maximum available number of repo results

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

Add history to the fields returned from RepoDB

Consolidate fields used in packages
- pkg/extensions/search/common/common_test
- pkg/extensions/search/common/common
Refactor duplicate code in GlobalSearch verification
Add vulnerability scan results to image:tag reply

Signed-off-by: Andrei Aaron <andaaron@cisco.com>

Refactor ExpandedRepoInfo to using RepoDB

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit fd7dc85c3a9d028fd8860d3791cad4df769ed005)

Init RepoDB at startup
 - sync with storage
 - ignore images without a tag

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit 359898facd6541b2aa99ee95080f7aabf28c2650)

Update request to get image:tag to use repodb

Signed-off-by: Andrei Aaron <andaaron@cisco.com>

Sync RepoDB logging
 - added logging for errors

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit 2e128f4d01712b34c70b5468285100b0657001bb)

sync-repodb minor error checking fix

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

Improve tests for syncing RepoDB with storage

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit b18408c6d64e01312849fc18b929e3a2a7931e9e)

Update scoring rule for repos
  - now prioritize matches to the end of the repo name

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit 6961346ccf02223132b3b12a2132c80bd1b6b33c)

Upgrade search filters to permit multiple values
  - multiple values for os and arch

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit 3ffb72c6fc0587ff827a03fe4f76a13b27b876a0)

feature(repodb): add pagination for RepoListWithNewestImage

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
(cherry picked from commit 32c917f2dc65363b0856345289353559a8027aee)

test(fix): fix tests failing since repodb is used for listing all repos

1. One of the tests was verifying disk/oci related erros and is not applicable
2. Another test was actually broken in an older PR, the default store and
the substore were using the same repo names (the substore ones were unprefixed),
which should not be the case, this was causing a single entry to show
in the RepoDB instead of two separate entries for each test image
Root cause in: b61aff62cd (diff-b86e11fa5a3102b336caebec3b30a9d35e26af554dd8658f124dba2404b7d24aR88)

Signed-off-by: Andrei Aaron <andaaron@cisco.com>

chore: move code reponsible for transforming objects to gql_generated types to separate package

Signed-off-by: Andrei Aaron <andaaron@cisco.com>

Process input for global search
  - Clean input: query, filter strings
  - Add validation for global search input

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit f1ca8670fbe4a4a327ea25cf459237dbf23bb78a)

fix: only call cve scanning for data shown to the user

Signed-off-by: Andrei Aaron <andaaron@cisco.com>

GQL omit scanning for CVE if field is not required

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit 5479ce45d6cb2abcf5fbccadeaf6f3393c3f6bf1)

Fix filtering logic in RepoDB
  - filter parameter was set to false instead of being calculator from the later image

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit a82d2327e34e5da617af0b7ca78a2dba90999f0a)

bug(repodb): Checking signature returns error if signed image is not found
  - we considere a signature image orfan when the image it signs is not found
  - we need this to ignore such signatures in certain cases

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit d0418505f76467accd8e1ee34fcc2b2a165efae5)

feat(repodb): CVE logic to use repoDB

Also update some method signatures to remove usage of:
github.com/google/go-containerregistry/pkg/v1

Signed-off-by: Andrei Aaron <andaaron@cisco.com>

* feat(repodb): refactor repodb update logic

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* fix(repodb): minor fixes

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(repodb): move repodb logic inside meta directory under pkg

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(repodb): change factory class for repodb initialization with factory metrod

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(repodb): simplify repodb configuration
  - repodb now shares config parameters with the cache
  - config taken directly from storage config

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* fix(authors): fix authors information to work properly with repodb

Ideally this commit would be squshed in the repodb commit
but as-is it is easier to cherry-pick on other branches

Signed-off-by: Andrei Aaron <andaaron@cisco.com>

* feat(repodb): dynamodb support for repodb
  - clean-up repodb code + coverage improvements

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(dynamo): tables used by dynamo are created automatically if they don't exists
  - if the table exists nothing happens

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* test(repodb): coverage tests
  - minor fix for CVEListForImage to fix the tests
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(repodb): add descriptor with media type

  - to represent images and multi-arch images

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(repodb): support signatures on repo level

  - added to follow the behavior of signing and signature verification tools
    that work on a manifest level for each repo
  - all images with different tags but the same manifest will be signed at once

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(repodb): old repodb version migration support

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(repodb): tests for coverage

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(repodb): WIP fixing tests

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(repodb): work on patchRepoDB tests

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* fix(repodb): create dynamo tables only for linux amd

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* fix(ci): fix a typo in ci-cd.yml

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>

Signed-off-by: Andrei Aaron <andaaron@cisco.com>
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Andrei Aaron <andaaron@cisco.com>
Co-authored-by: Andrei Aaron <aaaron@luxoft.com>
2023-01-09 12:37:44 -08:00
peusebiu
f69b104838
fix(test): use correct aws region for dynamodb (#1093)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-01-09 09:04:56 -08:00
Ramkumar Chinchani
08a8b3d44c
chore(deps): fix dependabot alerts (#1090)
https://github.com/project-zot/zot/pull/1087
https://github.com/project-zot/zot/pull/1088
https://github.com/project-zot/zot/pull/1089
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-12-29 10:05:22 +02:00
peusebiu
0e7b49996a
ci: start localstack container only if needed (#1086)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-12-23 11:19:51 -08:00
Lisca Ana-Roberta
14238d4a8d
fix: removed resty calls from sync (#1016)
Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com>
2022-12-22 10:19:42 -08:00
peusebiu
50bdc2f402
fix(sync): also add docker v2 mediatype as supported in sync (#1084)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-12-22 09:23:49 -08:00
Nicol
85a7053d9c
refactor: Cleanup/simplify tests: uploads (#1082)
Signed-off-by: Nicol Draghici <idraghic@cisco.com>
2022-12-21 13:33:55 -08:00
Ramkumar Chinchani
4f4b676b50
docs: update README.md (#1078)
All the docs have been relocated to https://zotregistry.io

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-12-21 10:46:31 +02:00
Ramkumar Chinchani
271ffb53cc
chore(deps): fix dependabot alerts (#1080)
https://github.com/project-zot/zot/pull/1079
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-12-20 15:29:21 -08:00
Ramkumar Chinchani
1c9db99b0d
chore(deps): fix dependabot alerts (#1077)
https://github.com/project-zot/zot/pull/1071
https://github.com/project-zot/zot/pull/1075
https://github.com/project-zot/zot/pull/1076


Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-12-20 11:59:04 -08:00
Ramkumar Chinchani
2571e8e5fc
chore(deps): fix dependabot alerts (#1074)
https://github.com/project-zot/zot/pull/1069
https://github.com/project-zot/zot/pull/1070
https://github.com/project-zot/zot/pull/1071
https://github.com/project-zot/zot/pull/1072
https://github.com/project-zot/zot/pull/1073


Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-12-19 14:59:31 -08:00
peusebiu
7103953777
fix(sync): fix sync on demand with docker library (#1065)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-12-16 09:33:46 -08:00
Nicol
05f75e041c
refactor: Cleanup/simplify test cases (#1041)
Signed-off-by: Nicol Draghici <idraghic@cisco.com>
2022-12-15 13:22:18 -08:00
Ramkumar Chinchani
9136479206
docs: add pkg.go.dev badge (#1061)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-12-13 09:05:04 -08:00
Ramkumar Chinchani
2efa80d1ec
chore(deps): fix dependabot alerts (#1060)
https://github.com/project-zot/zot/pull/1057
https://github.com/project-zot/zot/pull/1058
https://github.com/project-zot/zot/pull/1059
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-12-12 14:09:34 -08:00
junya koyama
f201678943 fix(swagger): update docs.go - fix typos
Signed-off-by: junya koyama <arukiidou@yahoo.co.jp>
2022-12-12 17:47:49 +02:00
Andreea Lupu
ec05137eda
feat(scrub): add scrub logic for ImageIndex media type (#1031)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2022-12-10 16:38:01 -08:00
alexstan12
8fb11180d4
fix: queries with images as arguments without a reference should return an error (#1040)
Currently there is no push-back on queries that should contain image names but
have only the repo name. This commit adds a check that will return an error for images
w/o a reference(tag or digest).

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-12-09 11:40:06 -08:00
peusebiu
024b13efe6
fix(sync): syncing OCI artifacts with distribution package fails (#1013)
sync OCI artifacts using REST APIs

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-12-09 11:38:00 -08:00
Ramkumar Chinchani
37e6c6db0e
ci: remove superfluous parts of github workflows (#1050)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-12-07 12:46:50 -08:00
peusebiu
cbf88bd668
fix(test): consolidate flaky scrub test (#1042)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-12-07 11:04:33 -08:00
Ramkumar Chinchani
d78f1d962f
chore(deps): fix dependabot alerts (#1048)
https://github.com/project-zot/zot/pull/1043
https://github.com/project-zot/zot/pull/1044
https://github.com/project-zot/zot/pull/1045
https://github.com/project-zot/zot/pull/1046
https://github.com/project-zot/zot/pull/1047

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-12-06 09:09:03 -08:00
Ramkumar Chinchani
69f0cf6bb4
fix(config): warn if cve is used with remote storage driver (#1034)
* fix(config): warn if cve is used with remote storage driver

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>

* fix: also check if search is enabled

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Co-authored-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2022-11-30 09:35:06 +02:00
Ramkumar Chinchani
72abab4947
perf(zb): cleanup tool output (#1033)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-11-29 14:04:36 -08:00
Ramkumar Chinchani
ffa223de43
chore(deps): fix dependabot alerts (#1030)
https://github.com/project-zot/zot/pull/1024
https://github.com/project-zot/zot/pull/1025
https://github.com/project-zot/zot/pull/1026
https://github.com/project-zot/zot/pull/1027
https://github.com/project-zot/zot/pull/1028

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-11-29 16:41:01 +02:00
peusebiu
c82748ab39
Upgrade oras cli (#1021)
* chore: upgrade oras cli

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

* fix(oci references): referrers API should not return 404 if repo exists

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-11-28 10:43:53 -08:00
peusebiu
1d97d1e796
feat: gc untagged manifests and signatures without references (#948)
closes #906 #907

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-11-24 11:00:22 -08:00
peusebiu
4753a0f587
fix(routes): set Content-Length on GetManifest (#1019)
notation sign image index exits with "missing Content-Length" error

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-11-24 10:53:48 -08:00
alexstan12
f75bce3085
feat(graphql): add an api to return referrers (#1009)
UI can now make use of OCI artifacts and references using `Referrers` gQL query.
It returns a list of descriptors that refer on their `subject` field to another
digest.

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-11-23 10:53:28 -08:00
Ramkumar Chinchani
8746a49268
fix: handle remote cache driver misconfig (#1011)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-11-23 00:19:15 -08:00
Ramkumar Chinchani
070941b1b3
fix(oras): relax artifactType check to allow other types (#1010)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-11-23 00:15:33 -08:00
Catalin-George Hofnar
31b9481713
feat(cache): dynamodb implementation (#953)
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2022-11-22 10:29:57 -08:00
peusebiu
49c3d05706
fix(storage): sanitize storage locks (#1003)
removed all locks from common code
removed locks from GetBlobContent

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-11-22 10:13:08 -08:00
peusebiu
6ee3d802e7
test(sync): added blackbox tests for sync (#1008)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-11-22 10:10:32 -08:00
peusebiu
168d21da1e
fix(storage): deleting manifests with identical digests (#951)
Suppose we push two identical manifests (sharing same digest) but with
different tags, then deleting by digest should throw an error otherwise
we end up deleting all image tags (with gc) or dangling references
(without gc)

This behaviour is controlled via Authorization, added a new policy
action named detectManifestsCollision which enables this behaviour

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-11-18 09:35:28 -08:00
Lisca Ana-Roberta
4e13619dc8
fix: signatures now showing correctly (#954)
Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com>
2022-11-15 09:06:25 -08:00
peusebiu
e96c80c344
feat(sync,s3): added s3 logic for ORAS and OCI artifacts (#985)
added sync logic for OCI artifacts

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-11-14 22:21:49 -08:00
Ramkumar Chinchani
fb3e4ec2ef
docs: fix README.md 2022-11-14 14:42:58 -08:00
Ramkumar Chinchani
b473d34ca9
docs: update README with minimal image and urls (#1001)
Thanks @dtzar for the initial PR.
https://github.com/project-zot/zot/pull/992

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-11-14 14:03:36 -08:00
Ramkumar Chinchani
7790b944e3
chore(deps): fix dependabot alerts (#1000)
https://github.com/project-zot/zot/pull/996
https://github.com/project-zot/zot/pull/997
https://github.com/project-zot/zot/pull/998
https://github.com/project-zot/zot/pull/999

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-11-14 12:29:28 -08:00