Initial code was contributed by Bogdan BIVOLARU <104334+bogdanbiv@users.noreply.github.com>
Moved implementation from a separate db to repodb by Andrei Aaron <aaaron@luxoft.com>
Not done yet:
- run/test dynamodb implementation, only boltdb was tested
- add additional coverage for existing functionality
- add web-based APIs to toggle the stars/bookmarks on/off
Initially graphql mutation was discussed for the missing API but
we decided REST endpoints would be better suited for configuration
feat(userdb): complete functionality for userdb integration
- dynamodb rollback changes to user starred repos in case increasing the total star count fails
- dynamodb increment/decrement repostars in repometa when user stars/unstars a repo
- dynamodb check anonymous user permissions are working as intendend
- common test handle anonymous users
- RepoMeta2RepoSummary set IsStarred and IsBookmarked
feat(userdb): rest api calls for toggling stars/bookmarks on/off
test(userdb): blackbox tests
test(userdb): move preferences tests in a different file with specific build tags
feat(repodb): add is-starred and is-bookmarked fields to repo-meta
- removed duplicated logic for determining if a repo is starred/bookmarked
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Co-authored-by: Andrei Aaron <aaaron@luxoft.com>
upgrade to github.com/aws/aws-sdk-go@v1.44.237
upgrade to github.com/aquasecurity/trivy@v0.38.3
upgrade to oras.land/oras-go@v1.2.3
upgrade to github.com/google/go-containerregistry@v0.14.0
upgrade to github.com/moby/buildkit@v0.11.4
Note we can't switch to trivy 0.39.0 as well as some other updates
because they would also require upgrade of cosign to v2 with
breaking api changes
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* refactor(repodb): moving common utilities under pkg/meta
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
* refactor(repodb): moved update, version components under pkg/meta
- updated wrapper initialization to recieve a log object in constructor
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
* refactor(repodb): moved repodb initialization from controller to pkg/meta/repodb
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
---------
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Changed repodb to store more information about the referrer needed for the referrers query
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
If we push an artifact and give it a tag, repodb would crash because of the null pointer dereferencing
Now when iterating over the tags of a repo and stumbling upon a unsupported media type, it's being ignored
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
This is for consistency with the other calls, and should help in making
all ZUI handling of ZOT errors consistent
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
- refactor(cve): remove the global of type cveinfo.CveInfo from the extensions package
Replace it with an attribute on controller level
- refactor(controller): extract initialization logic from controller.Run()
- test(cve): mock cve scanner in cli tests
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
The 'test' makefile target runs the tests for both 'minimal' and equivalent of the former 'extended' build.
The trivy package tests were run twice, even if the trivy logic is unreachable if search is disabled.
With this update we should see a cut of about 150s of test time.
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* docs(graphql): rewrote search.md
docs(graphql): added pagination and filter docs for gql
Squash of both commits:
(cherry picked from commit 2268fa0510b32b27f2c1f71e9889ec769877553b)
(cherry picked from commit c96adc88b2fb9edff90e7e4b01a8885511ceb0df)
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Some additional updates by Andrei Aaron to keep up with the changes on main.
Also add more comments to schema.graphql
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* style: run a graphql schemma linter as a github workflow
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
---------
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Catalin Hofnar <catalin.hofnar@gmail.com>
We are now using v2.0.1 in the cve cache logic.
Unfortunately we are also using v0.5.4 indirectly, as it is required for gqlgen, see:
e6114a2c6a/go.mod (L7)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
- derivedImageList and baseImageList now use FilterTags to obtain results,
each with its own filter function
- images that have the exact same manifest as the one provided as a
parameter are no longer considered base images or derived images
- both calls can be made with specific pagination parameters, and the
response will include PageInfo
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
fix(tests): fix one of the pagination tests
The results were not reliable as the 2 returned tags were sorted by created date/time
which was not set, resulting in an unpredictable order
Signed-off-by: Andrei Aaron <andaaron@cisco.com>
(cherry picked from commit be504200a1127371422aeb0e5c0219e2a1ead20a)
(cherry picked from commit ed8d797e639f262a63840120afe92da7db9a7600)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Signed-off-by: Andrei Aaron <andaaron@cisco.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Alex Stan <alexandrustan96@yahoo.ro>
* feat(repodb): implement pagination for ImageList and integrate it with RepoDB
- it can now return all images from all repos, when provided repo parameter is ""
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
(cherry picked from commit c003dcec9f805564946935e7eb091632f605035e)
(cherry picked from commit 72feba979b9ddd452465a652bb31f439584a046c)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* ci(timeouts): increase ci-cd workflow timeout for the build and test step
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Alex Stan <alexandrustan96@yahoo.ro>
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
add page info to dynamo-> feat(repodb): add PageInfo to GlobalSearch and RepoListWithNewestImage results
(cherry picked from commit 4fed42bb4bbc68199281d9d9a4e09b97fbd4759b)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Alex Stan <alexandrustan96@yahoo.ro>
1. chore(trivy): update trivy library version
The trivy team switched github.com/urfave/cli for viper so
there are some other code changes as well.
Since we don't use github.com/urfave/cli directly in our software
we needed to add a tools.go in order for "go mod tidy" to not delete it.
See this pattern explained in:
- https://github.com/99designs/gqlgen#quick-start
- https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module
- https://github.com/go-modules-by-example/index/blob/master/010_tools/README.md#walk-through
The jobs using "go get -u" have been updated to use "go install", since go get
modifies the go.mod by upgrading some of the packages, but downgrading trivy to an older
version with broken dependencies
2. fix(storage) Update local storage to ignore folder names not compliant with dist spec
Also updated trivy to download the DB and cache results under the rootDir/_trivy folder
3. fix(s3): one of the s3 tests was missing the skipIt call
This caused a failure when running locally without s3 being available
4. make sure the offline scanning is enabled, and zot only downloads the trivy DB
on the regular schedule, and doesn't download the DB on every image scan
ci: increase build and test timeout as tests are reaching the limit more often
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(repodb): add pagination for ImageListForDigest and implement FilterTags
ImageListForDigest can now return paginated results, directly from DB.
It uses FilterTags, a new method to filter tags (obviously) based on
the criteria provided in the filter function.
Pagination of tags is now slightly different, it shows all results if
no limit and offset are provided.
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
bug(tests): cli tests for digests expecting wrong size
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
(cherry picked from commit 369216df931a4053c18278a8d89f86d2e1e6a436)
fix(repodb): do not include repo metadata in search results if no matching tags are identified
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* fix(repodb): Fix an issue in FilterTags where repometa was not proceesed correctly
The filter function was called only once per manifest digest.
The function is supposed to also take into consideration repometa,
but only the first repometa-manifestmeta pair was processed.
Also increase code coverage.
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Alex Stan <alexandrustan96@yahoo.ro>
