* feat: add redis cache support
https://github.com/project-zot/zot/pull/2005
Fixes https://github.com/project-zot/zot/issues/2004
* feat: add redis cache support
Currently, we have dynamoDB as the remote shared cache but ideal only
for the cloud use case.
For on-prem use case, add support for redis.
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
* feat(redis): added blackbox tests for redis
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
* feat(redis): dummy implementation of MetaDB interface for redis cache
Signed-off-by: Alexei Dodon <adodon@cisco.com>
* feat: check validity of driver configuration on metadb instantiation
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat: multiple fixes for redis cache driver implementation
- add missing method GetAllBlobs
- add redis cache tests, with and without mocking
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): redis implementation for MetaDB
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): use redsync to block concurrent write access to the redis DB
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): update .github/workflows/cluster.yaml to also test redis
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(metadb): add keyPrefix parameter for redis and remove unneeded method meta.Crate()
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): support RedisCluster configuration and add unit tests
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): more tests for redis metadb implementation
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): add more examples and update examples/README.md
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): move option parsing and redis client initialization under pkg/api/config/redis
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* chore(cachedb): move Cache interface to pkg/storage/types
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): reorganize code in pkg/storage/cache.go
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): call redis.SetLogger() with the zot logger as parameter
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
* feat(redis): rename pkg/meta/redisdb to pkg/meta/redis
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
---------
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Signed-off-by: Alexei Dodon <adodon@cisco.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: a <a@tuxpa.in>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
Co-authored-by: Petu Eusebiu <peusebiu@cisco.com>
Co-authored-by: Alexei Dodon <adodon@cisco.com>
wait for workers to finish before exiting
should fix tests reporting they couldn't remove rootDir because it's being
written by tasks
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
- MetaDB stores the time of the last update of a repo
- During startup we check if the layout has been updated after the last recorded change in the db
- If this is the case, the repo is parsed and updated in the DB otherwise it's skipped
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
- Cosign supports 2 types of signature formats:
1. Using tag -> each new signature of the same manifest is
added as a new layer of the signature manifest having that
specific tag("{alghoritm}-{digest_of_signed_manifest}.sig")
2. Using referrers -> each new signature of the same manifest is
added as a new manifest
- For adding these cosign signature to metadb, we reserved index 0 of the
list of cosign signatures for tag-based signatures. When a new tag-based
signature is added for the same manifest, the element on first position
in its list of cosign signatures(in metadb) will be updated/overwritten.
When a new cosign signature(using referrers) will be added for the same
manifest this new signature will be appended to the list of cosign
signatures.
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
fix(gc): fix cleaning deduped blobs because they have the modTime of
the original blobs, fixed by updating the modTime when hard linking
the blobs.
fix(gc): failing to parse rootDir at zot startup when using s3 storage
because there are no files under rootDir and we can not create empty dirs
on s3, fixed by creating an empty file under rootDir.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
1. Move existing CVE DB download generator/task login under the cve package
2. Add a new CVE scanner task generator and task type to run in the background, as well as tests for it
3. Move the CVE cache in its own package
4. Add a CVE scanner methods to check if an entry is present in the cache, and to retreive the results
5. Modify the FilterTags MetaDB method to not exit on first error
This is needed in order to pass all tags to the generator,
instead of the generator stopping at the first set of invalid data
6. Integrate the new scanning task generator with the existing zot code.
7. Fix an issue where the CVE scan results for multiarch images was not cached
8. Rewrite some of the older CVE tests to use the new image-utils test package
9. Use the CVE scanner as attribute of the controller instead of CveInfo.
Remove functionality of CVE DB update from CveInfo, it is now responsible,
as the name states, only for providing CVE information.
10. The logic to get maximum severity and cve count for image sumaries now uses only the scanner cache.
11. Removed the GetCVESummaryForImage method from CveInfo as it was only used in tests
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
- using secrets manager for storing public keys and certificates
- adding a default truststore for notation verification and upload all certificates to this default truststore
- removig `truststoreName` query param from notation api for uploading certificates
(cherry picked from commit eafcc1a213)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
