Infrastructure/zot
0
Fork 0
mirror of https://github.com/project-zot/zot.git synced 2024-12-23 22:27:35 -05:00
Code Issues Activity
872 commits 12 branches 109 tags 208 MiB
Commit graph

52 commits

Author SHA1 Message Date
peusebiu
377aff1853
fix(sync): fixed skipping docker images when they already synced (#1521) 
before syncing an image we first check if it's already present in our storage
to do that we get the manifest from remote and compare it with the local one
but in the case of syncing docker images, because the conversion to OCI format is done while
syncing, we get a docker manifest before conversion, so sync detects that local manifest and
remote one are different, so it starts syncing again.

to overcome this, convert remote docker manifests to OCI manifests and then compare.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2023-06-21 11:05:52 -07:00
peusebiu
612a12e5a8
refactor(sync): use task scheduler (#1301) 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2023-05-31 10:26:23 -07:00
LaurentiuNiculae
a3f355c278
refactor(storage): refactoring storage (#1459) 
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
 2023-05-26 11:08:19 -07:00
LaurentiuNiculae
7bf40e7308
fix(sync): fixed way of updating repodb when syncing a signature (#1439) 
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
 2023-05-15 12:02:23 +03:00
LaurentiuNiculae
7d7bc9d5e4
feat(api): added oci-subject header when pushing an image with subject field (#1415) 
- as requested by the latest version of the oci distribution spec

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
 2023-05-12 09:32:01 -07:00
LaurentiuNiculae
ea79be64da
refactor(artifact): remove oci artifact support (#1359) 
* refactor(artifact): remove oci artifact support
- add header to referrers call to indicated applied artifact type filters

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(gc): simplify gc logic to increase coverage

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

---------

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
 2023-05-10 10:15:33 -07:00
peusebiu
e299ae199a
fix(sync): skip non distributable layers (#1421) 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2023-05-04 09:38:33 -07:00
Nicol
c169698c95
feat: remove usage of zerolog.Logger.Msgf() from zot code (#1382) 
Signed-off-by: Nicol Draghici <idraghic@cisco.com>
 2023-04-27 19:44:22 -07:00
Andreea Lupu
40bf76add5
chore(go.mod): upgrade trivy and cosign (#1387) 
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
 2023-04-27 09:35:10 -07:00
peusebiu
96232bb11c
test(sync): consolidate all sync tests (#1332) 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2023-04-07 09:36:27 +03:00
LaurentiuNiculae
4c156234cb
feat(repodb): sync-repodb WIP (#1241) 
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
 2023-03-09 10:41:48 -08:00
peusebiu
79783b4b06
feat(sync): skip already synced images in sync ondemand (#1234) 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2023-03-07 09:58:42 -08:00
Bogdan Bivolaru
7c3bf86a6b
refactor: Centralise extensions config entries (#1177) 
Except for registry sync config

Signed-off-by: Bogdan BIVOLARU <104334+bogdanbiv@users.noreply.github.com>
 2023-02-15 22:20:28 -08:00
Lisca Ana-Roberta
14238d4a8d
fix: removed resty calls from sync (#1016) 
Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com>
 2022-12-22 10:19:42 -08:00
peusebiu
50bdc2f402
fix(sync): also add docker v2 mediatype as supported in sync (#1084) 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-12-22 09:23:49 -08:00
peusebiu
7103953777
fix(sync): fix sync on demand with docker library (#1065) 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-12-16 09:33:46 -08:00
peusebiu
024b13efe6
fix(sync): syncing OCI artifacts with distribution package fails (#1013) 
sync OCI artifacts using REST APIs

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-12-09 11:38:00 -08:00
peusebiu
49c3d05706
fix(storage): sanitize storage locks (#1003) 
removed all locks from common code
removed locks from GetBlobContent

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-11-22 10:13:08 -08:00
peusebiu
e96c80c344
feat(sync,s3): added s3 logic for ORAS and OCI artifacts (#985) 
added sync logic for OCI artifacts

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-11-14 22:21:49 -08:00
Catalin-George Hofnar
4170d2adbc
refactor(cache): rewrote/refactored cachedb functionality to use interface (#667) 
Moved boltdb to a driver implementation for such interface
Added CreateCacheDatabaseDriver in controller
Fixed default directory creation (boltDB will only create the file, not the dir
Added coverage tests
Added example config for boltdb
Re-added caching on subpaths, rewrote CreateCacheDatabaseDriver
Fix tests
Made cacheDriver argument mandatory for NewImageStore, added more validation, added defaults
Moved cache interface to own file, removed useRelPaths from config
Got rid of cache config, refactored
Moved cache to own package and folder
Renamed + removed cache factory to backend, replaced CloudCache to RemoteCache
Moved storage constants back to storage package
moved cache interface and factory to storage package, changed remoteCache defaulting

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
 2022-11-02 15:53:08 -07:00
peusebiu
2d877aaea1
fix(sync): also sync on demand digests, not only tags, closes #902 (#932) 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-10-27 09:39:59 -07:00
Andrei Aaron
ac6c6a844c
refactor(digests): standardise representation of digests to digest.Digest (#898) 
- Digests were represented by different ways
  - We needed a uniform way to represent the digests and enforce a format
  - also replace usage of github.com/google/go-containerregistry/pkg/v1
    with github.com/opencontainers/image-spec/specs-go/v1

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit 96b2f29d6d57070a913ce419149cd481c0723815)
(cherry picked from commit 3d41b583daea654c98378ce3dcb78937d71538e8)

Co-authored-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
 2022-10-22 13:46:13 -07:00
peusebiu
5f99f9a445
fix(sync): fixed broken logic to get tags for repo (#900) 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-10-22 00:26:14 -07:00
Andrei Aaron
38b00e3507
chore(lint): gci to separate zot from other imports (#870) 
Signed-off-by: Andrei Aaron <andaaron@cisco.com>
 2022-10-20 09:39:20 -07:00
peusebiu
3c0c51fcbe
fix(sync): also sync image index mediatype (#847) 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-10-06 09:41:16 -07:00
Nicol
33a431ef43
Update go version to 1.19 (#829) 
* ci: Update go version to 1.19

Signed-off-by: Nicol Draghici <idraghic@cisco.com>

* ci: Fix lint issues

Signed-off-by: Nicol Draghici <idraghic@cisco.com>

* ci: Added needprivileges to lint, made needprivileges pass lint

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Signed-off-by: Nicol Draghici <idraghic@cisco.com>

Signed-off-by: Nicol Draghici <idraghic@cisco.com>
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Co-authored-by: Catalin Hofnar <catalin.hofnar@gmail.com>
 2022-10-05 13:21:14 +03:00
peusebiu
8237f8d20a
storage: Move common code in helper functions, closes #730 (#820) 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-09-30 10:35:16 -07:00
slab713
8ffb053cec
Replaced deprecated io/ioutil functions (#768) 
Signed-off-by: slab713 <109306207+slab713@users.noreply.github.com>
 2022-09-02 15:56:02 +03:00
Andrei Aaron
bd9ad998cd Fix file handlers not being closed after calls to ImageStore.GetBlob 
This is to fixes hitting the FD limit when reading blobs from the disk in the graphql API

Signed-off-by: Andrei Aaron <andaaron@cisco.com>
 2022-08-19 09:22:13 -07:00
Lisca Ana-Roberta
87fc941b3c image level lint: enforce manifest mandatory annotations 
closes #536

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
 2022-07-27 11:48:04 +03:00
Catalin Hofnar
a8a65a6c37 Modified sync log calls to include error type (#336) 
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
 2022-06-15 09:45:49 -07:00
Petu Eusebiu
bd730150a8 sync: allow HTTP redirects when GETing signatures blobs 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-05-25 10:19:36 -07:00
Catalin Hofnar
20a60cbad4 Enhance sync logic - stop blob redownloads and re-pushes (#479 #480) 
Changed imagesToCopyFromUpstream to return a map[string][]types.ImageReference from just an array of refs
Rewrote some logic in sync.go to use the new signature of imagesToCopyFromUpstream
Split getLocalImageRef by adding function getLocalCachePath
Adapted tests for new changes, added some tests
Merged #481

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
 2022-05-16 10:05:01 -07:00
Alex Stan
d325c8b5f4 Fix problems signaled by new linter version v1.45.2 
PR (linter: upgrade linter version #405) triggered lint job which failed
with many errors generated by various linters. Configurations were added to
golangcilint.yaml and several refactorings were made in order to improve the
results of the linter.

maintidx linter disabled

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
 2022-04-27 09:55:44 -07:00
Petu Eusebiu
89c5f4f604 sync: fix functions which compare signatures 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-04-15 10:31:37 -07:00
Petu Eusebiu
f53dc9eb8d sync: Add a new flag to enforce syncing only signed images, closes #455 
sync: When checking if a image is already synced also check for changes in upstream signatures.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-03-24 10:50:01 -07:00
laurentiuNiculae
0d148e1d6b new config option for sync-destination 
Signed-off-by: laurentiuNiculae <themelopeus@gmail.com>
 2022-03-21 08:12:34 -07:00
Petu Eusebiu
6d04ab3cdc sync: support reloading sync config when the config file changes 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-03-18 11:24:11 -07:00
Petu Eusebiu
45968e0bb7 sync: fix inconsistent test, used inject error fw for hard to reach test cases 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-03-01 09:56:27 -08:00
Ramkumar Chinchani
38a110314b gc: add a gcDelay param 
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
 2022-02-14 14:03:22 -08:00
Petu Eusebiu
35eeedb22a sync: pull only missing images, not everything, closes #335 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-02-14 08:09:18 -08:00
Ramkumar Chinchani
d2aa016cdb storage: flush/sync contents to disk on file close 
Behavior controlled by configuration (default=off)
It is a trade-off between performance and consistency.

References:
[1] https://github.com/golang/go/issues/20599

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
 2022-02-07 12:08:01 -08:00
Petu Eusebiu
f89925fb27 sync: periodically retry if on-demand fails inline, closes #281 
sync: don't return error on sync signatures, just skip them, closes #375
sync: sync signatures on demand
sync on demand: in case of parallel requests pull image just once, closes #344

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-02-07 09:04:13 -08:00
Petu Eusebiu
a0e65379c8 sync: for a prefix, allow multiple registries as a list instead of only one, closes #343 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-02-01 09:45:09 -08:00
Petu Eusebiu
b9250a783a Use InsecureSkipVerify only with https upstreams 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-01-29 20:00:27 -08:00
Petu Eusebiu
1109bb4dde sync: Added support for syncing notary/cosign signatures, closes #261 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2022-01-13 08:45:59 -08:00
Ramkumar Chinchani
ac3801ea2d lint: upgrade golangci-lint 
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
 2021-12-20 17:20:35 -08:00
Petu Eusebiu
63a75216ed sync: allow for saving to a subpath, closes #307 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2021-12-07 10:13:20 -08:00
Ramkumar Chinchani
96226af869 move references to zotregistry.io and project-zot 
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
 2021-12-05 10:52:27 -08:00
Petu Eusebiu
fff6107310 Sync prefix can be an exact match or a glob pattern, closes #297 
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
 2021-11-29 13:10:13 -08:00