0
Fork 0
mirror of https://github.com/project-zot/zot.git synced 2024-12-23 22:27:35 -05:00
Commit graph

52 commits

Author SHA1 Message Date
Ramkumar Chinchani
c0f93caacb
feat(artifact): add OCI references support (#936)
Thanks @jdolitsky et al for kicking off these changes at:
https://github.com/oci-playground/zot/commits/main

Thanks @sudo-bmitch for reviewing the patch

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-11-08 00:38:16 -08:00
Catalin-George Hofnar
4170d2adbc
refactor(cache): rewrote/refactored cachedb functionality to use interface (#667)
Moved boltdb to a driver implementation for such interface
Added CreateCacheDatabaseDriver in controller
Fixed default directory creation (boltDB will only create the file, not the dir
Added coverage tests
Added example config for boltdb
Re-added caching on subpaths, rewrote CreateCacheDatabaseDriver
Fix tests
Made cacheDriver argument mandatory for NewImageStore, added more validation, added defaults
Moved cache interface to own file, removed useRelPaths from config
Got rid of cache config, refactored
Moved cache to own package and folder
Renamed + removed cache factory to backend, replaced CloudCache to RemoteCache
Moved storage constants back to storage package
moved cache interface and factory to storage package, changed remoteCache defaulting

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2022-11-02 15:53:08 -07:00
Catalin-George Hofnar
c6ffbce6cf
refactor(tests): removed globals from digest test, removed some more hardcoded digests (#923)
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2022-10-26 19:14:16 +03:00
Andrei Aaron
ac6c6a844c
refactor(digests): standardise representation of digests to digest.Digest (#898)
- Digests were represented by different ways
  - We needed a uniform way to represent the digests and enforce a format
  - also replace usage of github.com/google/go-containerregistry/pkg/v1
    with github.com/opencontainers/image-spec/specs-go/v1

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
(cherry picked from commit 96b2f29d6d57070a913ce419149cd481c0723815)
(cherry picked from commit 3d41b583daea654c98378ce3dcb78937d71538e8)

Co-authored-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2022-10-22 13:46:13 -07:00
Ramkumar Chinchani
763287873e
fix(config): make all extension config consistent (#888)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-10-21 15:33:54 +03:00
Andrei Aaron
38b00e3507
chore(lint): gci to separate zot from other imports (#870)
Signed-off-by: Andrei Aaron <andaaron@cisco.com>
2022-10-20 09:39:20 -07:00
peusebiu
91dd5496fc
chore: rename search route prefix (#887)
* chore: rename search route prefix

* chore: use builtin time.Duration.Truncate() on latencies

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-10-18 20:46:06 -07:00
Catalin Hofnar
caf88ddb1e
refactor(tests): remove hardcoded digests (#871)
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2022-10-18 20:43:56 -07:00
Andrei Aaron
815366024b
fix(lastUpdated): fix image lastUpdated timestamp logic (#863)
The lastUpdated field was picked from the first entry in image history
Now it is the created time of the image, or the last entry in
image history, if created time is unavailable

Signed-off-by: Andrei Aaron <andaaron@cisco.com>
2022-10-10 17:43:05 -07:00
Nicol
33a431ef43
Update go version to 1.19 (#829)
* ci: Update go version to 1.19

Signed-off-by: Nicol Draghici <idraghic@cisco.com>

* ci: Fix lint issues

Signed-off-by: Nicol Draghici <idraghic@cisco.com>

* ci: Added needprivileges to lint, made needprivileges pass lint

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Signed-off-by: Nicol Draghici <idraghic@cisco.com>

Signed-off-by: Nicol Draghici <idraghic@cisco.com>
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
Co-authored-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2022-10-05 13:21:14 +03:00
peusebiu
8237f8d20a
storage: Move common code in helper functions, closes #730 (#820)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-09-30 10:35:16 -07:00
Bogdan Bivolaru
67294cc669
Add graphql query for retrieving imgSummary based on repo:tag image id. (#814)
Refactor Image GqlResolver to better suit GetManifest.
Changed GetManifest to also return digest.

Signed-off-by: Bogdan BIVOLARU <104334+bogdanbiv@users.noreply.github.com>
2022-09-30 10:32:32 -07:00
Andrei Aaron
e0d808b196
Include image vulnerability information in ImageSummary (#798)
Return this data as part of GlobalSearch and RepoListWithNewestImage
query results.
This commit also includes refactoring of the CVE scanning logic in
order to better encapsulate trivy specific logic, remove CVE scanning
logic from the graphql resolver.

Signed-off-by: Andrei Aaron <andaaron@cisco.com>
2022-09-28 11:39:54 -07:00
Lisca Ana-Roberta
0f7b174fc0
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com> (#713)
list all images that have are base images for the given image + zli command

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
2022-09-23 09:23:31 -07:00
Andrei Aaron
7517f2a5bb
list all images that have all layers of the base image included (2) (#813)
* list all images that are base images for the given image + zli command

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

* Fix a failing test

The test expected the image size to be the size of the layer, not the manifest+config+layer

Signed-off-by: Andrei Aaron <andaaron@cisco.com>

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
Signed-off-by: Andrei Aaron <andaaron@cisco.com>
Co-authored-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
2022-09-22 22:08:58 +03:00
Alex Stan
42c947fa03 Enrich ImageSummary with a new field representing image History
Closing #748

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-09-21 12:50:11 -07:00
Petu Eusebiu
601e4fcad4 graphql: Populate ImageSummary missing fields:
Description, Labels, Licenses, Title, Documentation, Source

closes #786

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-09-20 11:04:45 -07:00
slab713
8ffb053cec
Replaced deprecated io/ioutil functions (#768)
Signed-off-by: slab713 <109306207+slab713@users.noreply.github.com>
2022-09-02 15:56:02 +03:00
Alex Stan
6ae793eb51 skip manifests inside index.json that don't have an tag annotation
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-09-01 10:32:19 -07:00
Catalin Hofnar
9ca5fa1029 Implement RepoListWithNewestImage to return [RepoSummary]
Removed access by index in repoListWithNewestImage

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2022-08-29 13:06:17 +03:00
Alex Stan
49e8167dbe graphql: Apply authorization on /_search endpoint
- AccessControlContext now resides in a separate package from where it can be imported,
along with the contextKey that will be used to set and retrieve this context value.

- AccessControlContext has a new field called Username, that will be of use for future
implementations in graphQL resolvers.

- GlobalSearch resolver now uses this context to filter repos available to the logged user.

- moved logic for uploading images in tests so that it can be used in every package

- tests were added for multiple request scenarios, when zot-server requires authz
on specific repos

- added tests with injected errors for extended coverage

- added tests for status code error injection utilities

Closes https://github.com/project-zot/zot/issues/615

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-08-26 21:31:26 +03:00
Ramkumar Chinchani
2ff8e8b7d2 fix dependabot alerts
https://github.com/project-zot/zot/pull/737
https://github.com/project-zot/zot/pull/738
https://github.com/project-zot/zot/pull/739
https://github.com/project-zot/zot/pull/740

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-08-23 09:38:30 -07:00
Roxana Nemulescu
ab9a20c1ae Add GraphQL API for getting the information necessary to list images in the zot cli without download manifests.
If this GraphQL API is available, try that first, else fallback to the slowpath.

Signed-off-by: Roxana Nemulescu <roxana.nemulescu@gmail.com>
2022-08-23 16:32:00 +03:00
Ramkumar Chinchani
4b8e288cd3 fix dependabot alerts
https://github.com/project-zot/zot/pull/712
https://github.com/project-zot/zot/pull/714

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-08-13 00:02:36 -07:00
Alex Stan
a5ed99178e replace dependency of tagsInfo and repoInfo with just a list of manifests
- replace dependency of tagsInfo and repoInfo with a list of manifests, since it provides
all the needed data
- Mock tests added

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-08-09 17:54:18 -07:00
Alex Stan
0c70ae8a4e RepoInfo structure now includes new field representing RepoSummary
ExpandedRepoInfo currently returns RepoInfo that is a list of Manifests.
To comply with the newest UI requirements, a new field called Summary,
referring to RepoSummary structure, was added.

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-08-05 19:22:22 +03:00
Andrei Aaron
b5f27c5b50 RepoSummary has a new attribute NewestTag of type ImageSummary
ImageListWithLatestTag currently returns a list of ImageInfo objects.
It needs to return consistent results with the API used for Global search as the same information will be used by the UI in the same type or cards.
So we need to update RepoSummary to include the data which right now is present in ImageInfo, but missing from RepoSummary (information on the latest tag in that specific repo).
Will update return type of ImageListWithLatestTag in a later PR (issue tracked in a separate GH issue)

Closes #666

Signed-off-by: Andrei Aaron <andaaron@cisco.com>
2022-07-27 19:41:00 +03:00
Lisca Ana-Roberta
87fc941b3c image level lint: enforce manifest mandatory annotations
closes #536

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
2022-07-27 11:48:04 +03:00
Laurentiu Niculae
58f8cd5d7d test calculated size
Signed-off-by: Laurentiu Niculae <themelopeus@gmail.com>
2022-07-20 10:03:11 -07:00
Laurentiu Niculae
80369140f1 add image info to parameter
Signed-off-by: Laurentiu Niculae <themelopeus@gmail.com>
2022-07-20 10:03:11 -07:00
Laurentiu Niculae
7e3d063319 freeform querry api
Signed-off-by: Laurentiu Niculae <themelopeus@gmail.com>
2022-07-20 10:03:11 -07:00
Andrei Aaron
43160dcc43 Update to graphql 1.17.13
We encountered some problems with using the existing folder structure,
but it looks like running the tooling with the latest versions works after
we regenerated the project using 'gql init' and refactoring to separate
the login previously in resolvers.go.

- the autogenerated code is now under the gql_generated folder
- the file resolvers.go now contains only the code which is not
rewritten by the gqlgen framework
- the file schema.resolvers.go is rewritten when gqlgen runs,
and we'll only keep there the actual resolvers matching query names
Changes we observed to schema.resolvers.go when gqlgen runs include
reordering methods, and renaming function parameters to match the
names used in schema.graphql
- we now have a gqlgen.yaml config file which governs the behavior of
gqlgen (can be tweaked to restructure the folder structure of the
generated code in the future)

Looks like the new graphql server has better validation
1 Returns 422 instead of 200 for missing query string - had to update tests
2 Correctly uncovered an error in a test for a bad `%` in query string.

As as result of 2, a `masked` bug was found in the way we check if images are
signed with Notary, the signatures were reasched for with the media type
of the image manifest itself instead of the media type for notation.
Fixed this bug, and improved error messages.
This bug would have also been reproducible with main branch if the bad `%`
in the test would have fixed.

Updated the linter to ignore some issues with the code which is
always rewritten when running:
`go run github.com/99designs/gqlgen@v0.17.13 generate`

Add a workflow to test gqlgen works and has no uncommitted changes

Signed-off-by: Andrei Aaron <andaaron@cisco.com>
2022-07-18 12:55:40 -07:00
Alex Stan
ada21ed842 Manage builds with different combinations of extensions
Files were added to be built whether an extension is on or off.
New build tags were added for each extension, while minimal and extended disappeared.

added custom binary naming depending on extensions used and changed references from binary to binary-extended

added automated blackbox tests for sync, search, scrub, metrics

added contributor guidelines

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-06-30 09:53:52 -07:00
Shivam Mishra
b61aff62cd check notary v2 signature while looking for available signatures
expanded repo info also provides information if manifests of repo is signed or not
previously it was looking for only cosign signature.

Signed-off-by: Shivam Mishra <shimish2@cisco.com>
2022-06-03 17:45:22 -07:00
Lisca Ana-Roberta
62775cc095 fixed failed tests for all skopeo versions
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

skopeo verifications

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

skopeo verifications modified makefile

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

modified how to get digest and fixed makefile

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

fixed failed tests for all skopeo versions

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

echo skopeo version

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

skopeo verifications

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

skopeo verifications modified makefile

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

modified how to get digest and fixed makefile

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

skopeo failed tests fixed

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

changed function name

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

fixed lost modifications

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

fixed code coverage and dead code

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
2022-05-27 08:19:07 -07:00
Shivam Mishra
36c9631000 ext: use distribution spec route prefix for extension api
Following the spec defined here https://github.com/opencontainers/distribution-spec/tree/main/extensions

Signed-off-by: Shivam Mishra <shimish2@cisco.com>
2022-05-22 16:35:16 -07:00
Alex Stan
d325c8b5f4 Fix problems signaled by new linter version v1.45.2
PR (linter: upgrade linter version #405) triggered lint job which failed
with many errors generated by various linters. Configurations were added to
golangcilint.yaml and several refactorings were made in order to improve the
results of the linter.

maintidx linter disabled

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-04-27 09:55:44 -07:00
Petu Eusebiu
be910cf01c lint: Move out config reloader context from controller struct
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-03-24 10:50:01 -07:00
Eng Zer Jun
0d77b60de7 test: use T.TempDir to create temporary test directory
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-03-07 12:28:49 -08:00
Ramkumar Chinchani
38a110314b gc: add a gcDelay param
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-02-14 14:03:22 -08:00
Shivam Mishra
37d150e32f search: graphql api to give detailed repo info
DetailedRepoInfo graphql api returns detailed repo info given repo name
repo contains its manifests info
Each manifest entry contains digest,signed, tag and layers info
Each layer info containes digest, size

Signed-off-by: Shivam Mishra <shimish2@cisco.com>
2022-02-10 16:34:13 -08:00
Alexei Dodon
47c9b6244e Added config enable=true/false for extensions with default value as enabled closes #258
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2022-02-09 09:53:49 -08:00
Ramkumar Chinchani
d2aa016cdb storage: flush/sync contents to disk on file close
Behavior controlled by configuration (default=off)
It is a trade-off between performance and consistency.

References:
[1] https://github.com/golang/go/issues/20599

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-02-07 12:08:01 -08:00
Ramkumar Chinchani
87084f286b storage: improve/fix oci image validation
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-02-02 13:31:41 -08:00
Ramkumar Chinchani
ac3801ea2d lint: upgrade golangci-lint
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2021-12-20 17:20:35 -08:00
Ramkumar Chinchani
96226af869 move references to zotregistry.io and project-zot
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2021-12-05 10:52:27 -08:00
Alexei Dodon
e900b09cfb Fix data races in tests, closes #255
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2021-11-17 13:23:59 -08:00
Alexei Dodon
8e4d828867 Implement an API for performance monitoring
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2021-11-12 11:14:10 -08:00
Petu Eusebiu
19003e8a71 Added new extension "sync"
Periodically poll registries and pull images according to sync's config
Added sync on demand, syncing when clients asks for an image which
zot doesn't have.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2021-10-21 10:32:46 -07:00
Petu Eusebiu
7d077eaf5a Added storage interface 2021-10-11 17:38:46 -07:00