0
Fork 0
mirror of https://github.com/project-zot/zot.git synced 2024-12-16 21:56:37 -05:00
Commit graph

356 commits

Author SHA1 Message Date
Alexei Dodon
2fd7bfc37a
fix: metrics endpoint must be secured behind authN (#1864)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-02 16:37:21 +03:00
Andrei Aaron
ba6f347d8d
refactor(pkg/test): split logic in pkg/test/common.go into multiple packages (#1861)
Which could be imported independently. See more details:
1. "zotregistry.io/zot/pkg/test/common" - currently used as
   tcommon "zotregistry.io/zot/pkg/test/common" - inside pkg/test
   test "zotregistry.io/zot/pkg/test/common" - in tests
   . "zotregistry.io/zot/pkg/test/common" - in tests
Decouple zb from code in test/pkg in order to keep the size small.

2. "zotregistry.io/zot/pkg/test/image-utils" - curently used as
   . "zotregistry.io/zot/pkg/test/image-utils"

3. "zotregistry.io/zot/pkg/test/deprecated" -  curently used as
   "zotregistry.io/zot/pkg/test/deprecated"
This one will bre replaced gradually by image-utils in the future.

4. "zotregistry.io/zot/pkg/test/signature" - (cosign + notation) use as
   "zotregistry.io/zot/pkg/test/signature"

5. "zotregistry.io/zot/pkg/test/auth" - (bearer + oidc)  curently used as
   authutils "zotregistry.io/zot/pkg/test/auth"

 6. "zotregistry.io/zot/pkg/test/oci-utils" -  curently used as
   ociutils "zotregistry.io/zot/pkg/test/oci-utils"

Some unused functions were removed, some were replaced, and in
a few cases specific funtions were moved to the files they were used in.

Added an interface for the StoreController, this reduces the number of imports
of the entire image store, decreasing binary size for tests.
If the zb code was still coupled with pkg/test, this would have reflected in zb size.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-27 11:34:48 -07:00
Andreea Lupu
92e382ce39
refactor(scrub): replace umoci logic in scrub implementation (#1845)
- implement scrub also for S3 storage by replacing umoci
- change scrub implementation for ImageIndex
- take the `Subject` into consideration when running scrub
- remove test code relying on the umoci library. Since we started
relying on images in test/data, and we create our own images using
go code we can obtain digests by other means. (cherry picked from commit 489d4e2d23c1b4e48799283f8281024bbef6123f)

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-09-26 11:02:11 -07:00
Ramkumar Chinchani
9096031aeb
chore: fix dependabot alerts (#1855)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-09-25 23:03:13 +03:00
Andrei Aaron
6bd7abe28b
fix(tests): call ImageStore constructor with correct parameters (#1846)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-23 07:58:58 +00:00
peusebiu
1df743f173
fix(gc): sync repodb when gc'ing manifests (#1819)
fix(gc): fix cleaning deduped blobs because they have the modTime of
the original blobs, fixed by updating the modTime when hard linking
the blobs.
fix(gc): failing to parse rootDir at zot startup when using s3 storage
because there are no files under rootDir and we can not create empty dirs
on s3, fixed by creating an empty file under rootDir.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-22 11:51:20 -07:00
Andrei Aaron
7c78f80a96
feat(cve): implement CVE scanning as background tasks (#1833)
1. Move existing CVE DB download generator/task login under the cve package
2. Add a new CVE scanner task generator and task type to run in the background, as well as tests for it
3. Move the CVE cache in its own package
4. Add a CVE scanner methods to check if an entry is present in the cache, and to retreive the results
5. Modify the FilterTags MetaDB method to not exit on first error
This is needed in order to pass all tags to the generator,
instead of the generator stopping at the first set of invalid data
6. Integrate the new scanning task generator with the existing zot code.
7. Fix an issue where the CVE scan results for multiarch images was not cached
8. Rewrite some of the older CVE tests to use the new image-utils test package
9. Use the CVE scanner as attribute of the controller instead of CveInfo.
Remove functionality of CVE DB update from CveInfo, it is now responsible,
as the name states, only for providing CVE information.
10. The logic to get maximum severity and cve count for image sumaries now uses only the scanner cache.
11. Removed the GetCVESummaryForImage method from CveInfo as it was only used in tests

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-22 11:49:17 -07:00
Andrei Aaron
a11fe2d195
feat(pprof): add profiling route handler to debug runtime (#1818)
(cherry picked from commit 56ddb70f624e7070ad0d3531d498675f9f82c664)

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Co-authored-by: Alex Stan <alexandrustan96@yahoo.ro>
2023-09-18 14:05:41 -07:00
Andrei Aaron
bcdd9988f5
fix(cve): cummulative fixes and improvements for CVE scanning logic (#1810)
1. Only scan CVEs for images returned by graphql calls
Since pagination was refactored to account for image indexes, we had started
to run the CVE scanner before pagination was applied, resulting in
decreased ZOT performance if CVE information was requested

2. Increase in medory-cache of cve results to 1m, from 10k digests.

3. Update CVE model to use CVSS severity values in our code.
Previously we relied upon the strings returned by trivy directly,
and the sorting they implemented.
Since CVE severities are standardized, we don't need to pass around
an adapter object just for pagination and sorting purposes anymore.
This also improves our testing since we don't mock the sorting functions anymore.

4. Fix a flaky CLI test not waiting for the zot service to start.

5. Add the search build label on search/cve tests which were missing it.

6. The boltdb update method was used in a few places where view was supposed to be called.

7. Add logs for start and finish of parsing MetaDB.

8. Avoid unmarshalling twice to obtain annotations for multiarch images.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-17 15:12:20 -07:00
Alexei Dodon
f58597ade9
refactor: Reduce zli binary size (#1805)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-15 15:17:01 -07:00
LaurentiuNiculae
8e18917b07
refactor(test): move image utils for tests in a separate module (#1789)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-09-15 09:53:15 -07:00
Alexei Dodon
14206dd6f3
refactor: Review metrics endpoints (#1770)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-15 14:49:34 +03:00
LaurentiuNiculae
aae8b7b4e3
feat(cli): add sort-by flag to sub commands (#1768)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-09-14 10:51:17 -07:00
LaurentiuNiculae
c210e3f377
fix(convert): fix the update rule of download count for images (#1802)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-09-14 10:48:23 -07:00
peusebiu
3d8d47d601
fix(config): fix config reloader panic (#1806)
reloading config from one without extensions
to one with extensions caused a panic

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-14 12:34:18 +03:00
peusebiu
3dbaf2b3ff
fix(sync): ping func should not try to read response body (#1757)
closes: #1703

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-13 20:00:51 +03:00
Andreea Lupu
3518941d6d
fix: change log msg for updating signatures validity (#1804)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-09-13 15:48:31 +03:00
Alexei Dodon
48bf7f69f8
refactor: Reduce zb binary size (#1783)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-13 10:28:14 +03:00
Andreea Lupu
e3876879b0
fix: do not recreate trustpolicy secret if the content doesn't change (#1800)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-09-12 15:21:39 +03:00
Ramkumar Chinchani
6461b661f1
chore: fix dependabot alerts (#1797)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-09-11 20:21:56 -07:00
Andreea Lupu
68a197d7cc
fix: add retry logic to recreate existing trustpolicy secret (#1776)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-09-11 10:13:22 +03:00
Alexei Dodon
18e591f52a
fix: DATA RACE in TestNewExporter (#1766)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-08 15:00:16 +03:00
Andreea Lupu
5a3fac40db
feat: upload cosign public key and notation certificates to cloud (#1744)
- using secrets manager for storing public keys and certificates
- adding a default truststore for notation verification and upload all certificates to this default truststore
- removig `truststoreName` query param from notation api for uploading certificates


(cherry picked from commit eafcc1a213)

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-09-08 10:03:58 +03:00
Alexei Dodon
f5b63963be
refactor: Reduce binary size of zot-minimal; Added CI check for binary size (#1758)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-06 19:58:00 +03:00
peusebiu
59dc4c3229
feat(scheduler): pass the shutdown/reload ctx to running tasks (#1671)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-05 09:48:56 -07:00
peusebiu
c6b822f3dd
refactor(authz): use a struct for user access control info operations (#1682)
fix(authz): fix isAdmin not using groups to determine if a user is admin.
fix(authz): return 401 instead of 403

403 is correct as per HTTP spec
However authz is not part of dist-spec and clients know only about 401
So this is a compromise.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-01 11:13:53 -07:00
peusebiu
b80deb9927
refactor(storage): refactor storage into a single ImageStore (#1656)
unified both local and s3 ImageStore logic into a single ImageStore
added a new driver interface for common file/dirs manipulations
to be implemented by different storage types

refactor(gc): drop umoci dependency, implemented internal gc

added retentionDelay config option that specifies
the garbage collect delay for images without tags

this will also clean manifests which are part of an index image
(multiarch) that no longer exist.

fix(dedupe): skip blobs under .sync/ directory

if startup dedupe is running while also syncing is running
ignore blobs under sync's temporary storage

fix(storage): do not allow image indexes modifications

when deleting a manifest verify that it is not part of a multiarch image
and throw a MethodNotAllowed error to the client if it is.
we don't want to modify multiarch images

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-01 10:54:39 -07:00
LaurentiuNiculae
112fbec5b6
refactor(cli): added equivalent subcommands for each flag combination under every command (#1674)
- image command is now deprecated in favor of 'images'
- cve command is now deprecated in favor of 'cves'

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-08-30 20:12:24 +03:00
Damien Degois
289acfabbd
feat(authn): add generic oidc and allow customizable name (#1691)
Rebased and squashed

Signed-off-by: Damien Degois <damien@degois.info>
2023-08-24 12:33:35 +03:00
Alexei Dodon
247f6dcd3f
feat: propagate detailed error msgs to client (OCI dist-spec format) (#1681)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-08-23 20:59:52 +03:00
Andreea Lupu
cacf54e8cb
refactor: move /pkg/meta/signatures under /pkg/extensions/imagetrust (#1712)
- the size of the binary-minimal becomes 32MB
- "signatures" package is renamed into "imagetrust" and moved under extensions
- if the binary is not built using "imagetrust" tag then the signatures verification will
not be performed

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-08-19 08:52:03 +03:00
LaurentiuNiculae
0731fd3828
test(refactor): refactor tests that use zot-test to use smaller images (#1690)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-08-18 11:46:11 +03:00
Andreea Lupu
76277f5ebd
fix: remove inline GC and schedule a background task instead (#1610)
* fix: remove inline GC and set a default value of gc interval

- remove inline GC
- add a default value of GC interval
- run the GC periodically by default with the default value if no interval provided
- generate GC tasks with a random delay(0-30s) between
- add IsReady() method to scheduler.TaskGenerator interface

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>

* ci: add test for gc with short interval

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>

---------

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-08-07 12:55:19 -07:00
LaurentiuNiculae
fce9a02ed5
fix(convert): now returned annotations for an index will fallback to annotations from a random manifest if the annotations field is not present on the index manifest (#1667)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-08-07 10:25:05 +03:00
Andrei Aaron
77149aa85c
refactor(extensions)!: refactor the extensions URLs and errors (#1636)
BREAKING CHANGE: The functionality provided by the mgmt endpoint has beed redesigned - see details below
BREAKING CHANGE: The API keys endpoint has been moved -  see details below
BREAKING CHANGE: The mgmt extension config has been removed - endpoint is now enabled by having both the search and the ui extensions enabled
BREAKING CHANGE: The API keys configuration has been moved from extensions to http>auth>apikey

mgmt and imagetrust extensions:
- separate the _zot/ext/mgmt into 3 separate endpoints: _zot/ext/auth, _zot/ext/notation, _zot/ext/cosign
- signature verification logic is in a separate `imagetrust` extension
- better hanling or errors in case of signature uploads: logging and error codes (more 400 and less 500 errors)
- add authz on signature uploads (and add a new middleware in common for this purpose)
- remove the mgmt extension configuration - it is now enabled if the UI and the search extensions are enabled

userprefs estension:
- userprefs are enabled if both search and ui extensions are enabled (as opposed to just search)

apikey extension is removed and logic moved into the api folder
- Move apikeys code out of pkg/extensions and into pkg/api
- Remove apikey configuration options from the extensions configuration and move it inside the http auth section
- remove the build label apikeys

other changes:
- move most of the logic adding handlers to the extensions endpoints out of routes.go and into the extensions files.
- add warnings in case the users are still using configurations with the obsolete settings for mgmt and api keys
- add a new function in the extension package which could be a single point of starting backgroud tasks for all extensions
- more clear methods for verifying specific extensions are enabled
- fix http methods paired with the UI handlers
- rebuild swagger docs

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-08-02 21:58:34 +03:00
LaurentiuNiculae
9e38ca51e3
feat(pagination): move pagination and sorting image summary results after conversion (#1637)
fix(config): check for config media type when pushing to repodb

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-07-31 22:16:09 +03:00
LaurentiuNiculae
a5c92bcac3
test(refactor): change upload image function signature (#1655)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-07-28 17:53:46 +03:00
LaurentiuNiculae
abba6aa3cf
feat(test): added image builder for use in tests (#1583)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-07-26 13:08:04 +03:00
peusebiu
86a83ca6e3
fix(authn): fix several issues with authn, closes #1632 (#1633)
- apply Access-Control-Allow-Credentials only if authn is enabled
- enable Logout route for basic auth
- fixed Logout godoc
- fix Access-Control-Allow-Methods on Logout route
- added allowOrigin option in config example

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-07-19 09:27:04 -07:00
LaurentiuNiculae
28de980319
feat(refator): refactoring repodb into meta (#1626)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-07-18 10:27:26 -07:00
Ramkumar Chinchani
fe9c9750b5
chore: fix dependabot alerts (#1631)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-07-17 23:37:54 +03:00
Andrei Aaron
0a5241b025
fix(ci): trivy test to expect 3 CVEs instead of 2 in the vulnerable test layer (#1623)
Trivy started detecting CVE-2023-2975, so one of the checks expecting only 2 CVEs was failing

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-07-16 08:27:59 -07:00
Ramkumar Chinchani
6cd4455da1
chore: fix dependabot alerts (#1621)
* chore: fix dependabot alerts

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* test: fix the validation for digests

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

---------

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-07-15 12:56:09 +03:00
peusebiu
d3f27b4ba6
feat: add additional manifest validations (#1609)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-07-13 09:31:39 -07:00
peusebiu
e682457401
fix(test): sync inconsistent test (#1611)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-07-12 10:21:28 -07:00
Andrei Aaron
faa410a0c3
feat(cli): Fix multiple issues with zli output (#1612)
https://github.com/project-zot/zot/issues/1591
    - I will rename "IMAGE NAME" to "REPOSITORY" in order to make the header easier to parse
    - The order of the images cannot be predicted if zot is getting them 1 by 1 using the REST API for manifests, so they cannot be sorted when printed. We could wait on all calls to return but that may take minutes, and printing partial results as they become available is better.
    - The order of the images can be predicted when relying on the zot specific search API, but that is not available in all zot servers depending on build options. I added sorting ascending by default. We are planning to implement configurable sorting in a separate PR - see the work under https://github.com/project-zot/zot/pull/1577
    - With regards to the column widths/alignments that was discussed before, and the issue is we don't know the values beforehand for the REST API based responses. As mentioned above printing partial results as they become available is better.
    - The column widths/alignments are partially fixed in this PR for the search API, but we should properly fix this in - see https://github.com/project-zot/zot/pull/851

https://github.com/project-zot/zot/issues/1592
    - Fix missing space after help message

https://github.com/project-zot/zot/issues/1598
    - Fix table headers showing for json/yaml format
    - Fix spacing shown with json format, use 1 row per shown entry in order to be compatible with json lines format: https://jsonlines.org/
    - Add document header `---` to every image shown in yaml format to separate the entries

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-07-12 10:21:12 -07:00
Andrei Aaron
418a1a006c
feat(cve): ability to return CVEs per image os and architecture (#1607)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-07-11 09:29:04 -07:00
peusebiu
aaf03c75fc
build: fix mgmt and userprefs when building them separately (#1601)
move queryHasParams in common package
fixes building mgmt and userprefs separately

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-07-10 19:40:14 +03:00
peusebiu
1d01b644ea
feat(sync): sync can include self url in registry.URLs (#1562)
sync now ignores self referencing urls, this will help
in clustering mode where we can have the same config
for multiple zots

closes #1335

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-07-10 12:27:21 +03:00
peusebiu
17d1338af1
feat: integrate openID auth logic and user profile management (#1381)
This change introduces OpenID authn by using providers such as Github,
Gitlab, Google and Dex.
User sessions are now used for web clients to identify
and persist an authenticated users session, thus not requiring every request to
use credentials.
Another change is apikey feature, users can create/revoke their api keys and use them
to authenticate when using cli clients such as skopeo.

eg:
login:
/auth/login?provider=github
/auth/login?provider=gitlab
and so on

logout:
/auth/logout

redirectURL:
/auth/callback/github
/auth/callback/gitlab
and so on

If network policy doesn't allow inbound connections, this callback wont work!

for more info read documentation added in this commit.

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Co-authored-by: Alex Stan <alexandrustan96@yahoo.ro>
2023-07-07 09:27:10 -07:00
Andreea Lupu
41b05c60dd
feat: upload certificates and public keys for verifying signatures (#1485)
In order to verify signatures, users could upload their certificates and public keys using these routes:
	-> for public keys:
		/v2/_zot/ext/mgmt?resource=signatures&tool=cosign
	-> for certificates:
		/v2/_zot/ext/mgmt?resource=signatures&tool=notation&truststoreType=ca&truststoreName=name
Then the public keys will be stored under $rootdir/_cosign and the certificates will be stored under
$rootdir/_notation/truststore/x509/$truststoreType/$truststoreName.
Also, for notation case, the "truststores" field of $rootir/_notation/trustpolicy.json file will be
updated with a new entry "$truststoreType:$truststoreName".
Also based on the uploaded files, the information about the signatures validity will be updated
periodically.

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-07-06 14:57:59 +03:00
peusebiu
49e4d93f42
fix(test): TestConfigReloader, wait for trivy db download (#1543)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-07-06 14:17:49 +03:00
LaurentiuNiculae
0a04b2a4ed
feat(cve): implemented trivy image scan for multiarch images (#1510)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-07-06 11:36:26 +03:00
LaurentiuNiculae
96d9d318df
feat(referrers): added index support for referrers queries (#1560)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-07-05 09:42:16 -07:00
alexstan12
62889c3cb1
refactor: split AuthZ mdw in 2 different parts, each for a specific purpose (#1542)
- AuthzHandler has now been split in BaseAuthzHandler and DistSpecAuthzHandler
The former populates context with user specific data needed in most handlers, while
the latter executes access logic specific to distribution-spec handlers.

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2023-07-05 09:37:52 -07:00
Andrei Aaron
7fee57e7cc
fix(CVE): attempt to scan now returns early with an error if trivyDB metadata json is missing (#1548)
Also modify zli to retry in case of such errors,
assuming the trivyDB will eventually be downloaded by the scheduled task.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-07-05 09:08:16 -07:00
Lisca Ana-Roberta
d4f200c2e1
fix: changing default numWorkers, making it customizable and refactoring scheduler (#1563)
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
2023-07-04 11:03:29 +03:00
Ramkumar Chinchani
1300fdfa88
refactor: filenames should use _ not - (#1547)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-06-22 11:54:41 -07:00
peusebiu
d881f4e916
fix(sync): flaky test on fetching tags (#1546)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-06-22 10:43:47 -07:00
alexstan12
ea7dbf9e5c
refactor: move helper functions under common, in usage specific named files (#1540)
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2023-06-22 14:29:45 +03:00
peusebiu
377aff1853
fix(sync): fixed skipping docker images when they already synced (#1521)
before syncing an image we first check if it's already present in our storage
to do that we get the manifest from remote and compare it with the local one
but in the case of syncing docker images, because the conversion to OCI format is done while
syncing, we get a docker manifest before conversion, so sync detects that local manifest and
remote one are different, so it starts syncing again.

to overcome this, convert remote docker manifests to OCI manifests and then compare.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-06-21 11:05:52 -07:00
peusebiu
ea84752214
fix(test): fix flaky test (#1544)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-06-21 19:53:14 +03:00
Lisca Ana-Roberta
aab149610f
fix: removed quotation marks from enum in swagger docs (#1539)
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
2023-06-20 15:32:19 +03:00
Lisca Ana-Roberta
aa16c955b3
fix: added swagger doc generation for mgmt and userprefs (#1530)
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
2023-06-19 10:43:25 -07:00
peusebiu
fc6d6356fb
feat(sync): sync references(signatures/artifacts) recursively (#1500)
sync now also pulls chained artifacts recursively
eg:
 image->sbom->sbom signature
 image->artifact->artifact

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-06-16 10:27:33 -07:00
Andrei Aaron
f9f9422d13
ci(disk usage): disk related fixes and improvements (#1524)
ci(workflow): show disk usage and free up disk space used by unneeded tooling
ci(tests): routes tests: do not copy large images if they are not used later
ci(trivy): update a test: download trivy.db to a temporary folder

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-06-15 15:07:28 -07:00
Ramkumar Chinchani
4d6ca493f2
chore: fix dependabot alerts (#1501)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-06-09 10:27:42 -07:00
Andrei Aaron
96d00cd0ef
fix(cve): Fix CVE scanning in images containing Jar files (#1475) 2023-06-01 00:37:46 +03:00
peusebiu
612a12e5a8
refactor(sync): use task scheduler (#1301)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-05-31 10:26:23 -07:00
LaurentiuNiculae
a3f355c278
refactor(storage): refactoring storage (#1459)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-05-26 11:08:19 -07:00
peusebiu
9acd19f7ea
fix(extensions): consolidate extensions headers returned to UI by extensions (#1473)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-05-25 11:44:54 -07:00
Lisca Ana-Roberta
6a7035c599
fix: removed duplicate structures from service.go and moved them to pkg/common (#1436)
Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com>
2023-05-25 11:27:49 -07:00
LaurentiuNiculae
2b8479f7f2
feat(userprefs): update documentation and list extensions endpoint (#1456)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-05-25 14:46:52 +03:00
Andreea Lupu
970997f3a8
feat(graphql & repodb): add info about signature validity (#1344)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-05-24 09:46:16 -07:00
Ramkumar Chinchani
83ae1aad70
chore(go.mod): fix dependabot alerts (#1466)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-05-23 10:14:43 +03:00
LaurentiuNiculae
c0170b0811
feat(routes): move the cors handler from /v2 to only where it's needed (#1457)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-05-19 21:22:51 -07:00
Ramkumar Chinchani
2be5459c8e
chore: fix dependabot alerts (#1458)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-05-17 00:37:34 -07:00
LaurentiuNiculae
f4501e6b6b
feat(search): add artifact type to manifest summary gql structure (#1448)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-05-15 10:44:49 -07:00
LaurentiuNiculae
912854f29b
fix(sync): fix digest set into repodb (#1446)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-05-15 10:43:41 -07:00
LaurentiuNiculae
7bf40e7308
fix(sync): fixed way of updating repodb when syncing a signature (#1439)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-05-15 12:02:23 +03:00
Lisca Ana-Roberta
e262fbea64
feat: verifying and enabling necessary extensions for ui (#1369)
Signed-off-by: Ana-Roberta Lisca <ana.kagome@yahoo.com>
2023-05-12 09:43:14 -07:00
LaurentiuNiculae
7d7bc9d5e4
feat(api): added oci-subject header when pushing an image with subject field (#1415)
- as requested by the latest version of the oci distribution spec

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-05-12 09:32:01 -07:00
LaurentiuNiculae
b7ef88c96d
fix(search): added the missing headers for search route (#1438)
- added allow methods and allowed headers

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-05-11 16:05:14 +03:00
LaurentiuNiculae
ea79be64da
refactor(artifact): remove oci artifact support (#1359)
* refactor(artifact): remove oci artifact support
- add header to referrers call to indicated applied artifact type filters

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

* feat(gc): simplify gc logic to increase coverage

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>

---------

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-05-10 10:15:33 -07:00
LaurentiuNiculae
3be690c2ac
feat(userpreferences): update allowed methods header for user preferences routes (#1430)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-05-10 10:09:53 -07:00
peusebiu
d62dbcdf63
fix(sync): fix syncing signatures when using destination in sync's config (#1429)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-05-08 10:16:20 -07:00
LaurentiuNiculae
449f0d0ac3
fix(repoinfo): fix userprefs values for repos returned by expanded repo info (#1413)
- now isBookmarked and isStarred are updated correctly

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-05-04 09:51:21 -07:00
peusebiu
e299ae199a
fix(sync): skip non distributable layers (#1421)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-05-04 09:38:33 -07:00
Ramkumar Chinchani
42df4c505a
chore: fix dependabot alerts (#1403)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-05-01 12:49:10 -07:00
Nicol
c169698c95
feat: remove usage of zerolog.Logger.Msgf() from zot code (#1382)
Signed-off-by: Nicol Draghici <idraghic@cisco.com>
2023-04-27 19:44:22 -07:00
Andreea Lupu
40bf76add5
chore(go.mod): upgrade trivy and cosign (#1387)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-04-27 09:35:10 -07:00
LaurentiuNiculae
3d8a4022bd
feat(global-search): add filtering options by starred and bookmarked (#1336)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-04-27 08:11:13 -07:00
Ramkumar Chinchani
635d07ae04
chore: update golang (to 1.20.x) and golangci-linter (#1388)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-04-27 00:09:46 -07:00
LaurentiuNiculae
9cc990d7ca
feat(repodb): add user related information to repodb (#1317)
Initial code was contributed by Bogdan BIVOLARU <104334+bogdanbiv@users.noreply.github.com>
Moved implementation from a separate db to repodb by Andrei Aaron <aaaron@luxoft.com>

Not done yet:
- run/test dynamodb implementation, only boltdb was tested
- add additional coverage for existing functionality
- add web-based APIs to toggle the stars/bookmarks on/off

Initially graphql mutation was discussed for the missing API but
we decided REST endpoints would be better suited for configuration



feat(userdb): complete functionality for userdb integration

- dynamodb rollback changes to user starred repos in case increasing the total star count fails
- dynamodb increment/decrement repostars in repometa when user stars/unstars a repo
- dynamodb check anonymous user permissions are working as intendend
- common test handle anonymous users
- RepoMeta2RepoSummary set IsStarred and IsBookmarked



feat(userdb): rest api calls for toggling stars/bookmarks on/off



test(userdb): blackbox tests



test(userdb): move preferences tests in a different file with specific build tags



feat(repodb): add is-starred and is-bookmarked fields to repo-meta

- removed duplicated logic for determining if a repo is starred/bookmarked

Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Co-authored-by: Andrei Aaron <aaaron@luxoft.com>
2023-04-24 11:13:15 -07:00
peusebiu
d818293cc1
test(sync): make sure sync doesn't write on shutdown (#1370)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-04-19 11:34:41 -07:00
Nicol
0586c6227e
refactor: remove pkg/extensions/search/common and move the code to the appropriate packages (#1358)
Signed-off-by: Nicol Draghici <idraghic@cisco.com>
2023-04-18 11:07:47 -07:00
Andrei Aaron
e63faa8898
fix(csp): upgrade UI and fix zap failure (#1372)
The zap scanner started to check the csp header, which is causing a warning.

We also need to ignore the rule, as both settings are read by the scanner.

Per https://w3c.github.io/webappsec-csp/#example-7bb4ce67 we can have multiple
Content-Security-Policy headers, and the most restrictive policies apply.
This rule doesn't seem to be applied by zap.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-04-13 13:48:09 -07:00
Ramkumar Chinchani
e6b81bb354
chore(go.mod): fix dependabot alerts (#1365)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-04-12 14:10:47 +03:00
Ramkumar Chinchani
8f809bda29
chore(go.mod): fix dependabot alerts (#1351)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-04-10 14:09:54 -07:00
Nicol
3510ef0fb0
refactor: move pkg/extensions/search/common/oci_layout.go under pkg/test/ (#1325)
Signed-off-by: Nicol Draghici <idraghic@cisco.com>
2023-04-07 09:52:26 -07:00