0
Fork 0
mirror of https://github.com/project-zot/zot.git synced 2024-12-16 21:56:37 -05:00
Commit graph

190 commits

Author SHA1 Message Date
peusebiu
458d40fb48
fix(nightly): increase nightly delays and timeouts (#1877)
change the startup dedupe blackbox test source image with a lighter one

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-12 16:42:07 +03:00
Alexei Dodon
039648a445
ci(licenses): check if licenses file is updated (#1898)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-09 20:23:51 -07:00
Ramkumar Chinchani
ed775914df
chore: fix dependabot alerts (#1911)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-09 11:35:42 -07:00
Ramkumar Chinchani
0fb23848b4
chore: fix dependabot alerts (#1885)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-04 12:29:31 -07:00
Ramkumar Chinchani
c6407354b5
build: fix container image targets (#1856)
Use stacker to build OCI image.
Use podman/docker to build docker image.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-04 09:12:56 -07:00
peusebiu
0eb984426e
fix(nightly): create bucket for s3 testing with localstack (#1872)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-10-02 15:31:50 +03:00
Ramkumar Chinchani
56e3580687
ci: update localstack to 2.3.1 (#1869)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-09-29 11:32:27 -07:00
Andrei Aaron
0ec6f8acf8
ci(localstack): pin localstack python package to 2.2.0 and pull container image from ghcr (#1867)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-29 11:40:46 +03:00
Alexei Dodon
3a9a932791
fix: reduce test run time (#1832)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-28 21:59:52 +03:00
Ramkumar Chinchani
9096031aeb
chore: fix dependabot alerts (#1855)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-09-25 23:03:13 +03:00
peusebiu
f164fb9e03
fix(ci): fix nighlty builds and print zot log on failure (#1799)
now gc stress on s3 storage is using minio for ci/cd builds
gc stress on s3 storage is using localstack for nightly builds

fixed(gc): make sure we don't remove repo if there are blobs
being uploaded or the number of blobs gc'ed is not 0

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-20 19:25:06 +03:00
Ramkumar Chinchani
f8002c7dd3
chore: fix dependabot alerts (#1827)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-09-18 11:34:57 -07:00
Alexei Dodon
f58597ade9
refactor: Reduce zli binary size (#1805)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-15 15:17:01 -07:00
Alexei Dodon
48bf7f69f8
refactor: Reduce zb binary size (#1783)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-13 10:28:14 +03:00
Andrei Aaron
6011705931
chore(dependabot): increase the limit of dependabot PRs (#1788)
Given the default limit is 5 and some packages like AWS release daily,
having a weekly schedule results in the same packages being flagged for update
most of the time.

Let's increase to 10 to make sure there are no issues in other libraries

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-11 11:02:11 -07:00
Andrei Aaron
6115eed4ec
refactor(makefile): consolidate the make targets used for bats tests (#1746)
New examples of running tests:

1. To run a specific bats file (with and without verbose output):
make run-blackbox-tests BATS_TEST_FILE_PATH=test/blackbox/delete_images.bats
make run-blackbox-tests BATS_TEST_FILE_PATH=test/blackbox/delete_images.bats BATS_VERBOSITY=2

2. To run the CI tests (with and without verbose output)
make run-blackbox-ci
make run-blackbox-ci BATS_VERBOSITY=2

BATS_TEST_FILE_PATH is used to pass on the test file to run using `run-blackbox-tests`
BATS_VERBOSITY controls the verbosity of the bats framework output, if unspecified the output only
contains test results and failure message in case of failures.
If BATS_VERBOSITY is 1, then also show commands as they are executed.
If BATS_VERBOSITY is 2, on top of the above it also shows output of passed tests.

Other changes in this PR:
- Update some of the tests to show logs after the run ends.
- Run the linters before the tests, as it saves time on failures when running in GH

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-07 21:06:21 +03:00
Alexei Dodon
f5b63963be
refactor: Reduce binary size of zot-minimal; Added CI check for binary size (#1758)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-06 19:58:00 +03:00
Ramkumar Chinchani
75a76005b4
chore: fix dependabot alerts (#1763)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-09-05 22:30:49 +03:00
peusebiu
c6b822f3dd
refactor(authz): use a struct for user access control info operations (#1682)
fix(authz): fix isAdmin not using groups to determine if a user is admin.
fix(authz): return 401 instead of 403

403 is correct as per HTTP spec
However authz is not part of dist-spec and clients know only about 401
So this is a compromise.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-01 11:13:53 -07:00
peusebiu
b80deb9927
refactor(storage): refactor storage into a single ImageStore (#1656)
unified both local and s3 ImageStore logic into a single ImageStore
added a new driver interface for common file/dirs manipulations
to be implemented by different storage types

refactor(gc): drop umoci dependency, implemented internal gc

added retentionDelay config option that specifies
the garbage collect delay for images without tags

this will also clean manifests which are part of an index image
(multiarch) that no longer exist.

fix(dedupe): skip blobs under .sync/ directory

if startup dedupe is running while also syncing is running
ignore blobs under sync's temporary storage

fix(storage): do not allow image indexes modifications

when deleting a manifest verify that it is not part of a multiarch image
and throw a MethodNotAllowed error to the client if it is.
we don't want to modify multiarch images

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-01 10:54:39 -07:00
Alexei Dodon
41bbb23e30
fix: bats test refactoring (#1731)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-08-30 12:24:28 -07:00
Alexei Dodon
247f6dcd3f
feat: propagate detailed error msgs to client (OCI dist-spec format) (#1681)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-08-23 20:59:52 +03:00
Alexei Dodon
ca2904762a
fix(ci/cd): detect uncommited swagger docs (#1724)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-08-23 19:21:43 +03:00
Alexei Dodon
e510df7c22
fix: swagger Makefile target broken on darwin (#1701)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-08-15 13:07:56 -07:00
Ramkumar Chinchani
e129d4003b
chore: fix dependabot alerts (#1702)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-08-15 10:15:50 +03:00
LaurentiuNiculae
ed90e3bd24
test(annotations): print logs for annotations bats tests (#1680)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-08-07 13:51:57 -07:00
Andreea Lupu
76277f5ebd
fix: remove inline GC and schedule a background task instead (#1610)
* fix: remove inline GC and set a default value of gc interval

- remove inline GC
- add a default value of GC interval
- run the GC periodically by default with the default value if no interval provided
- generate GC tasks with a random delay(0-30s) between
- add IsReady() method to scheduler.TaskGenerator interface

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>

* ci: add test for gc with short interval

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>

---------

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-08-07 12:55:19 -07:00
Ramkumar Chinchani
cad564b9ee
ci(conformance): revert to upstream OCI checks (#1678)
Reverts part of de24bd937f

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-08-04 13:38:28 -07:00
Andrei Aaron
77149aa85c
refactor(extensions)!: refactor the extensions URLs and errors (#1636)
BREAKING CHANGE: The functionality provided by the mgmt endpoint has beed redesigned - see details below
BREAKING CHANGE: The API keys endpoint has been moved -  see details below
BREAKING CHANGE: The mgmt extension config has been removed - endpoint is now enabled by having both the search and the ui extensions enabled
BREAKING CHANGE: The API keys configuration has been moved from extensions to http>auth>apikey

mgmt and imagetrust extensions:
- separate the _zot/ext/mgmt into 3 separate endpoints: _zot/ext/auth, _zot/ext/notation, _zot/ext/cosign
- signature verification logic is in a separate `imagetrust` extension
- better hanling or errors in case of signature uploads: logging and error codes (more 400 and less 500 errors)
- add authz on signature uploads (and add a new middleware in common for this purpose)
- remove the mgmt extension configuration - it is now enabled if the UI and the search extensions are enabled

userprefs estension:
- userprefs are enabled if both search and ui extensions are enabled (as opposed to just search)

apikey extension is removed and logic moved into the api folder
- Move apikeys code out of pkg/extensions and into pkg/api
- Remove apikey configuration options from the extensions configuration and move it inside the http auth section
- remove the build label apikeys

other changes:
- move most of the logic adding handlers to the extensions endpoints out of routes.go and into the extensions files.
- add warnings in case the users are still using configurations with the obsolete settings for mgmt and api keys
- add a new function in the extension package which could be a single point of starting backgroud tasks for all extensions
- more clear methods for verifying specific extensions are enabled
- fix http methods paired with the UI handlers
- rebuild swagger docs

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-08-02 21:58:34 +03:00
Ramkumar Chinchani
42f9f78125
ci: add freebsd target for releases (#1660)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-07-31 13:51:03 -07:00
Ramkumar Chinchani
75b94a3c8b
chore: fix dependabot alerts (#1649) 2023-07-25 08:14:37 +03:00
Ramkumar Chinchani
de24bd937f
fix(conformance): OCI-Filters-Applied should return a literal (#1640)
* fix(conformance): OCI-Filters-Applied should return a literal

https://github.com/opencontainers/distribution-spec/issues/448

Should only indicate what filter-type was applied and not what exact
values it was filtered on.

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

* ci(conformance): point to temporary repo

Revert this commit once upstream changes have been merged

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>

---------

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-07-22 16:25:02 -07:00
Ramkumar Chinchani
fe9c9750b5
chore: fix dependabot alerts (#1631)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-07-17 23:37:54 +03:00
peusebiu
d3f27b4ba6
feat: add additional manifest validations (#1609)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-07-13 09:31:39 -07:00
Ramkumar Chinchani
730ef4aada
chore: fix dependabot alerts (#1613)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-07-13 10:14:03 +03:00
peusebiu
cda6916b45
fix: don't allow blobs to be deleted if in use (#1559)
dist-spec APIs independently allow deletion of blobs and manifests.
Doing the former when in use by an image manifest or index is simply
error-prone. So disallow it.

Fixes issue #1509

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-07-10 12:24:45 +03:00
peusebiu
cd6f679359
fix(ci/cd): stop localstack after ci/cd pipeline (#1590)
this should clean localstack persisted data in tests

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-07-07 12:40:06 -07:00
peusebiu
17d1338af1
feat: integrate openID auth logic and user profile management (#1381)
This change introduces OpenID authn by using providers such as Github,
Gitlab, Google and Dex.
User sessions are now used for web clients to identify
and persist an authenticated users session, thus not requiring every request to
use credentials.
Another change is apikey feature, users can create/revoke their api keys and use them
to authenticate when using cli clients such as skopeo.

eg:
login:
/auth/login?provider=github
/auth/login?provider=gitlab
and so on

logout:
/auth/logout

redirectURL:
/auth/callback/github
/auth/callback/gitlab
and so on

If network policy doesn't allow inbound connections, this callback wont work!

for more info read documentation added in this commit.

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Co-authored-by: Alex Stan <alexandrustan96@yahoo.ro>
2023-07-07 09:27:10 -07:00
peusebiu
7d7e4416b5
fix(nightly): fix nightly builds (#1584)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-07-06 16:50:12 +03:00
Ramkumar Chinchani
137e5bd793
chore: fix dependabot alerts (#1581)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-07-05 11:42:24 +03:00
Ramkumar Chinchani
aad6db279b
chore: fix dependabot alerts (#1576)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-07-04 09:19:26 -07:00
Andreea Lupu
7ec748f58a
fix: change commit message for pushing changes to project-zot/helm-charts (#1564)
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-06-28 20:42:31 +03:00
Ramkumar Chinchani
d78ef0b1bd
ci: update oci dist-spec conformance test (#1552)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-06-24 18:51:47 -07:00
Ramkumar Chinchani
d30d7a9330
chore: fix dependabot alerts (#1537)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-06-19 12:34:50 -07:00
peusebiu
d45cfbf5d2
feat(ci): give minio container more time to start (#1527)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-06-16 12:24:38 +03:00
Ramkumar Chinchani
7dd17be96d
chore: fix dependabot alerts (#1517)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-06-16 10:28:43 +03:00
Andrei Aaron
f9f9422d13
ci(disk usage): disk related fixes and improvements (#1524)
ci(workflow): show disk usage and free up disk space used by unneeded tooling
ci(tests): routes tests: do not copy large images if they are not used later
ci(trivy): update a test: download trivy.db to a temporary folder

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-06-15 15:07:28 -07:00
Ramkumar Chinchani
d9e5f33e7e
chore: fix security alerts (#1493)
https://github.com/project-zot/zot/security/code-scanning/4293

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-06-01 16:53:50 -07:00
peusebiu
612a12e5a8
refactor(sync): use task scheduler (#1301)
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-05-31 10:26:23 -07:00
Andrei Aaron
60148138d5
ci(cri-o): update cri-o installation steps (#1470)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-05-23 17:01:10 +03:00