* fix(scheduler): data race when pushing new tasks
the problem here is that scheduler can be closed in two ways:
- canceling the context given as argument to scheduler.RunScheduler()
- running scheduler.Shutdown()
because of this shutdown can trigger a data race between calling scheduler.inShutdown()
and actually pushing tasks into the pool workers
solved that by keeping a quit channel and listening on both quit channel and ctx.Done()
and closing the worker chan and scheduler afterwards.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
* refactor(scheduler): refactor into a single shutdown
before this we could stop scheduler either by closing the context
provided to RunScheduler(ctx) or by running Shutdown().
simplify things by getting rid of the external context in RunScheduler().
keep an internal context in the scheduler itself and pass it down to all tasks.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
---------
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
Which could be imported independently. See more details:
1. "zotregistry.io/zot/pkg/test/common" - currently used as
tcommon "zotregistry.io/zot/pkg/test/common" - inside pkg/test
test "zotregistry.io/zot/pkg/test/common" - in tests
. "zotregistry.io/zot/pkg/test/common" - in tests
Decouple zb from code in test/pkg in order to keep the size small.
2. "zotregistry.io/zot/pkg/test/image-utils" - curently used as
. "zotregistry.io/zot/pkg/test/image-utils"
3. "zotregistry.io/zot/pkg/test/deprecated" - curently used as
"zotregistry.io/zot/pkg/test/deprecated"
This one will bre replaced gradually by image-utils in the future.
4. "zotregistry.io/zot/pkg/test/signature" - (cosign + notation) use as
"zotregistry.io/zot/pkg/test/signature"
5. "zotregistry.io/zot/pkg/test/auth" - (bearer + oidc) curently used as
authutils "zotregistry.io/zot/pkg/test/auth"
6. "zotregistry.io/zot/pkg/test/oci-utils" - curently used as
ociutils "zotregistry.io/zot/pkg/test/oci-utils"
Some unused functions were removed, some were replaced, and in
a few cases specific funtions were moved to the files they were used in.
Added an interface for the StoreController, this reduces the number of imports
of the entire image store, decreasing binary size for tests.
If the zb code was still coupled with pkg/test, this would have reflected in zb size.
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
- refactor(cve): remove the global of type cveinfo.CveInfo from the extensions package
Replace it with an attribute on controller level
- refactor(controller): extract initialization logic from controller.Run()
- test(cve): mock cve scanner in cli tests
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Conformance spec requires responding to PATCH requests with
response header 'Content-Range' and value <range>
Signed-off-by: Bogdan BIVOLARU <104334+bogdanbiv@users.noreply.github.com>
PR (linter: upgrade linter version #405) triggered lint job which failed
with many errors generated by various linters. Configurations were added to
golangcilint.yaml and several refactorings were made in order to improve the
results of the linter.
maintidx linter disabled
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Periodically poll registries and pull images according to sync's config
Added sync on demand, syncing when clients asks for an image which
zot doesn't have.
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
With recent docker client-side changes, on 'docker pull' we see:
"Error response from daemon: missing or empty Content-Type header"
Hence, set Content-Type header.
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
added support to point multiple storage locations in zot by running multiple instance of zot in background.
see examples/config-multiple.json for more info about config.
Closes#181
The idea initially was to use bazel to do our builds, however golang
build system is now good enough and our code base is entirely go.
It is also slowing down our travis ci/cd pipeline.
Go version changed to 1.14.4
Golangci-lint changed to 1.26.0
Bazel version changed to 3.0.0
Bazel rules_go version changed to 0.23.3
Bazel gazelle version changed to v0.21.0
Bazel build tools version changed to 0.25.1
Bazel skylib version changed to 1.0.2
Upstream conformance tests are being updated, so we need to align along
with our internal GC and dedupe features.
Add a new example config file which plays nice with conformance tests.
DeleteImageManifest() updated to deal with the case where the same
manifest can be created with multiple tags and deleted with the same
digest - so all entries must be deleted.
DeleteBlob() delete the digest key (bucket) when last reference is
dropped
As the number of repos and layers increases, the greater the probability
that layers are duplicated. We dedupe using hard links when content is
the same. This is intended to be purely a storage layer optimization.
Access control when available is orthogonal this optimization.
Add a durable cache to help speed up layer lookups.
Update README.
Add more unit tests.
Previously, CheckManifest() was not checking for repo not found
condition and would default to 500 status code.
Add the check now to return 404.
Fixes issue #74
Fixes issue #67.
As per dist spec, DELETE of a image manifest can only be done with
digest as <reference> param. Previously, tags were being allowed as
well. This is not conformant to the spec.
dist-spec compliance tests are now becoming a part of dist-spec repo
itself - we want to be compliant
pkg/api/regex.go:
* revert uppercasing in repository names
pkg/api/routes.go:
* ListTags() should support the URL params 'n' and 'last'
for pagination
* s/uuid/session_id/g to use the dist-spec's naming
* Fix off-by-one error in GetBlobUpload()'s http response "Range" header
* DeleteManifest() success status code is 202
* Fix PatchBlobUpload() to account for "streamed" use case
where neither "Content-Length" nor "Content-Range" headers are set
pkg/storage/storage.go:
* Add a "streamed" version of PutBlobChunk() called PutBlobChunkStreamed()
pkg/compliance/v1_0_0/check.go:
* fix unit tests to account for changed response status codes
zot ci/cd tests are too stict, so separate and relax them for compliance
tests.
Location header is set in some cases, but some clarification is needed
in URL construction.
Fix some incorrect compliance tests.
"mount blob" was the only feature we didn't handle.
We don't fully support it yet because we don't have access control
support, but at least handle it and return something sane to remain
compliant.
This adds a new --json flag to the compliance subcommand, which
will output the compliance test results as minified JSON to stdout.
Also a few other small additions:
- Exit 1 if compliance tests fail
- Use random port for test server using freeport library (added)
Signed-off-by: Josh Dolitsky <393494+jdolitsky@users.noreply.github.com>
