0
Fork 0
mirror of https://github.com/project-zot/zot.git synced 2024-12-16 21:56:37 -05:00
Commit graph

238 commits

Author SHA1 Message Date
alexstan12
16e9822c7f
Add fuzz tests for storage_fs (#601)
This commit uses native go fuzzing to fuzz test implementations
of storage in storage_fs.

moved fuzzing testdata for storage_fs in separate repo

added make target and script for importing fuzz data and running all fuzz tests

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-07-27 20:37:55 +03:00
Andrei Aaron
b5f27c5b50 RepoSummary has a new attribute NewestTag of type ImageSummary
ImageListWithLatestTag currently returns a list of ImageInfo objects.
It needs to return consistent results with the API used for Global search as the same information will be used by the UI in the same type or cards.
So we need to update RepoSummary to include the data which right now is present in ImageInfo, but missing from RepoSummary (information on the latest tag in that specific repo).
Will update return type of ImageListWithLatestTag in a later PR (issue tracked in a separate GH issue)

Closes #666

Signed-off-by: Andrei Aaron <andaaron@cisco.com>
2022-07-27 19:41:00 +03:00
Lisca Ana-Roberta
87fc941b3c image level lint: enforce manifest mandatory annotations
closes #536

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
2022-07-27 11:48:04 +03:00
Alex Stan
4fd727a10c changed filenames in pkg/extensions
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-07-26 16:56:20 +03:00
Andrei Aaron
10d9b1514b Fixes/Improvements to pkg/cli/stress_test.go
- Decrease RLIMIT_NOFILE and the number of goroutines used to reach this limit (from 512 to 100)
- Reset RLIMIT_NOFILE to the initial value before the test finishes
- Remove panic
- Use temporary dir managed by test framework
- Swith to using test logging in pkg/cli/stress_test.go
- Execute commands without `bash -c` in pkg/cli/stress_test.go

First item is needed as the GH runner seems to stop the test if stressed too much.
The lower number is still good enough to reproduce the test conditions

Signed-off-by: Andrei Aaron <andaaron@cisco.com>
2022-07-26 13:01:16 +03:00
Laurentiu Niculae
58f8cd5d7d test calculated size
Signed-off-by: Laurentiu Niculae <themelopeus@gmail.com>
2022-07-20 10:03:11 -07:00
Laurentiu Niculae
80369140f1 add image info to parameter
Signed-off-by: Laurentiu Niculae <themelopeus@gmail.com>
2022-07-20 10:03:11 -07:00
Laurentiu Niculae
7e3d063319 freeform querry api
Signed-off-by: Laurentiu Niculae <themelopeus@gmail.com>
2022-07-20 10:03:11 -07:00
Alex Stan
a31869f270 fix GetReferrers function to be able to retrieve referrers of any specified artifactType
Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-07-19 09:44:23 -07:00
Andrei Aaron
43160dcc43 Update to graphql 1.17.13
We encountered some problems with using the existing folder structure,
but it looks like running the tooling with the latest versions works after
we regenerated the project using 'gql init' and refactoring to separate
the login previously in resolvers.go.

- the autogenerated code is now under the gql_generated folder
- the file resolvers.go now contains only the code which is not
rewritten by the gqlgen framework
- the file schema.resolvers.go is rewritten when gqlgen runs,
and we'll only keep there the actual resolvers matching query names
Changes we observed to schema.resolvers.go when gqlgen runs include
reordering methods, and renaming function parameters to match the
names used in schema.graphql
- we now have a gqlgen.yaml config file which governs the behavior of
gqlgen (can be tweaked to restructure the folder structure of the
generated code in the future)

Looks like the new graphql server has better validation
1 Returns 422 instead of 200 for missing query string - had to update tests
2 Correctly uncovered an error in a test for a bad `%` in query string.

As as result of 2, a `masked` bug was found in the way we check if images are
signed with Notary, the signatures were reasched for with the media type
of the image manifest itself instead of the media type for notation.
Fixed this bug, and improved error messages.
This bug would have also been reproducible with main branch if the bad `%`
in the test would have fixed.

Updated the linter to ignore some issues with the code which is
always rewritten when running:
`go run github.com/99designs/gqlgen@v0.17.13 generate`

Add a workflow to test gqlgen works and has no uncommitted changes

Signed-off-by: Andrei Aaron <andaaron@cisco.com>
2022-07-18 12:55:40 -07:00
Petu Eusebiu
2496fef3c2 Fix data race on trivydb download in tests.
Multiple go routines downloading trivy db
triggers data race on trivy internal db.Path().
In each go routine wait for db download to start.
closes #636

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-07-14 09:31:15 -07:00
Petu Eusebiu
003de3a80a Fix config reloader in tests
config file may get removed before fsnotify starts watching it
make sure the config file gets removed when test ends, closes #608

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-07-14 09:31:15 -07:00
Petu Eusebiu
01d742718f ci/cd: fix oras cli flags after it got updated
installing notation and oras not needed anymore

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-07-12 10:02:51 -07:00
Petu Eusebiu
7954add73a Fix data races in tests closes #599, closes #598
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-06-30 13:33:47 -07:00
Alex Stan
ada21ed842 Manage builds with different combinations of extensions
Files were added to be built whether an extension is on or off.
New build tags were added for each extension, while minimal and extended disappeared.

added custom binary naming depending on extensions used and changed references from binary to binary-extended

added automated blackbox tests for sync, search, scrub, metrics

added contributor guidelines

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-06-30 09:53:52 -07:00
Ramkumar Chinchani
eed48c1715 refactor filenames to reflect functionality
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-06-21 21:42:54 -07:00
Catalin Hofnar
a8a65a6c37 Modified sync log calls to include error type (#336)
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2022-06-15 09:45:49 -07:00
Lisca Ana-Roberta
111b80625d added repos command to list repositories
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
2022-06-15 02:22:18 -07:00
Shivam Mishra
620bc7c517 routes: strip query parameter from request URL
reuqest url also contains query parameter due to this in some scenarios
location header is setting up incorrectly, strip query parameter from
request url to correctly setup location header.

Closes #573 #575

Signed-off-by: Shivam Mishra <shimish2@cisco.com>
2022-06-08 22:50:37 -07:00
Shivam Mishra
f52c950d04 fix sample request url in search extension README
Signed-off-by: Shivam Mishra <shimish2@cisco.com>
2022-06-07 11:24:19 -07:00
Catalin Hofnar
0b6fdc23ea Added sync onDemand test for ORAS artifact
Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2022-06-06 10:38:42 -07:00
Shivam Mishra
b61aff62cd check notary v2 signature while looking for available signatures
expanded repo info also provides information if manifests of repo is signed or not
previously it was looking for only cosign signature.

Signed-off-by: Shivam Mishra <shimish2@cisco.com>
2022-06-03 17:45:22 -07:00
laurentiuNiculae
c9b32c73ae added more types of severity
Signed-off-by: laurentiuNiculae <themelopeus@gmail.com>
2022-06-03 09:44:54 -07:00
Andreea-Lupu
081ba0b2f2 fix periodic background tasks - gc and scrub
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2022-06-02 08:58:02 -07:00
Lisca Ana-Roberta
62775cc095 fixed failed tests for all skopeo versions
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

skopeo verifications

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

skopeo verifications modified makefile

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

modified how to get digest and fixed makefile

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

fixed failed tests for all skopeo versions

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

echo skopeo version

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

skopeo verifications

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

skopeo verifications modified makefile

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

modified how to get digest and fixed makefile

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

skopeo failed tests fixed

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

changed function name

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

fixed lost modifications

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>

fixed code coverage and dead code

Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
2022-05-27 08:19:07 -07:00
Lisca Ana-Roberta
e5a14670db code coverage improvement
Signed-off-by: Lisca Ana-Roberta <ana.kagome@yahoo.com>
2022-05-26 08:54:05 -07:00
Shivam Mishra
0dd00e7883 fix extension endpoints
Signed-off-by: Shivam Mishra <shimish2@cisco.com>
2022-05-25 13:46:43 -07:00
Ramkumar Chinchani
6b841809e3 fix CVE-2022-29173/GHSA-66x3-6cw3-v5gj
https://github.com/project-zot/zot/security/dependabot/16

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-05-25 11:34:24 -07:00
Petu Eusebiu
da4acaf178 sync: preserve upstream digests after syncing images
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-05-25 10:19:36 -07:00
Petu Eusebiu
bd730150a8 sync: allow HTTP redirects when GETing signatures blobs
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-05-25 10:19:36 -07:00
Petu Eusebiu
aeb8a5da39 sync: specify contentType in headers when GETing cosign manifest
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-05-25 10:19:36 -07:00
Shivam Mishra
dcdeb935fd use zot as an extension name, ext as a component and search as a module
add endpoints field in ext discover api

distribution spec extension discover api has endpoints field required.

https://github.com/opencontainers/distribution-spec/blob/main/extensions/_oci.md#extensions-property-descriptions

Signed-off-by: Shivam Mishra <shimish2@cisco.com>
2022-05-24 19:12:40 -07:00
Petu Eusebiu
5e22acbbc4 s3: added logic for deduping blobs
Because s3 doesn't support hard links we store duplicated blobs
as empty files. When the original blob is deleted its content is
moved to the the next duplicated blob and so on.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-05-24 17:00:10 -07:00
Shivam Mishra
36c9631000 ext: use distribution spec route prefix for extension api
Following the spec defined here https://github.com/opencontainers/distribution-spec/tree/main/extensions

Signed-off-by: Shivam Mishra <shimish2@cisco.com>
2022-05-22 16:35:16 -07:00
laurentiuNiculae
7d8af50aec mocked tests for routes
Signed-off-by: laurentiuNiculae <themelopeus@gmail.com>
2022-05-20 13:27:09 -07:00
Ramkumar Chinchani
287ac05ddc update linter version to 1.46.2
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2022-05-20 11:53:56 -07:00
Petu Eusebiu
7c3a8f9d07 Report unknown keys when parsing configuration files
Report missing mandatory ldap keys

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-05-16 14:13:31 -07:00
Catalin Hofnar
20a60cbad4 Enhance sync logic - stop blob redownloads and re-pushes (#479 #480)
Changed imagesToCopyFromUpstream to return a map[string][]types.ImageReference from just an array of refs
Rewrote some logic in sync.go to use the new signature of imagesToCopyFromUpstream
Split getLocalImageRef by adding function getLocalCachePath
Adapted tests for new changes, added some tests
Merged #481

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2022-05-16 10:05:01 -07:00
Shivam Mishra
e04a9bf6e2 use TempDir instead of /tmp/zot in tests
Closes #508

Signed-off-by: Shivam Mishra <shimish2@cisco.com>
2022-05-05 10:38:53 -07:00
laurentiuNiculae
bb95af5b4d default policy only authorization
unit tests for manifest integrity when updating

Signed-off-by: laurentiuNiculae <themelopeus@gmail.com>
2022-05-03 11:46:55 -07:00
Alex Stan
d325c8b5f4 Fix problems signaled by new linter version v1.45.2
PR (linter: upgrade linter version #405) triggered lint job which failed
with many errors generated by various linters. Configurations were added to
golangcilint.yaml and several refactorings were made in order to improve the
results of the linter.

maintidx linter disabled

Signed-off-by: Alex Stan <alexandrustan96@yahoo.ro>
2022-04-27 09:55:44 -07:00
Andreea-Lupu
5e35dfa28f make gc periodic
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2022-04-18 10:25:29 -07:00
Petu Eusebiu
89c5f4f604 sync: fix functions which compare signatures
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-04-15 10:31:37 -07:00
Petu Eusebiu
4e20ab8a5d go.mod: update dependencies
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-04-15 10:31:37 -07:00
Petu Eusebiu
c62dae06c9 s3: fix initRepo not creating index.json in some edge cases
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-04-14 15:06:40 -07:00
laurentiuNiculae
0d4cc8736d Target for cheking not commited config files.
Signed-off-by: laurentiuNiculae <themelopeus@gmail.com>

Separated updateDistSpec functionality

Removed rewriting of config when distSpecVersion was wrong
2022-04-14 10:28:38 -07:00
Catalin Hofnar
475d97b1ad Separate make commands that require sudo
Reworked privileged cert test so it runs in Go by moving make logic to Go logic

Signed-off-by: Catalin Hofnar <catalin.hofnar@gmail.com>
2022-04-05 10:12:43 -07:00
Andreea-Lupu
9454c77be2 make scrub inline and periodic
Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2022-04-01 13:38:24 -07:00
Alexei Dodon
ad519e2d3e Leave zot repositories in a consistent state after zot hits fd limit closes #359
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2022-03-31 13:25:15 -07:00
Petu Eusebiu
03dd558ec6 sync: fix inconsistent test
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2022-03-31 09:19:07 -07:00