diff --git a/README.md b/README.md index 2107649..eb027e3 100644 --- a/README.md +++ b/README.md @@ -5,18 +5,18 @@ [![Test Coverage](https://codecov.io/gh/willnorris/imageproxy/branch/main/graph/badge.svg)](https://codecov.io/gh/willnorris/imageproxy) [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/2611/badge)](https://bestpractices.coreinfrastructure.org/projects/2611) -imageproxy is a caching image proxy server written in go. It features: +imageproxy is a caching image proxy server written in go. It features: - - basic image adjustments like resizing, cropping, and rotation - - access control using allowed hosts list or request signing (HMAC-SHA256) - - support for jpeg, png, webp (decode only), tiff, and gif image formats - (including animated gifs) - - caching in-memory, on disk, or with Amazon S3, Google Cloud Storage, Azure - Storage, or Redis - - easy deployment, since it's pure go +- basic image adjustments like resizing, cropping, and rotation +- access control using allowed hosts list or request signing (HMAC-SHA256) +- support for jpeg, png, webp (decode only), tiff, and gif image formats + (including animated gifs) +- caching in-memory, on disk, or with Amazon S3, Google Cloud Storage, Azure + Storage, or Redis +- easy deployment, since it's pure go Personally, I use it primarily to dynamically resize images hosted on my own -site (read more in [this post][]). But you can also enable request signing and +site (read more in [this post][]). But you can also enable request signing and use it as an SSL proxy for remote images, similar to [atmos/camo][] but with additional image adjustment options. @@ -29,24 +29,24 @@ You can see the go versions that are tested against in [.github/workflows/tests. [most recent major go releases]: https://golang.org/doc/devel/release.html [.github/workflows/tests.yml]: ./.github/workflows/tests.yml -## URL Structure ## +## URL Structure imageproxy URLs are of the form `http://localhost/{options}/{remote_url}`. -### Options ### +### Options Options are available for cropping, resizing, rotation, flipping, and digital -signatures among a few others. Options for are specified as a comma delimited -list of parameters, which can be supplied in any order. Duplicate parameters +signatures among a few others. Options for are specified as a comma delimited +list of parameters, which can be supplied in any order. Duplicate parameters overwrite previous values. See the full list of available options at . -### Remote URL ### +### Remote URL The URL of the original image to load is specified as the remainder of the -path, without any encoding. For example, +path, without any encoding. For example, `http://localhost/200/https://willnorris.com/logo.jpg`. In order to [optimize caching][], it is recommended that URLs not contain query @@ -54,45 +54,45 @@ strings. [optimize caching]: http://www.stevesouders.com/blog/2008/08/23/revving-filenames-dont-use-querystring/ -### Examples ### +### Examples The following live examples demonstrate setting different options on [this source image][small-things], which measures 1024 by 678 pixels. [small-things]: https://willnorris.com/images/imageproxy/small-things.jpg -Options | Meaning | Image ---------|------------------------------------------|------ -200x | 200px wide, proportional height | 200x -x0.15 | 15% original height, proportional width | x0.15 -100x150 | 100 by 150 pixels, cropping as needed | 100x150 -100 | 100px square, cropping as needed | 100 -150,fit | scale to fit 150px square, no cropping | 150,fit -100,r90 | 100px square, rotated 90 degrees | 100,r90 -100,fv,fh | 100px square, flipped horizontal and vertical | 100,fv,fh -200x,q60 | 200px wide, proportional height, 60% quality | 200x,q60 -200x,png | 200px wide, converted to PNG format | 200x,png -cx175,cw400,ch300,100x | crop to 400x300px starting at (175,0), scale to 100px wide | cx175,cw400,ch300,100x +| Options | Meaning | Image | +| ---------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 200x | 200px wide, proportional height | 200x | +| x0.15 | 15% original height, proportional width | x0.15 | +| 100x150 | 100 by 150 pixels, cropping as needed | 100x150 | +| 100 | 100px square, cropping as needed | 100 | +| 150,fit | scale to fit 150px square, no cropping | 150,fit | +| 100,r90 | 100px square, rotated 90 degrees | 100,r90 | +| 100,fv,fh | 100px square, flipped horizontal and vertical | 100,fv,fh | +| 200x,q60 | 200px wide, proportional height, 60% quality | 200x,q60 | +| 200x,png | 200px wide, converted to PNG format | 200x,png | +| cx175,cw400,ch300,100x | crop to 400x300px starting at (175,0), scale to 100px wide | cx175,cw400,ch300,100x | The [smart crop feature](https://godoc.org/willnorris.com/go/imageproxy#hdr-Smart_Crop) can best be seen by comparing crops of [this source image][judah-sheets], with and without smart crop enabled. -Options | Meaning | Image ---------|------------------------------------------|------ -150x300 | 150x300px, standard crop | 200x400,sc -150x300,sc | 150x300px, smart crop | 200x400 +| Options | Meaning | Image | +| ---------- | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 150x300 | 150x300px, standard crop | 200x400,sc | +| 150x300,sc | 150x300px, smart crop | 200x400 | [judah-sheets]: https://judahnorris.com/images/judah-sheets.jpg -Transformation also works on animated gifs. Here is [this source +Transformation also works on animated gifs. Here is [this source image][material-animation] resized to 200px square and rotated 270 degrees: [material-animation]: https://willnorris.com/images/imageproxy/material-animations.gif 200,r270 -## Getting Started ## +## Getting Started Install the package using: @@ -104,84 +104,85 @@ using: imageproxy This will start the proxy on port 8080, without any caching and with no allowed -host list (meaning any remote URL can be proxied). Test this by navigating to +host list (meaning any remote URL can be proxied). Test this by navigating to and you should see a 500px square coder octocat. -### Cache ### +### Cache By default, the imageproxy command does not cache responses, but caching can be -enabled using the `-cache` flag. It supports the following values: +enabled using the `-cache` flag. It supports the following values: - - `memory` - uses an in-memory LRU cache. By default, this is limited to - 100mb. To customize the size of the cache or the max age for cached items, - use the format `memory:size:age` where size is measured in mb and age is a - duration. For example, `memory:200:4h` will create a 200mb cache that will - cache items no longer than 4 hours. - - directory on local disk (e.g. `/tmp/imageproxy`) - will cache images - on disk +- `memory` - uses an in-memory LRU cache. By default, this is limited to + 100mb. To customize the size of the cache or the max age for cached items, + use the format `memory:size:age` where size is measured in mb and age is a + duration. For example, `memory:200:4h` will create a 200mb cache that will + cache items no longer than 4 hours. +- directory on local disk (e.g. `/tmp/imageproxy`) - will cache images + on disk - - s3 URL (e.g. `s3://region/bucket-name/optional-path-prefix`) - will cache - images on Amazon S3. This requires either an IAM role and instance profile - with access to your your bucket or `AWS_ACCESS_KEY_ID` and `AWS_SECRET_KEY` - environmental variables be set. (Additional methods of loading credentials - are documented in the [aws-sdk-go session - package](https://docs.aws.amazon.com/sdk-for-go/api/aws/session/)). +- s3 URL (e.g. `s3://region/bucket-name/optional-path-prefix`) - will cache + images on Amazon S3. This requires either an IAM role and instance profile + with access to your your bucket or `AWS_ACCESS_KEY_ID` and `AWS_SECRET_KEY` + environmental variables be set. (Additional methods of loading credentials + are documented in the [aws-sdk-go session + package](https://docs.aws.amazon.com/sdk-for-go/api/aws/session/)). - Additional configuration options ([further documented here][aws-options]) - may be specified as URL query string parameters, which are mostly useful - when working with s3-compatible services: - - "endpoint" - specify an alternate API endpoint - - "disableSSL" - set to "1" to disable SSL when calling the API - - "s3ForcePathStyle" - set to "1" to force the request to use path-style addressing + Additional configuration options ([further documented here][aws-options]) + may be specified as URL query string parameters, which are mostly useful + when working with s3-compatible services: - For example, when working with [minio](https://minio.io), which doesn't use - regions, provide a dummy region value and custom endpoint value: + - "endpoint" - specify an alternate API endpoint + - "disableSSL" - set to "1" to disable SSL when calling the API + - "s3ForcePathStyle" - set to "1" to force the request to use path-style addressing - s3://fake-region/bucket/folder?endpoint=minio:9000&disableSSL=1&s3ForcePathStyle=1 + For example, when working with [minio](https://minio.io), which doesn't use + regions, provide a dummy region value and custom endpoint value: - Similarly, for [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces/), - provide a dummy region value and the appropriate endpoint for your space: + s3://fake-region/bucket/folder?endpoint=minio:9000&disableSSL=1&s3ForcePathStyle=1 - s3://fake-region/bucket/folder?endpoint=sfo2.digitaloceanspaces.com + Similarly, for [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces/), + provide a dummy region value and the appropriate endpoint for your space: - [aws-options]: https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config + s3://fake-region/bucket/folder?endpoint=sfo2.digitaloceanspaces.com - - gcs URL (e.g. `gcs://bucket-name/optional-path-prefix`) - will cache images - on Google Cloud Storage. Authentication is documented in Google's - [Application Default Credentials - docs](https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application). - - azure URL (e.g. `azure://container-name/`) - will cache images on - Azure Storage. This requires `AZURESTORAGE_ACCOUNT_NAME` and - `AZURESTORAGE_ACCESS_KEY` environment variables to bet set. - - redis URL (e.g. `redis://hostname/`) - will cache images on - the specified redis host. The full URL syntax is defined by the [redis URI - registration](https://www.iana.org/assignments/uri-schemes/prov/redis). - Rather than specify password in the URI, use the `REDIS_PASSWORD` - environment variable. + [aws-options]: https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config + +- gcs URL (e.g. `gcs://bucket-name/optional-path-prefix`) - will cache images + on Google Cloud Storage. Authentication is documented in Google's + [Application Default Credentials + docs](https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application). +- azure URL (e.g. `azure://container-name/`) - will cache images on + Azure Storage. This requires `AZURESTORAGE_ACCOUNT_NAME` and + `AZURESTORAGE_ACCESS_KEY` environment variables to bet set. +- redis URL (e.g. `redis://hostname/`) - will cache images on + the specified redis host. The full URL syntax is defined by the [redis URI + registration](https://www.iana.org/assignments/uri-schemes/prov/redis). + Rather than specify password in the URI, use the `REDIS_PASSWORD` + environment variable. For example, to cache files on disk in the `/tmp/imageproxy` directory: imageproxy -cache /tmp/imageproxy Reload the [codercat URL][], and then inspect the contents of -`/tmp/imageproxy`. Within the subdirectories, there should be two files, one +`/tmp/imageproxy`. Within the subdirectories, there should be two files, one for the original full-size codercat image, and one for the resized 500px version. [codercat URL]: http://localhost:8080/500/https://octodex.github.com/images/codercat.jpg Multiple caches can be specified by separating them by spaces or by repeating -the `-cache` flag multiple times. The caches will be created in a [tiered +the `-cache` flag multiple times. The caches will be created in a [tiered fashion][]. Typically this is used to put a smaller and faster in-memory cache -in front of a larger but slower on-disk cache. For example, the following will +in front of a larger but slower on-disk cache. For example, the following will first check an in-memory cache for an image, followed by a gcs bucket: imageproxy -cache memory -cache gcs://my-bucket/ [tiered fashion]: https://godoc.org/github.com/die-net/lrucache/twotier -### Allowed Referrer List ### +### Allowed Referrer List You can limit images to only be accessible for certain hosts in the HTTP referrer header, which can help prevent others from hotlinking to images. It can @@ -189,16 +190,15 @@ be enabled by running: imageproxy -referrers example.com - -Reload the [codercat URL][], and you should now get an error message. You can +Reload the [codercat URL][], and you should now get an error message. You can specify multiple hosts as a comma separated list, or prefix a host value with `*.` to allow all sub-domains as well. -### Allowed and Denied Hosts List ### +### Allowed and Denied Hosts List You can limit the remote hosts that the proxy will fetch images from using the -`allowHosts` and `denyHosts` flags. This is useful, for example, for locking -the proxy down to your own hosts to prevent others from abusing it. Of course +`allowHosts` and `denyHosts` flags. This is useful, for example, for locking +the proxy down to your own hosts to prevent others from abusing it. Of course if you want to support fetching from any host, leave off these flags. Try it out by running: @@ -219,7 +219,7 @@ blocking reserved ranges like `127.0.0.0/8`, `192.168.0.0/16`, etc. If a host matches both an allowed and denied host, the request will be denied. -### Allowed Content-Type List ### +### Allowed Content-Type List You can limit what content types can be proxied by using the `contentTypes` flag. By default, this is set to `image/*`, meaning that imageproxy will @@ -227,16 +227,16 @@ process any image types. You can specify multiple content types as a comma separated list, and suffix values with `*` to perform a wildcard match. Set the flag to an empty string to proxy all requests, regardless of content type. -### Signed Requests ### +### Signed Requests -Instead of an allowed host list, you can require that requests be signed. This +Instead of an allowed host list, you can require that requests be signed. This is useful in preventing abuse when you don't have just a static list of hosts -you want to allow. Signatures are generated using HMAC-SHA256 against the +you want to allow. Signatures are generated using HMAC-SHA256 against the remote URL, and url-safe base64 encoding the result: base64urlencode(hmac.New(sha256, ).digest()) -The HMAC key is specified using the `signatureKey` flag. If this flag +The HMAC key is specified using the `signatureKey` flag. If this flag begins with an "@", the remainder of the value is interpreted as a file on disk which contains the HMAC key. @@ -244,7 +244,7 @@ Try it out by running: imageproxy -signatureKey "secretkey" -Reload the [codercat URL][], and you should see an error message. Now load a +Reload the [codercat URL][], and you should see an error message. Now load a [signed codercat URL][] (which contains the [signature option][]) and verify that it loads properly. @@ -252,9 +252,9 @@ that it loads properly. [signature option]: https://godoc.org/willnorris.com/go/imageproxy#hdr-Signature Some simple code samples for generating signatures in various languages can be -found in [docs/url-signing.md](/docs/url-signing.md). Multiple valid signature +found in [docs/url-signing.md](/docs/url-signing.md). Multiple valid signature keys may be provided to support key rotation by repeating the `signatureKey` -flag multiple times, or by providing a space-separated list of keys. To use a +flag multiple times, or by providing a space-separated list of keys. To use a key with a literal space character, load the key from a file using the "@" prefix documented above. @@ -262,34 +262,34 @@ If both a whiltelist and signatureKey are specified, requests can match either. In other words, requests that match one of the allowed hosts don't necessarily need to be signed, though they can be. -### Default Base URL ### +### Default Base URL Typically, remote images to be proxied are specified as absolute URLs. However, if you commonly proxy images from a single source, you can provide a -base URL and then specify remote images relative to that base. Try it out by +base URL and then specify remote images relative to that base. Try it out by running: imageproxy -baseURL https://octodex.github.com/ Then load the codercat image, specified as a URL relative to that base: -. Note that this is not an +. Note that this is not an effective method to mask the true source of the images being proxied; it is -trivial to discover the base URL being used. Even when a base URL is +trivial to discover the base URL being used. Even when a base URL is specified, you can always provide the absolute URL of the image to be proxied. -### Scaling beyond original size ### +### Scaling beyond original size By default, the imageproxy won't scale images beyond their original size. However, you can use the `scaleUp` command-line flag to allow this to happen: imageproxy -scaleUp true -### WebP and TIFF support ### +### WebP and TIFF support Imageproxy can proxy remote webp images, but they will be served in either jpeg or png format (this is because the golang webp library only supports webp -decoding) if any transformation is requested. If no format is specified, -imageproxy will use jpeg by default. If no transformation is requested (for +decoding) if any transformation is requested. If no format is specified, +imageproxy will use jpeg by default. If no transformation is requested (for example, if you are just using imageproxy as an SSL proxy) then the original webp image will be served as-is without any format conversion. @@ -298,49 +298,49 @@ default if any transformation is requested. To force encoding as tiff, pass the "tiff" option. Like webp, tiff images will be served as-is without any format conversion if no transformation is requested. - -Run `imageproxy -help` for a complete list of flags the command accepts. If +Run `imageproxy -help` for a complete list of flags the command accepts. If you want to use a different caching implementation, it's probably easiest to just make a copy of `cmd/imageproxy/main.go` and customize it to fit your needs... it's a very simple command. -### Environment Variables ### +### Environment Variables All configuration flags have equivalent environment variables of the form -`IMAGEPROXY_$NAME`. For example, an on-disk cache could be configured by calling +`IMAGEPROXY_$NAME`. For example, an on-disk cache could be configured by calling IMAGEPROXY_CACHE="/tmp/imageproxy" imageproxy -## Deploying ## +## Deploying In most cases, you can follow the normal procedure for building a deploying any -go application. For example: +go application. For example: - - `go build willnorris.com/go/imageproxy/cmd/imageproxy` - - copy resulting binary to `/usr/local/bin` - - copy [`etc/imageproxy.service`](etc/imageproxy.service) to - `/lib/systemd/system` and enable using `systemctl`. +- `go build willnorris.com/go/imageproxy/cmd/imageproxy` +- copy resulting binary to `/usr/local/bin` +- copy [`etc/imageproxy.service`](etc/imageproxy.service) to + `/lib/systemd/system` and enable using `systemctl`. Instructions have been contributed below for running on other platforms, but I don't have much experience with them personally. -### Heroku ### +### Heroku It's easy to vendorize the dependencies with `Godep` and deploy to Heroku. Take a look at [this GitHub repo](https://github.com/oreillymedia/prototype-imageproxy/tree/heroku) (make sure you use the `heroku` branch). -### AWS Elastic Beanstalk ### +### AWS Elastic Beanstalk [O’Reilly Media](https://github.com/oreillymedia) set up [a repository](https://github.com/oreillymedia/prototype-imageproxy) with everything you need to deploy imageproxy to Elastic Beanstalk. Just follow the instructions in the [README](https://github.com/oreillymedia/prototype-imageproxy/blob/master/Readme.md). -### Docker ### +### Docker A docker image is available at [`ghcr.io/willnorris/imageproxy`](https://github.com/willnorris/imageproxy/pkgs/container/imageproxy). You can run it by + ``` docker run -p 8080:8080 ghcr.io/willnorris/imageproxy -addr 0.0.0.0:8080 ``` @@ -353,14 +353,14 @@ ENTRYPOINT ["/app/imageproxy", "-addr 0.0.0.0:8080"] If running imageproxy inside docker with a bind-mounted on-disk cache, make sure the container is running as a user that has write permission to the mounted host -directory. See more details in +directory. See more details in [#198](https://github.com/willnorris/imageproxy/issues/198). Note that all configuration options can be set using [environment variables](#environment-variables), which is often the preferred approach for containers. -### nginx ### +### nginx Use the `proxy_pass` directive to send requests to your imageproxy instance. For example, to run imageproxy at the path "/api/imageproxy/", set: @@ -380,12 +380,13 @@ to alter the precedence order by setting: } ``` -## Clients ## +## Clients + - [Ruby](https://github.com/azolf/imageproxy_ruby) -## License ## +## License imageproxy is copyright its respective authors. All of my personal work on imageproxy through 2020 (which accounts for the majority of the code) is -copyright Google, my employer at the time. It is available under the [Apache +copyright Google, my employer at the time. It is available under the [Apache 2.0 License](./LICENSE).