doc: initial draft of plugin design

docs/plugin-design.md

@@ -0,0 +1,141 @@
+# Plugin Design Doc
+
+## Objective
+
+Rearchitect imageproxy to use a plugin-based system for most features like
+transformations, security, and caching. This should reduce build times and
+binary sizes in the common case, and provide a mechanism for users to easily
+add custom features that would not be added to core for various reasons.
+
+## Background
+
+I created imageproxy to [scratch a personal itch](https://wjn.me/b/J_), I
+needed a simple way to dynamically resize images for my personal website. I
+published it as an open source projects because that's what I do, and I'm happy
+to see others finding it useful for their needs as well.
+
+But inevitably, with more users came requests for additional features because
+people have different use cases and requirements. Some of these requests were
+relatively minor, and I was happy to add them. But one of the more common
+requests was to support different caching backends. Personally, I still use the
+on-disk cache, but many people wanted to use redis or a cloud provider like
+AWS, Azure, or GCP. For a long time I was resistant to adding support for
+these, mainly out of concern for inflating build times and binary sizes. I did
+eventually relent, and
+[#49](https://github.com/willnorris/imageproxy/issues/49) tracked adding
+support for the most common backends.
+
+Unfortunately my concerns proved true, and build times are *significantly*
+slower (TODO: add concrete numbers) now because of all the additional cloud
+SDKs that get compiled in. I don't personally care too much about binary size,
+since I'm not running in a constrained environment, but these build times are
+really wearing on me. Additionally, there are a number of outstanding pull
+requests for relatively obscure features that I don't really want to have to
+support in the main project. And quite honestly, there are a number of obscure
+features that did get merged in over the years that I kinda wish I could rip
+back out.
+
+### Plugin support in Go
+
+TODO: talk about options like
+ - RPC (https://github.com/hashicorp/go-plugin)
+ - pkg/plugin (https://golang.org/pkg/plugin/)
+ - embedded interpreter (https://github.com/robertkrimen/otto)
+ - custom binaries (https://github.com/mholt/caddy,
+   https://caddy.community/t/59)
+
+Spoiler: I'm planning on following the Caddy approach and using custom
+binaries.
+
+## Design
+
+I plan to model imageproxy after Caddy, moving all key functionality into
+separate plugins that register themselves with the server, and which all
+compile to a single statically-linked binary.  The core project will provide a
+great number of plugins to cover all of the existing functionality.  I also
+expect I'll be much more open to adding plugins for features I may not care as
+much about personally. Of course, users can also write their own plugins and
+link them in without needing to contribute them to core if they don't want to.
+
+I anticipate providing two or three build configurations in core:
+ - **full** - include all the plugins that are part of core (except where they
+   may conflict)
+ - **minimal** - some set of minimal features that only includes basic caching
+   options, limited transformation options, etc
+ - **my personal config** - I'll also definitely have a build that I use
+   personally on my site. I may decide to just make that the "minimal" build
+   and perhaps call it something different, rather than have a third
+   configuration.
+
+Custom configurations beyond what is provided by core can be done by creating a
+minimal main package that imports the plugins you care about and calling some
+kind of bootstrap method (similar to [what Caddy now
+does](https://caddy.community/t/59)).
+
+### Types of plugins
+
+(Initially in no particular order, just capturing thoughts. Lots to do here in
+thinking through the use cases and what kind of plugin API we really need to
+provide.)
+
+#### Caching backend
+
+This is one of the most common feature requests, and is also one of the worst
+offender for inflating build times and binary sizes because of the size of the
+dependencies that are typically required.  The minimal imageproxy build would
+probably only include the in-memory and on-disk caches. Anything that talked to
+an external store (redis, cloud providers, etc) would be pulled out.
+
+#### Transformation engine
+
+Today, imageproxy only performs transformations which can be done with pure Go
+libraries. There have been a number of requests (or at least questions) to use
+something like (vips)[https://github.com/DAddYE/vips] or
+(imagemagick)[https://github.com/gographics/imagick], which are both C
+libraries. They provide more options, and (likely) better performance, at the
+cost of complexity and loss of portability in using cgo. These would likely
+replace the entire transformation engine in imageproxy, so I don't know how
+they would interact with other plugins that merely extend the main engine (they
+probably wouldn't be able to interact at all).
+
+#### Transformation options
+
+Today, imageproxy performs minimal transformations, mostly around resizing,
+cropping, and rotation.  It doesn't support any kind of filters, brightness or
+contrast adjustment, etc. There are go libraries for them, they're just outside
+the scope of what I originally intended imageproxy for.  But I'd be happy to
+have plugins that do that kind of thing. These plugins would need to be able to
+hook into the option parsing engine so that they could register their URL
+options.
+
+#### Image format support
+
+There have been a number of requests for imge format support that require cgo
+libraries:
+
+ - **webp encoding** - needs cgo
+   [#114](https://github.com/willnorris/imageproxy/issues/114)
+ - **progressive jpegs** - probably needs cgo?
+   [#77](https://github.com/willnorris/imageproxy/issues/77)
+ - **gif to mp4** - maybe doable in pure go, but probably belongs in a plugin
+   [#136](https://github.com/willnorris/imageproxy/issues/136)
+ - **HEIF** - formate used by newer iPhones
+   ([HEIF](https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format))
+
+#### Option parsing
+
+Today, options are specified as the first component in the URL path, but
+[#66](https://github.com/willnorris/imageproxy/pull/66) proposes optionally
+moving that to a query parameter (for a good reason, actually). Maybe putting
+that in core is okay? Maybe it belongs in a plugin, in which case we'd need to
+expose an API for replacing the option parsing code entirely.
+
+#### Security options
+
+Some people want to add a host blacklist
+[#85](https://github.com/willnorris/imageproxy/pull/85), refusal to process
+non-image files [#53](https://github.com/willnorris/imageproxy/issues/53)
+[#119](https://github.com/willnorris/imageproxy/pull/119). I don't think there
+is an issue for it, but an early fork of the project added request signing that
+was compatible with nginx's [secure link
+module](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html).