prosody-modules: mod_http_upload_external/README.markdown annotate

annotate mod_http_upload_external/README.markdown @ 4542:fb4a50bf60f1

mod_prometheus: Invoke stats collection if in 'manual' mode Since 10d13e0554f9 a special value for statistics_interval "manual" exists, where a module is expected to invoke processing in connection to collection of stats. This makes internal collection and exporting to Prometheus happens at the same time with no chance of timers getting out of sync.

author	Kim Alvefur <zash@zash.se>
date	Tue, 13 Apr 2021 23:53:53 +0200
parents	16995e7624f0
children	c149edb37349

rev	line source
2334 c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	1 ---
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	2 description: HTTP File Upload (external service)
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	3 labels: 'Stage-Alpha'
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	4 ---
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	5
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	6 Introduction
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	7 ============
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	8
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	9 This module implements [XEP-0363], which lets clients upload files
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	10 over HTTP to an external web server.
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	11
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	12 This module generates URLs that are signed using a HMAC. Any web service that can authenticate
4509 16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	13 these URLs can be used.
2823 f14bea5da323 mod_http_upload_external: add Python service implementation Jonas Wielicki <jonas@wielicki.name> parents: 2334 diff changeset	14
f14bea5da323 mod_http_upload_external: add Python service implementation Jonas Wielicki <jonas@wielicki.name> parents: 2334 diff changeset	15 Implementations
f14bea5da323 mod_http_upload_external: add Python service implementation Jonas Wielicki <jonas@wielicki.name> parents: 2334 diff changeset	16 ---------------
f14bea5da323 mod_http_upload_external: add Python service implementation Jonas Wielicki <jonas@wielicki.name> parents: 2334 diff changeset	17
f14bea5da323 mod_http_upload_external: add Python service implementation Jonas Wielicki <jonas@wielicki.name> parents: 2334 diff changeset	18 * [PHP implementation](https://hg.prosody.im/prosody-modules/raw-file/tip/mod_http_upload_external/share.php)
f14bea5da323 mod_http_upload_external: add Python service implementation Jonas Wielicki <jonas@wielicki.name> parents: 2334 diff changeset	19 * [Python3+Flask implementation](https://github.com/horazont/xmpp-http-upload)
3168 73a610c3c7a9 mod_http_external: Link to prosody-filer (Go implementation) Matthew Wild <mwild1@gmail.com> parents: 2823 diff changeset	20 * [Go implementation, Prosody Filer](https://github.com/ThomasLeister/prosody-filer)
3189 57332ea0c1c7 mod_http_upload_external/README: Add Perl implementation by Holger to list Kim Alvefur <zash@zash.se> parents: 3168 diff changeset	21 * [Perl implementation for nginx](https://github.com/weiss/ngx_http_upload)
2823 f14bea5da323 mod_http_upload_external: add Python service implementation Jonas Wielicki <jonas@wielicki.name> parents: 2334 diff changeset	22
4509 16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	23 To implement your own service compatible with this module, check out the implementation notes below
2823 f14bea5da323 mod_http_upload_external: add Python service implementation Jonas Wielicki <jonas@wielicki.name> parents: 2334 diff changeset	24 (and if you publish your implementation - let us know!).
2334 c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	25
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	26 Configuration
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	27 =============
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	28
3959 5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled Kim Alvefur <zash@zash.se> parents: 3360 diff changeset	29 The module can be added as a new Component definition:
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled Kim Alvefur <zash@zash.se> parents: 3360 diff changeset	30
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled Kim Alvefur <zash@zash.se> parents: 3360 diff changeset	31 ``` {.lua}
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled Kim Alvefur <zash@zash.se> parents: 3360 diff changeset	32 Component "upload.example.org" "http_upload_external"
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled Kim Alvefur <zash@zash.se> parents: 3360 diff changeset	33 http_upload_external_base_url = "https://your.example.com/upload/service"
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled Kim Alvefur <zash@zash.se> parents: 3360 diff changeset	34 http_upload_external_secret = "your shared secret"
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled Kim Alvefur <zash@zash.se> parents: 3360 diff changeset	35 ```
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled Kim Alvefur <zash@zash.se> parents: 3360 diff changeset	36
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled Kim Alvefur <zash@zash.se> parents: 3360 diff changeset	37 It should not be added to modules_enabled.
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled Kim Alvefur <zash@zash.se> parents: 3360 diff changeset	38
2334 c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	39
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	40 External URL
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	41 ------------
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	42
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	43 You need to provide the path to the external service. Ensure it ends with '/'.
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	44
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	45 For example, to use the PHP implementation linked above, you might set it to:
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	46
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	47 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	48 http_upload_external_base_url = "https://your.example.com/path/to/share.php/"
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	49 ```
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	50
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	51 Secret
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	52 ------
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	53
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	54 Set a long and unpredictable string as your secret. This is so the upload service can verify that
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	55 the upload comes from mod_http_upload_external, and random strangers can't upload to your server.
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	56
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	57 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	58 http_upload_external_secret = "this is a secret string!"
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	59 ```
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	60
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	61 You need to set exactly the same secret string in your external service.
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	62
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	63 Limits
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	64 ------
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	65
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	66 A maximum file size can be set by:
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	67
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	68 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	69 http_upload_external_file_size_limit = 123 -- bytes
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	70 ```
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	71
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	72 Default is 100MB (100\1024\1024).
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	73
4509 16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	74 Access
16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	75 ------
16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	76
16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	77 You may want to give upload access to additional entities such as components
16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	78 by using the `http_upload_access` config option.
16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	79
16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	80 ``` {.lua}
16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	81 http_upload_access = {"gateway.example.com"};
16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	82 ```
16995e7624f0 mod_http_upload_external: add access control option Nicolas Cedilnik <nicoco@nicoco.fr> parents: 3959 diff changeset	83
2334 c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	84 Compatibility
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	85 =============
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	86
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	87 Works with Prosody 0.9.x and later.
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	88
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	89 Implementation
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	90 ==============
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	91
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	92 To implement your own external service that is compatible with this module, you need to expose a
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	93 simple API that allows the HTTP GET, HEAD and PUT methods on arbitrary URLs located on your service.
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	94
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	95 For example, if http_upload_external_base_url is set to `https://example.com/upload/` then your service
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	96 might receive the following requests:
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	97
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	98 Upload a new file:
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	99
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	100 ```
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	101 PUT https://example.com/upload/foo/bar.jpg?v=49e9309ff543ace93d25be90635ba8e9965c4f23fc885b2d86c947a5d59e55b2
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	102 ```
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	103
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	104 Recipient checks the file size and other headers:
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	105
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	106 ```
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	107 HEAD https://example.com/upload/foo/bar.jpg
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	108 ```
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	109
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	110 Recipient downloads the file:
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	111
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	112 ```
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	113 GET https://example.com/upload/foo/bar.jpg
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	114 ```
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	115
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	116 The only tricky logic is in validation of the PUT request. Firstly, don't overwrite existing files (return 409 Conflict).
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	117
3358 e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	118 Then you need to validate the auth token.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	119
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	120 ### Validating the auth token
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	121
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	122
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	123 \| Version \| Supports \|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	124 \|:--------\|:--------------------------------------------------------------------------------------------------------\|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	125 \| v \| Validates only filename and size. Does not support file type restrictions by the XMPP server. \|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	126 \| v2 \| Validates the filename, size and MIME type. This allows the server to implement MIME type restrictions. \|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	127
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	128 It is probable that a future v3 will be specified that allows carrying information about the uploader identity, allowing
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	129 the implementation of per-user quotas and limits.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	130
3360 0149954cee37 mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions Matthew Wild <mwild1@gmail.com> parents: 3359 diff changeset	131 Implementations may implement one or more versions of the protocol simultaneously. The XMPP server generates the URLs and
0149954cee37 mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions Matthew Wild <mwild1@gmail.com> parents: 3359 diff changeset	132 ultimately selects which version will be used.
0149954cee37 mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions Matthew Wild <mwild1@gmail.com> parents: 3359 diff changeset	133
0149954cee37 mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions Matthew Wild <mwild1@gmail.com> parents: 3359 diff changeset	134 XMPP servers MUST only generate URLs with one of the versions listed here. However in case multiple parameters are
0149954cee37 mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions Matthew Wild <mwild1@gmail.com> parents: 3359 diff changeset	135 present, upload services MUST only use the token from the highest parameter version that they support.
3358 e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	136
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	137 #### Version 1 (v)
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	138
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	139 The token will be in the URL query parameter 'v'. If it is absent, fail with 403 Forbidden.
2334 c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	140
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	141 Calculate the expected auth token by reading the value of the Content-Length header of the PUT request. E.g. for a 1MB file
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	142 will have a Content-Length of '1048576'. Append this to the uploaded file name, separated by a space (0x20) character.
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	143
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	144 For the above example, you would end up with the following string: "foo/bar.jpg 1048576"
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	145
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	146 The auth token is a SHA256 HMAC of this string, using the configured secret as the key. E.g.
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	147
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	148 ```
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	149 calculated_auth_token = hmac_sha256("foo/bar.jpg 1048576", "secret string")
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	150 ```
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	151
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	152 If this is not equal to the 'v' parameter provided in the upload URL, reject the upload with 403 Forbidden.
c728b2f77c7c mod_http_upload_external: Add README Matthew Wild <mwild1@gmail.com> parents: diff changeset	153
3358 e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	154 Security note: When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	155 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	156 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	157
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	158 #### Version 2 (v2)
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	159
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	160 The token will be in the URL query parameter 'v2'. If it is absent, fail with 403 Forbidden.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	161
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	162 \| Input \| Example \|Read from \|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	163 \|:--------------\|:------------\|:--------------------------------------------------------------------\|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	164 \|`file_path` \| foo/bar.jpg \| The URL of the PUT request, with the service's base prefix removed. \|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	165 \|`content_size` \| 1048576 \| Content-Size header \|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	166 \|`content_type` \| image/jpeg \| Content-Type header \|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	167
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	168 The parameters should be joined into a single string, separated by NUL bytes (`\0`):
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	169
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	170 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	171 signed_string = ( file_path + '\0' + content_size + '\0' + content_type )
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	172 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	173
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	174 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	175 signed_string = "foo/bar.jpg\01048576\0image/jpeg"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	176 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	177
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	178 The expected auth token is the SHA256 HMAC of this string, using the configured secret key as the key. E.g.:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	179
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	180 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	181 calculated_auth_token = hmac_sha256(signed_string, "secret string")
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	182 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	183
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	184 If this is not equal to the 'v2' parameter provided in the upload URL, reject the upload with 403 Forbidden.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	185
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	186 Security note: When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	187 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	188 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	189
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	190 ### Security considerations
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	191
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	192 #### HTTPS
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	193
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	194 All uploads and downloads should only be over HTTPS. The security of the served content is protected only
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	195 by the uniqueness present in the URLs themselves, and not using HTTPS may leak the URLs and contents to third-parties.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	196
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	197 Implementations should consider including HSTS and HPKP headers, with consent of the administrator.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	198
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	199 #### MIME types
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	200
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	201 If the upload Content-Type header matches any of the following MIME types, it MUST be preserved and included in the Content-Type
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	202 of any GET requests made to download the file:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	203
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	204 - `image/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	205 - `video/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	206 - `audio/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	207 - `text/plain`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	208
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	209 It is recommended that other MIME types are preserved, but served with the addition of the following header:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	210
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	211 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	212 Content-Disposition: attachment
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	213 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	214
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	215 This prevents the browser interpreting scripts and other resources that may potentially be malicious.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	216
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	217 Some browsers may also benefit from explicitly telling them not to try guessing the type of a file:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	218
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	219 ```
3359 3d01ab6b1186 mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>) Matthew Wild <mwild1@gmail.com> parents: 3358 diff changeset	220 X-Content-Type-Options: nosniff
3358 e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	221 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	222
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	223 #### Security headers
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	224
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	225 The following headers should be included to provide additional sandboxing of resources, considering the uploaded
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	226 content is not understood or trusted by the upload service:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	227
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	228 ```
3359 3d01ab6b1186 mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>) Matthew Wild <mwild1@gmail.com> parents: 3358 diff changeset	229 Content-Security-Policy: default-src 'none'
3d01ab6b1186 mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>) Matthew Wild <mwild1@gmail.com> parents: 3358 diff changeset	230 X-Content-Security-Policy: default-src 'none'
3d01ab6b1186 mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>) Matthew Wild <mwild1@gmail.com> parents: 3358 diff changeset	231 X-WebKit-CSP: default-src 'none'
3358 e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details Matthew Wild <mwild1@gmail.com> parents: 3189 diff changeset	232 ```

Mercurial > prosody-modules

annotate mod_http_upload_external/README.markdown @ 4542:fb4a50bf60f1