Mercurial > prosody-modules
annotate mod_http_upload_external/README.markdown @ 4411:c3d21182ebf3
mod_compliance_2021: XEP-0443: XMPP Compliance Suites 2021 self-test
author | Kim Alvefur <zash@zash.se> |
---|---|
date | Sat, 30 Jan 2021 01:10:06 +0100 |
parents | 5741e6511f3d |
children | 16995e7624f0 |
rev | line source |
---|---|
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
1 --- |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
2 description: HTTP File Upload (external service) |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
3 labels: 'Stage-Alpha' |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
4 --- |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
5 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
6 Introduction |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
7 ============ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
8 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
9 This module implements [XEP-0363], which lets clients upload files |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
10 over HTTP to an external web server. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
11 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
12 This module generates URLs that are signed using a HMAC. Any web service that can authenticate |
2823
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
13 these URLs can be used. |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
14 |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
15 Implementations |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
16 --------------- |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
17 |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
18 * [PHP implementation](https://hg.prosody.im/prosody-modules/raw-file/tip/mod_http_upload_external/share.php) |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
19 * [Python3+Flask implementation](https://github.com/horazont/xmpp-http-upload) |
3168
73a610c3c7a9
mod_http_external: Link to prosody-filer (Go implementation)
Matthew Wild <mwild1@gmail.com>
parents:
2823
diff
changeset
|
20 * [Go implementation, Prosody Filer](https://github.com/ThomasLeister/prosody-filer) |
3189
57332ea0c1c7
mod_http_upload_external/README: Add Perl implementation by Holger to list
Kim Alvefur <zash@zash.se>
parents:
3168
diff
changeset
|
21 * [Perl implementation for nginx](https://github.com/weiss/ngx_http_upload) |
2823
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
22 |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
23 To implement your own service compatible with this module, check out the implementation notes below |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
24 (and if you publish your implementation - let us know!). |
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
25 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
26 Configuration |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
27 ============= |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
28 |
3959
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
29 The module can be added as a new Component definition: |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
30 |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
31 ``` {.lua} |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
32 Component "upload.example.org" "http_upload_external" |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
33 http_upload_external_base_url = "https://your.example.com/upload/service" |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
34 http_upload_external_secret = "your shared secret" |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
35 ``` |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
36 |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
37 It should **not** be added to modules_enabled. |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
38 |
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
39 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
40 External URL |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
41 ------------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
42 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
43 You need to provide the path to the external service. Ensure it ends with '/'. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
44 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
45 For example, to use the PHP implementation linked above, you might set it to: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
46 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
47 ``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
48 http_upload_external_base_url = "https://your.example.com/path/to/share.php/" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
49 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
50 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
51 Secret |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
52 ------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
53 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
54 Set a long and unpredictable string as your secret. This is so the upload service can verify that |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
55 the upload comes from mod_http_upload_external, and random strangers can't upload to your server. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
56 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
57 ``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
58 http_upload_external_secret = "this is a secret string!" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
59 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
60 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
61 You need to set exactly the same secret string in your external service. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
62 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
63 Limits |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
64 ------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
65 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
66 A maximum file size can be set by: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
67 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
68 ``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
69 http_upload_external_file_size_limit = 123 -- bytes |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
70 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
71 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
72 Default is 100MB (100\*1024\*1024). |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
73 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
74 Compatibility |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
75 ============= |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
76 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
77 Works with Prosody 0.9.x and later. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
78 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
79 Implementation |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
80 ============== |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
81 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
82 To implement your own external service that is compatible with this module, you need to expose a |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
83 simple API that allows the HTTP GET, HEAD and PUT methods on arbitrary URLs located on your service. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
84 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
85 For example, if http_upload_external_base_url is set to `https://example.com/upload/` then your service |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
86 might receive the following requests: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
87 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
88 Upload a new file: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
89 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
90 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
91 PUT https://example.com/upload/foo/bar.jpg?v=49e9309ff543ace93d25be90635ba8e9965c4f23fc885b2d86c947a5d59e55b2 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
92 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
93 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
94 Recipient checks the file size and other headers: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
95 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
96 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
97 HEAD https://example.com/upload/foo/bar.jpg |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
98 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
99 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
100 Recipient downloads the file: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
101 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
102 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
103 GET https://example.com/upload/foo/bar.jpg |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
104 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
105 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
106 The only tricky logic is in validation of the PUT request. Firstly, don't overwrite existing files (return 409 Conflict). |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
107 |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
108 Then you need to validate the auth token. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
109 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
110 ### Validating the auth token |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
111 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
112 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
113 | Version | Supports | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
114 |:--------|:--------------------------------------------------------------------------------------------------------| |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
115 | v | Validates only filename and size. Does not support file type restrictions by the XMPP server. | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
116 | v2 | Validates the filename, size and MIME type. This allows the server to implement MIME type restrictions. | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
117 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
118 It is probable that a future v3 will be specified that allows carrying information about the uploader identity, allowing |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
119 the implementation of per-user quotas and limits. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
120 |
3360
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
121 Implementations may implement one or more versions of the protocol simultaneously. The XMPP server generates the URLs and |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
122 ultimately selects which version will be used. |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
123 |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
124 XMPP servers MUST only generate URLs with **one** of the versions listed here. However in case multiple parameters are |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
125 present, upload services MUST **only** use the token from the highest parameter version that they support. |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
126 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
127 #### Version 1 (v) |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
128 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
129 The token will be in the URL query parameter 'v'. If it is absent, fail with 403 Forbidden. |
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
130 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
131 Calculate the expected auth token by reading the value of the Content-Length header of the PUT request. E.g. for a 1MB file |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
132 will have a Content-Length of '1048576'. Append this to the uploaded file name, separated by a space (0x20) character. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
133 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
134 For the above example, you would end up with the following string: "foo/bar.jpg 1048576" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
135 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
136 The auth token is a SHA256 HMAC of this string, using the configured secret as the key. E.g. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
137 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
138 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
139 calculated_auth_token = hmac_sha256("foo/bar.jpg 1048576", "secret string") |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
140 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
141 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
142 If this is not equal to the 'v' parameter provided in the upload URL, reject the upload with 403 Forbidden. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
143 |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
144 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
145 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
146 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
147 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
148 #### Version 2 (v2) |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
149 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
150 The token will be in the URL query parameter 'v2'. If it is absent, fail with 403 Forbidden. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
151 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
152 | Input | Example |Read from | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
153 |:--------------|:------------|:--------------------------------------------------------------------| |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
154 |`file_path` | foo/bar.jpg | The URL of the PUT request, with the service's base prefix removed. | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
155 |`content_size` | 1048576 | Content-Size header | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
156 |`content_type` | image/jpeg | Content-Type header | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
157 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
158 The parameters should be joined into a single string, separated by NUL bytes (`\0`): |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
159 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
160 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
161 signed_string = ( file_path + '\0' + content_size + '\0' + content_type ) |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
162 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
163 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
164 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
165 signed_string = "foo/bar.jpg\01048576\0image/jpeg" |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
166 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
167 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
168 The expected auth token is the SHA256 HMAC of this string, using the configured secret key as the key. E.g.: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
169 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
170 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
171 calculated_auth_token = hmac_sha256(signed_string, "secret string") |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
172 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
173 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
174 If this is not equal to the 'v2' parameter provided in the upload URL, reject the upload with 403 Forbidden. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
175 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
176 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
177 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
178 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
179 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
180 ### Security considerations |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
181 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
182 #### HTTPS |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
183 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
184 All uploads and downloads should only be over HTTPS. The security of the served content is protected only |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
185 by the uniqueness present in the URLs themselves, and not using HTTPS may leak the URLs and contents to third-parties. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
186 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
187 Implementations should consider including HSTS and HPKP headers, with consent of the administrator. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
188 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
189 #### MIME types |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
190 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
191 If the upload Content-Type header matches any of the following MIME types, it MUST be preserved and included in the Content-Type |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
192 of any GET requests made to download the file: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
193 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
194 - `image/*` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
195 - `video/*` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
196 - `audio/*` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
197 - `text/plain` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
198 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
199 It is recommended that other MIME types are preserved, but served with the addition of the following header: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
200 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
201 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
202 Content-Disposition: attachment |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
203 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
204 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
205 This prevents the browser interpreting scripts and other resources that may potentially be malicious. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
206 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
207 Some browsers may also benefit from explicitly telling them not to try guessing the type of a file: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
208 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
209 ``` |
3359
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
210 X-Content-Type-Options: nosniff |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
211 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
212 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
213 #### Security headers |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
214 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
215 The following headers should be included to provide additional sandboxing of resources, considering the uploaded |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
216 content is not understood or trusted by the upload service: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
217 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
218 ``` |
3359
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
219 Content-Security-Policy: default-src 'none' |
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
220 X-Content-Security-Policy: default-src 'none' |
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
221 X-WebKit-CSP: default-src 'none' |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
222 ``` |