Mercurial > prosody-modules
annotate mod_http_upload_external/README.markdown @ 5679:e274431bf4ce
mod_http_status: Add IP allowlisting capabilities
Based on mod_http_openmetrics
author | Kim Alvefur <zash@zash.se> |
---|---|
date | Wed, 25 Oct 2023 17:18:50 +0200 |
parents | c149edb37349 |
children |
rev | line source |
---|---|
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
1 --- |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
2 description: HTTP File Upload (external service) |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
3 labels: 'Stage-Alpha' |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
4 --- |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
5 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
6 Introduction |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
7 ============ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
8 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
9 This module implements [XEP-0363], which lets clients upload files |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
10 over HTTP to an external web server. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
11 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
12 This module generates URLs that are signed using a HMAC. Any web service that can authenticate |
4509
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
13 these URLs can be used. |
2823
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
14 |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
15 Implementations |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
16 --------------- |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
17 |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
18 * [PHP implementation](https://hg.prosody.im/prosody-modules/raw-file/tip/mod_http_upload_external/share.php) |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
19 * [Python3+Flask implementation](https://github.com/horazont/xmpp-http-upload) |
3168
73a610c3c7a9
mod_http_external: Link to prosody-filer (Go implementation)
Matthew Wild <mwild1@gmail.com>
parents:
2823
diff
changeset
|
20 * [Go implementation, Prosody Filer](https://github.com/ThomasLeister/prosody-filer) |
3189
57332ea0c1c7
mod_http_upload_external/README: Add Perl implementation by Holger to list
Kim Alvefur <zash@zash.se>
parents:
3168
diff
changeset
|
21 * [Perl implementation for nginx](https://github.com/weiss/ngx_http_upload) |
2823
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
22 |
4509
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
23 To implement your own service compatible with this module, check out the implementation notes below |
2823
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
24 (and if you publish your implementation - let us know!). |
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
25 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
26 Configuration |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
27 ============= |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
28 |
3959
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
29 The module can be added as a new Component definition: |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
30 |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
31 ``` {.lua} |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
32 Component "upload.example.org" "http_upload_external" |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
33 http_upload_external_base_url = "https://your.example.com/upload/service" |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
34 http_upload_external_secret = "your shared secret" |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
35 ``` |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
36 |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
37 It should **not** be added to modules_enabled. |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
38 |
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
39 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
40 External URL |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
41 ------------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
42 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
43 You need to provide the path to the external service. Ensure it ends with '/'. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
44 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
45 For example, to use the PHP implementation linked above, you might set it to: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
46 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
47 ``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
48 http_upload_external_base_url = "https://your.example.com/path/to/share.php/" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
49 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
50 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
51 Secret |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
52 ------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
53 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
54 Set a long and unpredictable string as your secret. This is so the upload service can verify that |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
55 the upload comes from mod_http_upload_external, and random strangers can't upload to your server. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
56 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
57 ``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
58 http_upload_external_secret = "this is a secret string!" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
59 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
60 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
61 You need to set exactly the same secret string in your external service. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
62 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
63 Limits |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
64 ------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
65 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
66 A maximum file size can be set by: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
67 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
68 ``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
69 http_upload_external_file_size_limit = 123 -- bytes |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
70 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
71 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
72 Default is 100MB (100\*1024\*1024). |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
73 |
4509
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
74 Access |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
75 ------ |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
76 |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
77 You may want to give upload access to additional entities such as components |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
78 by using the `http_upload_access` config option. |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
79 |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
80 ``` {.lua} |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
81 http_upload_access = {"gateway.example.com"}; |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
82 ``` |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
83 |
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
84 Compatibility |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
85 ============= |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
86 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
87 Works with Prosody 0.9.x and later. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
88 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
89 Implementation |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
90 ============== |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
91 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
92 To implement your own external service that is compatible with this module, you need to expose a |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
93 simple API that allows the HTTP GET, HEAD and PUT methods on arbitrary URLs located on your service. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
94 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
95 For example, if http_upload_external_base_url is set to `https://example.com/upload/` then your service |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
96 might receive the following requests: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
97 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
98 Upload a new file: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
99 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
100 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
101 PUT https://example.com/upload/foo/bar.jpg?v=49e9309ff543ace93d25be90635ba8e9965c4f23fc885b2d86c947a5d59e55b2 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
102 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
103 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
104 Recipient checks the file size and other headers: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
105 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
106 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
107 HEAD https://example.com/upload/foo/bar.jpg |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
108 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
109 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
110 Recipient downloads the file: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
111 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
112 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
113 GET https://example.com/upload/foo/bar.jpg |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
114 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
115 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
116 The only tricky logic is in validation of the PUT request. Firstly, don't overwrite existing files (return 409 Conflict). |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
117 |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
118 Then you need to validate the auth token. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
119 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
120 ### Validating the auth token |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
121 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
122 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
123 | Version | Supports | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
124 |:--------|:--------------------------------------------------------------------------------------------------------| |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
125 | v | Validates only filename and size. Does not support file type restrictions by the XMPP server. | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
126 | v2 | Validates the filename, size and MIME type. This allows the server to implement MIME type restrictions. | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
127 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
128 It is probable that a future v3 will be specified that allows carrying information about the uploader identity, allowing |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
129 the implementation of per-user quotas and limits. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
130 |
3360
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
131 Implementations may implement one or more versions of the protocol simultaneously. The XMPP server generates the URLs and |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
132 ultimately selects which version will be used. |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
133 |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
134 XMPP servers MUST only generate URLs with **one** of the versions listed here. However in case multiple parameters are |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
135 present, upload services MUST **only** use the token from the highest parameter version that they support. |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
136 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
137 #### Version 1 (v) |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
138 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
139 The token will be in the URL query parameter 'v'. If it is absent, fail with 403 Forbidden. |
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
140 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
141 Calculate the expected auth token by reading the value of the Content-Length header of the PUT request. E.g. for a 1MB file |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
142 will have a Content-Length of '1048576'. Append this to the uploaded file name, separated by a space (0x20) character. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
143 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
144 For the above example, you would end up with the following string: "foo/bar.jpg 1048576" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
145 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
146 The auth token is a SHA256 HMAC of this string, using the configured secret as the key. E.g. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
147 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
148 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
149 calculated_auth_token = hmac_sha256("foo/bar.jpg 1048576", "secret string") |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
150 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
151 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
152 If this is not equal to the 'v' parameter provided in the upload URL, reject the upload with 403 Forbidden. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
153 |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
154 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
155 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
156 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
157 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
158 #### Version 2 (v2) |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
159 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
160 The token will be in the URL query parameter 'v2'. If it is absent, fail with 403 Forbidden. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
161 |
4556
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
162 | Input | Example |Read from | |
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
163 |:----------------|:------------|:--------------------------------------------------------------------| |
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
164 |`file_path` | foo/bar.jpg | The URL of the PUT request, with the service's base prefix removed. | |
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
165 |`content_length` | 1048576 | Content-Length header | |
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
166 |`content_type` | image/jpeg | Content-Type header | |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
167 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
168 The parameters should be joined into a single string, separated by NUL bytes (`\0`): |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
169 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
170 ``` |
4556
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
171 signed_string = ( file_path + '\0' + content_length + '\0' + content_type ) |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
172 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
173 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
174 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
175 signed_string = "foo/bar.jpg\01048576\0image/jpeg" |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
176 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
177 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
178 The expected auth token is the SHA256 HMAC of this string, using the configured secret key as the key. E.g.: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
179 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
180 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
181 calculated_auth_token = hmac_sha256(signed_string, "secret string") |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
182 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
183 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
184 If this is not equal to the 'v2' parameter provided in the upload URL, reject the upload with 403 Forbidden. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
185 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
186 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
187 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
188 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
189 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
190 ### Security considerations |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
191 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
192 #### HTTPS |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
193 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
194 All uploads and downloads should only be over HTTPS. The security of the served content is protected only |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
195 by the uniqueness present in the URLs themselves, and not using HTTPS may leak the URLs and contents to third-parties. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
196 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
197 Implementations should consider including HSTS and HPKP headers, with consent of the administrator. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
198 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
199 #### MIME types |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
200 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
201 If the upload Content-Type header matches any of the following MIME types, it MUST be preserved and included in the Content-Type |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
202 of any GET requests made to download the file: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
203 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
204 - `image/*` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
205 - `video/*` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
206 - `audio/*` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
207 - `text/plain` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
208 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
209 It is recommended that other MIME types are preserved, but served with the addition of the following header: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
210 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
211 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
212 Content-Disposition: attachment |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
213 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
214 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
215 This prevents the browser interpreting scripts and other resources that may potentially be malicious. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
216 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
217 Some browsers may also benefit from explicitly telling them not to try guessing the type of a file: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
218 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
219 ``` |
3359
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
220 X-Content-Type-Options: nosniff |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
221 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
222 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
223 #### Security headers |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
224 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
225 The following headers should be included to provide additional sandboxing of resources, considering the uploaded |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
226 content is not understood or trusted by the upload service: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
227 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
228 ``` |
3359
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
229 Content-Security-Policy: default-src 'none' |
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
230 X-Content-Security-Policy: default-src 'none' |
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
231 X-WebKit-CSP: default-src 'none' |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
232 ``` |