Mercurial > prosody-modules
annotate mod_http_upload_external/README.markdown @ 5416:2393dbae51ed
mod_http_oauth2: Add option for specifying TTL of registered clients
Meant to simplify configuration, since TTL vs ignoring expiration is
expected to be the main thing one would want to configure.
Unsure what the implications of having unlimited lifetime of clients
are, given no way to revoke them currently, short of rotating the
signing secret.
On one hand, it would be annoying to have the client expire.
On the other hand, it is trivial to re-register it.
author | Kim Alvefur <zash@zash.se> |
---|---|
date | Thu, 04 May 2023 18:41:33 +0200 |
parents | c149edb37349 |
children | fba64b043c52 |
rev | line source |
---|---|
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
1 --- |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
2 description: HTTP File Upload (external service) |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
3 labels: 'Stage-Alpha' |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
4 --- |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
5 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
6 Introduction |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
7 ============ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
8 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
9 This module implements [XEP-0363], which lets clients upload files |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
10 over HTTP to an external web server. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
11 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
12 This module generates URLs that are signed using a HMAC. Any web service that can authenticate |
4509
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
13 these URLs can be used. |
2823
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
14 |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
15 Implementations |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
16 --------------- |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
17 |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
18 * [PHP implementation](https://hg.prosody.im/prosody-modules/raw-file/tip/mod_http_upload_external/share.php) |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
19 * [Python3+Flask implementation](https://github.com/horazont/xmpp-http-upload) |
3168
73a610c3c7a9
mod_http_external: Link to prosody-filer (Go implementation)
Matthew Wild <mwild1@gmail.com>
parents:
2823
diff
changeset
|
20 * [Go implementation, Prosody Filer](https://github.com/ThomasLeister/prosody-filer) |
3189
57332ea0c1c7
mod_http_upload_external/README: Add Perl implementation by Holger to list
Kim Alvefur <zash@zash.se>
parents:
3168
diff
changeset
|
21 * [Perl implementation for nginx](https://github.com/weiss/ngx_http_upload) |
2823
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
22 |
4509
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
23 To implement your own service compatible with this module, check out the implementation notes below |
2823
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2334
diff
changeset
|
24 (and if you publish your implementation - let us know!). |
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
25 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
26 Configuration |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
27 ============= |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
28 |
3959
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
29 The module can be added as a new Component definition: |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
30 |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
31 ``` {.lua} |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
32 Component "upload.example.org" "http_upload_external" |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
33 http_upload_external_base_url = "https://your.example.com/upload/service" |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
34 http_upload_external_secret = "your shared secret" |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
35 ``` |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
36 |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
37 It should **not** be added to modules_enabled. |
5741e6511f3d
mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents:
3360
diff
changeset
|
38 |
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
39 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
40 External URL |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
41 ------------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
42 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
43 You need to provide the path to the external service. Ensure it ends with '/'. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
44 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
45 For example, to use the PHP implementation linked above, you might set it to: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
46 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
47 ``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
48 http_upload_external_base_url = "https://your.example.com/path/to/share.php/" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
49 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
50 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
51 Secret |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
52 ------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
53 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
54 Set a long and unpredictable string as your secret. This is so the upload service can verify that |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
55 the upload comes from mod_http_upload_external, and random strangers can't upload to your server. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
56 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
57 ``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
58 http_upload_external_secret = "this is a secret string!" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
59 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
60 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
61 You need to set exactly the same secret string in your external service. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
62 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
63 Limits |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
64 ------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
65 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
66 A maximum file size can be set by: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
67 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
68 ``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
69 http_upload_external_file_size_limit = 123 -- bytes |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
70 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
71 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
72 Default is 100MB (100\*1024\*1024). |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
73 |
4509
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
74 Access |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
75 ------ |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
76 |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
77 You may want to give upload access to additional entities such as components |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
78 by using the `http_upload_access` config option. |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
79 |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
80 ``` {.lua} |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
81 http_upload_access = {"gateway.example.com"}; |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
82 ``` |
16995e7624f0
mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents:
3959
diff
changeset
|
83 |
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
84 Compatibility |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
85 ============= |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
86 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
87 Works with Prosody 0.9.x and later. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
88 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
89 Implementation |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
90 ============== |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
91 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
92 To implement your own external service that is compatible with this module, you need to expose a |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
93 simple API that allows the HTTP GET, HEAD and PUT methods on arbitrary URLs located on your service. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
94 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
95 For example, if http_upload_external_base_url is set to `https://example.com/upload/` then your service |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
96 might receive the following requests: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
97 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
98 Upload a new file: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
99 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
100 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
101 PUT https://example.com/upload/foo/bar.jpg?v=49e9309ff543ace93d25be90635ba8e9965c4f23fc885b2d86c947a5d59e55b2 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
102 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
103 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
104 Recipient checks the file size and other headers: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
105 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
106 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
107 HEAD https://example.com/upload/foo/bar.jpg |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
108 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
109 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
110 Recipient downloads the file: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
111 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
112 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
113 GET https://example.com/upload/foo/bar.jpg |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
114 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
115 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
116 The only tricky logic is in validation of the PUT request. Firstly, don't overwrite existing files (return 409 Conflict). |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
117 |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
118 Then you need to validate the auth token. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
119 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
120 ### Validating the auth token |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
121 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
122 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
123 | Version | Supports | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
124 |:--------|:--------------------------------------------------------------------------------------------------------| |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
125 | v | Validates only filename and size. Does not support file type restrictions by the XMPP server. | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
126 | v2 | Validates the filename, size and MIME type. This allows the server to implement MIME type restrictions. | |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
127 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
128 It is probable that a future v3 will be specified that allows carrying information about the uploader identity, allowing |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
129 the implementation of per-user quotas and limits. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
130 |
3360
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
131 Implementations may implement one or more versions of the protocol simultaneously. The XMPP server generates the URLs and |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
132 ultimately selects which version will be used. |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
133 |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
134 XMPP servers MUST only generate URLs with **one** of the versions listed here. However in case multiple parameters are |
0149954cee37
mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents:
3359
diff
changeset
|
135 present, upload services MUST **only** use the token from the highest parameter version that they support. |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
136 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
137 #### Version 1 (v) |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
138 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
139 The token will be in the URL query parameter 'v'. If it is absent, fail with 403 Forbidden. |
2334
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
140 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
141 Calculate the expected auth token by reading the value of the Content-Length header of the PUT request. E.g. for a 1MB file |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
142 will have a Content-Length of '1048576'. Append this to the uploaded file name, separated by a space (0x20) character. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
143 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
144 For the above example, you would end up with the following string: "foo/bar.jpg 1048576" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
145 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
146 The auth token is a SHA256 HMAC of this string, using the configured secret as the key. E.g. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
147 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
148 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
149 calculated_auth_token = hmac_sha256("foo/bar.jpg 1048576", "secret string") |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
150 ``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
151 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
152 If this is not equal to the 'v' parameter provided in the upload URL, reject the upload with 403 Forbidden. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
153 |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
154 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
155 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
156 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
157 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
158 #### Version 2 (v2) |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
159 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
160 The token will be in the URL query parameter 'v2'. If it is absent, fail with 403 Forbidden. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
161 |
4556
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
162 | Input | Example |Read from | |
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
163 |:----------------|:------------|:--------------------------------------------------------------------| |
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
164 |`file_path` | foo/bar.jpg | The URL of the PUT request, with the service's base prefix removed. | |
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
165 |`content_length` | 1048576 | Content-Length header | |
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
166 |`content_type` | image/jpeg | Content-Type header | |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
167 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
168 The parameters should be joined into a single string, separated by NUL bytes (`\0`): |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
169 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
170 ``` |
4556
c149edb37349
Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents:
4509
diff
changeset
|
171 signed_string = ( file_path + '\0' + content_length + '\0' + content_type ) |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
172 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
173 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
174 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
175 signed_string = "foo/bar.jpg\01048576\0image/jpeg" |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
176 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
177 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
178 The expected auth token is the SHA256 HMAC of this string, using the configured secret key as the key. E.g.: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
179 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
180 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
181 calculated_auth_token = hmac_sha256(signed_string, "secret string") |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
182 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
183 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
184 If this is not equal to the 'v2' parameter provided in the upload URL, reject the upload with 403 Forbidden. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
185 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
186 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
187 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
188 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
189 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
190 ### Security considerations |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
191 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
192 #### HTTPS |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
193 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
194 All uploads and downloads should only be over HTTPS. The security of the served content is protected only |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
195 by the uniqueness present in the URLs themselves, and not using HTTPS may leak the URLs and contents to third-parties. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
196 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
197 Implementations should consider including HSTS and HPKP headers, with consent of the administrator. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
198 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
199 #### MIME types |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
200 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
201 If the upload Content-Type header matches any of the following MIME types, it MUST be preserved and included in the Content-Type |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
202 of any GET requests made to download the file: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
203 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
204 - `image/*` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
205 - `video/*` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
206 - `audio/*` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
207 - `text/plain` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
208 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
209 It is recommended that other MIME types are preserved, but served with the addition of the following header: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
210 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
211 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
212 Content-Disposition: attachment |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
213 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
214 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
215 This prevents the browser interpreting scripts and other resources that may potentially be malicious. |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
216 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
217 Some browsers may also benefit from explicitly telling them not to try guessing the type of a file: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
218 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
219 ``` |
3359
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
220 X-Content-Type-Options: nosniff |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
221 ``` |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
222 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
223 #### Security headers |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
224 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
225 The following headers should be included to provide additional sandboxing of resources, considering the uploaded |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
226 content is not understood or trusted by the upload service: |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
227 |
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
228 ``` |
3359
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
229 Content-Security-Policy: default-src 'none' |
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
230 X-Content-Security-Policy: default-src 'none' |
3d01ab6b1186
mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents:
3358
diff
changeset
|
231 X-WebKit-CSP: default-src 'none' |
3358
e49660ba3161
mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents:
3189
diff
changeset
|
232 ``` |