Mercurial > prosody-modules

annotate mod_http_upload_external/README.markdown @ 5909:070b0db6c4a0

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
mod_http_upload_external: Add link to Rust implementation (Thanks Luna)
author Kim Alvefur <zash@zash.se>
date Sat, 18 May 2024 14:16:49 +0200
parents 694b62d8a82f
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2334
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
1 ---
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
2 description: HTTP File Upload (external service)
5899
694b62d8a82f various/README: Fix 'labels' metadata, should be a list
Kim Alvefur <zash@zash.se>
parents: 5839
diff changeset
3 labels:
694b62d8a82f various/README: Fix 'labels' metadata, should be a list
Kim Alvefur <zash@zash.se>
parents: 5839
diff changeset
4 - Stage-Alpha
2334
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
5 ---
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
6
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
7 Introduction
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
8 ============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
9
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
10 This module implements [XEP-0363], which lets clients upload files
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
11 over HTTP to an external web server.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
12
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
13 This module generates URLs that are signed using a HMAC. Any web service that can authenticate
4509
16995e7624f0 mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents: 3959
diff changeset
14 these URLs can be used.
2823
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
15
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
16 Implementations
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
17 ---------------
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
18
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
19 * [PHP implementation](https://hg.prosody.im/prosody-modules/raw-file/tip/mod_http_upload_external/share.php)
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
20 * [Python3+Flask implementation](https://github.com/horazont/xmpp-http-upload)
3168
73a610c3c7a9 mod_http_external: Link to prosody-filer (Go implementation)
Matthew Wild <mwild1@gmail.com>
parents: 2823
diff changeset
21 * [Go implementation, Prosody Filer](https://github.com/ThomasLeister/prosody-filer)
3189
57332ea0c1c7 mod_http_upload_external/README: Add Perl implementation by Holger to list
Kim Alvefur <zash@zash.se>
parents: 3168
diff changeset
22 * [Perl implementation for nginx](https://github.com/weiss/ngx_http_upload)
5909
070b0db6c4a0 mod_http_upload_external: Add link to Rust implementation (Thanks Luna)
Kim Alvefur <zash@zash.se>
parents: 5899
diff changeset
23 * [Rust implementation](https://gitlab.com/nyovaya/xmpp-http-upload)
2823
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
24
4509
16995e7624f0 mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents: 3959
diff changeset
25 To implement your own service compatible with this module, check out the implementation notes below
2823
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2334
diff changeset
26 (and if you publish your implementation - let us know!).
2334
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
27
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
28 Configuration
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
29 =============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
30
3959
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents: 3360
diff changeset
31 The module can be added as a new Component definition:
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents: 3360
diff changeset
32
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents: 3360
diff changeset
33 ``` {.lua}
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents: 3360
diff changeset
34 Component "upload.example.org" "http_upload_external"
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents: 3360
diff changeset
35 http_upload_external_base_url = "https://your.example.com/upload/service"
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents: 3360
diff changeset
36 http_upload_external_secret = "your shared secret"
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents: 3360
diff changeset
37 ```
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents: 3360
diff changeset
38
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents: 3360
diff changeset
39 It should **not** be added to modules_enabled.
5741e6511f3d mod_http_upload_external: Discourage loading via modules_enabled
Kim Alvefur <zash@zash.se>
parents: 3360
diff changeset
40
2334
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
41
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
42 External URL
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
43 ------------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
44
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
45 You need to provide the path to the external service. Ensure it ends with '/'.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
46
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
47 For example, to use the PHP implementation linked above, you might set it to:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
48
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
49 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
50 http_upload_external_base_url = "https://your.example.com/path/to/share.php/"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
51 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
52
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
53 Secret
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
54 ------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
55
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
56 Set a long and unpredictable string as your secret. This is so the upload service can verify that
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
57 the upload comes from mod_http_upload_external, and random strangers can't upload to your server.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
58
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
59 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
60 http_upload_external_secret = "this is a secret string!"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
61 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
62
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
63 You need to set exactly the same secret string in your external service.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
64
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
65 Limits
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
66 ------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
67
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
68 A maximum file size can be set by:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
69
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
70 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
71 http_upload_external_file_size_limit = 123 -- bytes
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
72 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
73
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
74 Default is 100MB (100\*1024\*1024).
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
75
4509
16995e7624f0 mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents: 3959
diff changeset
76 Access
16995e7624f0 mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents: 3959
diff changeset
77 ------
16995e7624f0 mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents: 3959
diff changeset
78
16995e7624f0 mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents: 3959
diff changeset
79 You may want to give upload access to additional entities such as components
5839
fba64b043c52 mod_http_upload_external: Fix typo in access documentation.
aidan@jmad.org
parents: 4556
diff changeset
80 by using the `http_upload_external_access` config option.
4509
16995e7624f0 mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents: 3959
diff changeset
81
16995e7624f0 mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents: 3959
diff changeset
82 ``` {.lua}
5839
fba64b043c52 mod_http_upload_external: Fix typo in access documentation.
aidan@jmad.org
parents: 4556
diff changeset
83 http_upload_external_access = {"gateway.example.com"};
4509
16995e7624f0 mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents: 3959
diff changeset
84 ```
16995e7624f0 mod_http_upload_external: add access control option
Nicolas Cedilnik <nicoco@nicoco.fr>
parents: 3959
diff changeset
85
2334
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
86 Compatibility
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
87 =============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
88
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
89 Works with Prosody 0.9.x and later.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
90
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
91 Implementation
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
92 ==============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
93
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
94 To implement your own external service that is compatible with this module, you need to expose a
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
95 simple API that allows the HTTP GET, HEAD and PUT methods on arbitrary URLs located on your service.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
96
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
97 For example, if http_upload_external_base_url is set to `https://example.com/upload/` then your service
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
98 might receive the following requests:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
99
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
100 Upload a new file:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
101
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
102 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
103 PUT https://example.com/upload/foo/bar.jpg?v=49e9309ff543ace93d25be90635ba8e9965c4f23fc885b2d86c947a5d59e55b2
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
104 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
105
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
106 Recipient checks the file size and other headers:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
107
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
108 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
109 HEAD https://example.com/upload/foo/bar.jpg
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
110 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
111
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
112 Recipient downloads the file:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
113
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
114 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
115 GET https://example.com/upload/foo/bar.jpg
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
116 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
117
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
118 The only tricky logic is in validation of the PUT request. Firstly, don't overwrite existing files (return 409 Conflict).
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
119
3358
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
120 Then you need to validate the auth token.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
121
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
122 ### Validating the auth token
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
123
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
124
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
125 | Version | Supports |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
126 |:--------|:--------------------------------------------------------------------------------------------------------|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
127 | v | Validates only filename and size. Does not support file type restrictions by the XMPP server. |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
128 | v2 | Validates the filename, size and MIME type. This allows the server to implement MIME type restrictions. |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
129
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
130 It is probable that a future v3 will be specified that allows carrying information about the uploader identity, allowing
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
131 the implementation of per-user quotas and limits.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
132
3360
0149954cee37 mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents: 3359
diff changeset
133 Implementations may implement one or more versions of the protocol simultaneously. The XMPP server generates the URLs and
0149954cee37 mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents: 3359
diff changeset
134 ultimately selects which version will be used.
0149954cee37 mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents: 3359
diff changeset
135
0149954cee37 mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents: 3359
diff changeset
136 XMPP servers MUST only generate URLs with **one** of the versions listed here. However in case multiple parameters are
0149954cee37 mod_http_upload_external: Add note about correct behaviour in the presence of multiple versions
Matthew Wild <mwild1@gmail.com>
parents: 3359
diff changeset
137 present, upload services MUST **only** use the token from the highest parameter version that they support.
3358
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
138
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
139 #### Version 1 (v)
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
140
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
141 The token will be in the URL query parameter 'v'. If it is absent, fail with 403 Forbidden.
2334
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
142
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
143 Calculate the expected auth token by reading the value of the Content-Length header of the PUT request. E.g. for a 1MB file
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
144 will have a Content-Length of '1048576'. Append this to the uploaded file name, separated by a space (0x20) character.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
145
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
146 For the above example, you would end up with the following string: "foo/bar.jpg 1048576"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
147
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
148 The auth token is a SHA256 HMAC of this string, using the configured secret as the key. E.g.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
149
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
150 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
151 calculated_auth_token = hmac_sha256("foo/bar.jpg 1048576", "secret string")
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
152 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
153
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
154 If this is not equal to the 'v' parameter provided in the upload URL, reject the upload with 403 Forbidden.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
155
3358
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
156 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
157 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
158 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
159
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
160 #### Version 2 (v2)
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
161
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
162 The token will be in the URL query parameter 'v2'. If it is absent, fail with 403 Forbidden.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
163
4556
c149edb37349 Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents: 4509
diff changeset
164 | Input | Example |Read from |
c149edb37349 Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents: 4509
diff changeset
165 |:----------------|:------------|:--------------------------------------------------------------------|
c149edb37349 Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents: 4509
diff changeset
166 |`file_path` | foo/bar.jpg | The URL of the PUT request, with the service's base prefix removed. |
c149edb37349 Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents: 4509
diff changeset
167 |`content_length` | 1048576 | Content-Length header |
c149edb37349 Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents: 4509
diff changeset
168 |`content_type` | image/jpeg | Content-Type header |
3358
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
169
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
170 The parameters should be joined into a single string, separated by NUL bytes (`\0`):
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
171
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
172 ```
4556
c149edb37349 Fix mentions of 'Content-Size' (should be Content-Length) (thanks Roobre)
Matthew Wild <mwild1@gmail.com>
parents: 4509
diff changeset
173 signed_string = ( file_path + '\0' + content_length + '\0' + content_type )
3358
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
174 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
175
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
176 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
177 signed_string = "foo/bar.jpg\01048576\0image/jpeg"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
178 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
179
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
180 The expected auth token is the SHA256 HMAC of this string, using the configured secret key as the key. E.g.:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
181
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
182 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
183 calculated_auth_token = hmac_sha256(signed_string, "secret string")
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
184 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
185
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
186 If this is not equal to the 'v2' parameter provided in the upload URL, reject the upload with 403 Forbidden.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
187
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
188 **Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
189 comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
190 as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
191
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
192 ### Security considerations
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
193
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
194 #### HTTPS
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
195
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
196 All uploads and downloads should only be over HTTPS. The security of the served content is protected only
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
197 by the uniqueness present in the URLs themselves, and not using HTTPS may leak the URLs and contents to third-parties.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
198
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
199 Implementations should consider including HSTS and HPKP headers, with consent of the administrator.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
200
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
201 #### MIME types
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
202
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
203 If the upload Content-Type header matches any of the following MIME types, it MUST be preserved and included in the Content-Type
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
204 of any GET requests made to download the file:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
205
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
206 - `image/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
207 - `video/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
208 - `audio/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
209 - `text/plain`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
210
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
211 It is recommended that other MIME types are preserved, but served with the addition of the following header:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
212
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
213 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
214 Content-Disposition: attachment
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
215 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
216
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
217 This prevents the browser interpreting scripts and other resources that may potentially be malicious.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
218
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
219 Some browsers may also benefit from explicitly telling them not to try guessing the type of a file:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
220
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
221 ```
3359
3d01ab6b1186 mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents: 3358
diff changeset
222 X-Content-Type-Options: nosniff
3358
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
223 ```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
224
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
225 #### Security headers
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
226
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
227 The following headers should be included to provide additional sandboxing of resources, considering the uploaded
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
228 content is not understood or trusted by the upload service:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
229
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
230 ```
3359
3d01ab6b1186 mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents: 3358
diff changeset
231 Content-Security-Policy: default-src 'none'
3d01ab6b1186 mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents: 3358
diff changeset
232 X-Content-Security-Policy: default-src 'none'
3d01ab6b1186 mod_http_upload_external: Fix typo/copy-paste issues in headers (thanks jonas<U+2019>)
Matthew Wild <mwild1@gmail.com>
parents: 3358
diff changeset
233 X-WebKit-CSP: default-src 'none'
3358
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3189
diff changeset
234 ```