Mercurial > prosody-modules

annotate mod_http_upload_external/README.markdown @ 2491:5fbca7de2088

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
mod_smacks: Send out more ack requests where needed Under some circumstances it was possible that more than "max_unacked_stanzas" where left in the outgoing stanza queue without forcing an ack. This could happen, when more stanzas entered the queue while the last ack request was still unanswered. Now the test "#queue > max_unacked_stanzas" is done upon receiving an ack as well as when sending out stanzas, which fixes this bug.
author tmolitor <thilo@eightysoft.de>
date Sun, 12 Feb 2017 19:27:50 +0100
parents c728b2f77c7c
children f14bea5da323
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2334
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
1 ---
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
2 description: HTTP File Upload (external service)
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
3 labels: 'Stage-Alpha'
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
4 ---
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
5
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
6 Introduction
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
7 ============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
8
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
9 This module implements [XEP-0363], which lets clients upload files
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
10 over HTTP to an external web server.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
11
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
12 This module generates URLs that are signed using a HMAC. Any web service that can authenticate
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
13 these URLs can be used. There is a PHP implementation available
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
14 [here](https://hg.prosody.im/prosody-modules/raw-file/tip/mod_http_upload_external/share.php). To
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
15 implement your own service compatible with this module, check out the implementation notes below (and if you
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
16 publish your implementation - let us know!).
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
17
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
18 Configuration
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
19 =============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
20
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
21 Add `"http_upload_external"` to modules_enabled in your global section, or under the host(s) you wish
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
22 to use it on.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
23
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
24 External URL
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
25 ------------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
26
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
27 You need to provide the path to the external service. Ensure it ends with '/'.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
28
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
29 For example, to use the PHP implementation linked above, you might set it to:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
30
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
31 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
32 http_upload_external_base_url = "https://your.example.com/path/to/share.php/"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
33 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
34
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
35 Secret
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
36 ------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
37
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
38 Set a long and unpredictable string as your secret. This is so the upload service can verify that
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
39 the upload comes from mod_http_upload_external, and random strangers can't upload to your server.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
40
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
41 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
42 http_upload_external_secret = "this is a secret string!"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
43 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
44
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
45 You need to set exactly the same secret string in your external service.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
46
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
47 Limits
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
48 ------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
49
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
50 A maximum file size can be set by:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
51
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
52 ``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
53 http_upload_external_file_size_limit = 123 -- bytes
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
54 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
55
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
56 Default is 100MB (100\*1024\*1024).
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
57
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
58 Compatibility
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
59 =============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
60
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
61 Works with Prosody 0.9.x and later.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
62
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
63 Implementation
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
64 ==============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
65
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
66 To implement your own external service that is compatible with this module, you need to expose a
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
67 simple API that allows the HTTP GET, HEAD and PUT methods on arbitrary URLs located on your service.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
68
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
69 For example, if http_upload_external_base_url is set to `https://example.com/upload/` then your service
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
70 might receive the following requests:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
71
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
72 Upload a new file:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
73
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
74 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
75 PUT https://example.com/upload/foo/bar.jpg?v=49e9309ff543ace93d25be90635ba8e9965c4f23fc885b2d86c947a5d59e55b2
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
76 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
77
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
78 Recipient checks the file size and other headers:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
79
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
80 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
81 HEAD https://example.com/upload/foo/bar.jpg
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
82 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
83
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
84 Recipient downloads the file:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
85
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
86 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
87 GET https://example.com/upload/foo/bar.jpg
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
88 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
89
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
90 The only tricky logic is in validation of the PUT request. Firstly, don't overwrite existing files (return 409 Conflict).
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
91
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
92 Then you need to validate the auth token. This will be in the URL query parameter 'v'. If it is absent, fail with 403 Forbidden.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
93
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
94 Calculate the expected auth token by reading the value of the Content-Length header of the PUT request. E.g. for a 1MB file
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
95 will have a Content-Length of '1048576'. Append this to the uploaded file name, separated by a space (0x20) character.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
96
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
97 For the above example, you would end up with the following string: "foo/bar.jpg 1048576"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
98
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
99 The auth token is a SHA256 HMAC of this string, using the configured secret as the key. E.g.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
100
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
101 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
102 calculated_auth_token = hmac_sha256("foo/bar.jpg 1048576", "secret string")
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
103 ```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
104
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
105 If this is not equal to the 'v' parameter provided in the upload URL, reject the upload with 403 Forbidden.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
106
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
107 Note: your language/environment may provide a function for doing a constant-time comparison of these, to guard against
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
108 timing attacks that may be used to discover the secret key.