Mercurial > prosody-modules
annotate mod_storage_xmlarchive/README.markdown @ 3135:e166ccc7a779
mod_measure_client_identities: Collect statistics about client identities.
author | Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> |
---|---|
date | Fri, 22 Jun 2018 01:06:18 +0200 |
parents | 7713cd4fff8f |
children | a6722a35f35e |
rev | line source |
---|---|
1803
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
1 --- |
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
2 labels: |
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
3 - 'Stage-Beta' |
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
4 - 'Type-Storage' |
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
5 - ArchiveStorage |
2359
33e85070dc60
mod_storage_xmlarchive: Expand the summary a little bit
Kim Alvefur <zash@zash.se>
parents:
2291
diff
changeset
|
6 summary: XML file based archive storage |
2817
f052b62c653c
mod_storage_xmlarchive/README: Minor tweak
Kim Alvefur <zash@zash.se>
parents:
2815
diff
changeset
|
7 --- |
1782 | 8 |
1803
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
9 Introduction |
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
10 ============ |
1782 | 11 |
1803
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
12 This module implements stanza archives using files, similar to the |
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
13 default "internal" storage. |
1782 | 14 |
1940
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
15 Configuration |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
16 ============= |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
17 |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
18 To use this with [mod\_mam] add this to your config: |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
19 |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
20 ``` lua |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
21 storage = { |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
22 archive2 = "xmlarchive" |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
23 } |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
24 ``` |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
25 |
1969
e63dba236a2a
mod_storage_xmlarchive: Use datamanager.append_raw (had that code duplicated here)
Kim Alvefur <zash@zash.se>
parents:
1940
diff
changeset
|
26 To use it with [mod\_mam\_muc] or [mod\_http\_muc\_log]: |
1940
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
27 |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
28 ``` lua |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
29 storage = { |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
30 muc_log = "xmlarchive" |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
31 } |
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
32 ``` |
1782 | 33 |
1969
e63dba236a2a
mod_storage_xmlarchive: Use datamanager.append_raw (had that code duplicated here)
Kim Alvefur <zash@zash.se>
parents:
1940
diff
changeset
|
34 Refer to [Prosodys data storage documentation][doc:storage] for more |
e63dba236a2a
mod_storage_xmlarchive: Use datamanager.append_raw (had that code duplicated here)
Kim Alvefur <zash@zash.se>
parents:
1940
diff
changeset
|
35 information. |
1803
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
36 |
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
37 Note that this module does not implement the "keyval" storage method and |
1940
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
38 can't be used by anything other than archives. |
1782 | 39 |
1803
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
40 Compatibility |
4d73a1a6ba68
Convert all wiki pages to Markdown
Kim Alvefur <zash@zash.se>
parents:
1782
diff
changeset
|
41 ============= |
1782 | 42 |
2291
1eeda34d668b
mod_storage_xmlarchive/README: Should work in 0.9 now
Kim Alvefur <zash@zash.se>
parents:
2048
diff
changeset
|
43 ------ --------------- |
1940
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
44 0.10 Works |
2291
1eeda34d668b
mod_storage_xmlarchive/README: Should work in 0.9 now
Kim Alvefur <zash@zash.se>
parents:
2048
diff
changeset
|
45 0.9 Should work |
1940
453eee191e18
mod_storage_xmlarchive/README: Add example configuration
Kim Alvefur <zash@zash.se>
parents:
1803
diff
changeset
|
46 0.8 Does not work |
2291
1eeda34d668b
mod_storage_xmlarchive/README: Should work in 0.9 now
Kim Alvefur <zash@zash.se>
parents:
2048
diff
changeset
|
47 ------ --------------- |
2815
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
48 |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
49 Conversion to or from internal storage |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
50 -------------------------------------- |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
51 |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
52 This module stores data in a way that overlaps with the more recent |
2817
f052b62c653c
mod_storage_xmlarchive/README: Minor tweak
Kim Alvefur <zash@zash.se>
parents:
2815
diff
changeset
|
53 archive support in `mod_storage_internal`, meaning e.g. [mod_migrate] |
2815
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
54 will not be able to cleanly convert to or from the `xmlarchive` format. |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
55 |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
56 To mitigate this, an migration command has been added to |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
57 `mod_storage_xmlarchive`: |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
58 |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
59 ``` bash |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
60 prosodyctl mod_storage_xmlarchive convert $DIR internal $STORE $JID |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
61 ``` |
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
62 |
2818 | 63 Where `$DIR` is `to` or `from`, `$STORE` is e.g. `archive` or `archive2` |
2815
d48d4d9ccae7
mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format
Kim Alvefur <zash@zash.se>
parents:
2359
diff
changeset
|
64 for MAM and `muc_log` for MUC logs. Finally, `$JID` is the JID of the |
2855
7713cd4fff8f
mod_storage_xmlarchive/README: Fix typo
Kim Alvefur <zash@zash.se>
parents:
2854
diff
changeset
|
65 user or MUC room to be migrated, which can be repeated. |
2854
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
66 |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
67 Data structure |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
68 ============== |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
69 |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
70 Data is split in three kinds of files and messages are grouped by day. |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
71 Prosodys `util.datamanager` is used, so all special characters in these |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
72 filenames are escaped and reside under `hostname/store` in Prosodys Data |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
73 directory, commonly `/var/lib/prosody`. |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
74 |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
75 `username.list` |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
76 : A list of dates in `YYYY-MM-DD` format. |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
77 |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
78 `username@YYYY-MM-DD.list` |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
79 : Index containing metadata for messages stored on that day. |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
80 |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
81 `username@YYYY-MM-DD.xml` |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
82 : Messages in textual XML format, separated by newlines. |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
83 |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
84 This makes it fairly simple and fast to find messages by timestamp. |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
85 Queries that are not time based, but limited to a specific contact may |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
86 be expensive as potentially the entire archive will be read. |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
87 |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
88 Each archive ID is of the form `YYYY-MM-DD-random`, making lookups by |
687b19cad4f5
mod_storage_xmlarchive/README: Add description of how data is stored
Kim Alvefur <zash@zash.se>
parents:
2818
diff
changeset
|
89 archive id just as simple as time based queries. |