prosody-modules: mod_storage_xmlarchive/README.markdown annotate

annotate mod_storage_xmlarchive/README.markdown @ 3673:11ebf1da416b

mod_muc_webchat_url: Don't save templated value This prevents changing of other settings without touching the webchat URL from saving the template value. Otherwise the URL will still be advertised when configuring members-only or hidden.

author	Kim Alvefur <zash@zash.se>
date	Tue, 17 Sep 2019 17:22:35 +0200
parents	f25b0623b276
children	dddcf094af5c

rev	line source
1803 4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	1 ---
4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	2 labels:
4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	3 - 'Stage-Beta'
4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	4 - 'Type-Storage'
4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	5 - ArchiveStorage
2359 33e85070dc60 mod_storage_xmlarchive: Expand the summary a little bit Kim Alvefur <zash@zash.se> parents: 2291 diff changeset	6 summary: XML file based archive storage
2817 f052b62c653c mod_storage_xmlarchive/README: Minor tweak Kim Alvefur <zash@zash.se> parents: 2815 diff changeset	7 ---
1782 29f3d6b7ad16 Import wiki pages Kim Alvefur <zash@zash.se> parents: diff changeset	8
1803 4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	9 Introduction
4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	10 ============
1782 29f3d6b7ad16 Import wiki pages Kim Alvefur <zash@zash.se> parents: diff changeset	11
1803 4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	12 This module implements stanza archives using files, similar to the
3434 a6722a35f35e mod_storage_xmlarchive: Expand description in README Kim Alvefur <zash@zash.se> parents: 2855 diff changeset	13 default "internal" storage. Unlike "internal", it saves messages in two
a6722a35f35e mod_storage_xmlarchive: Expand description in README Kim Alvefur <zash@zash.se> parents: 2855 diff changeset	14 files per day (and per user), one containing metadata and one containing
a6722a35f35e mod_storage_xmlarchive: Expand description in README Kim Alvefur <zash@zash.se> parents: 2855 diff changeset	15 the actual messages in XML format (hence the name).
a6722a35f35e mod_storage_xmlarchive: Expand description in README Kim Alvefur <zash@zash.se> parents: 2855 diff changeset	16
a6722a35f35e mod_storage_xmlarchive: Expand description in README Kim Alvefur <zash@zash.se> parents: 2855 diff changeset	17 Splitting data per day improves performance for larger archives as it
a6722a35f35e mod_storage_xmlarchive: Expand description in README Kim Alvefur <zash@zash.se> parents: 2855 diff changeset	18 does not have to look through data from other days.
1782 29f3d6b7ad16 Import wiki pages Kim Alvefur <zash@zash.se> parents: diff changeset	19
1940 453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	20 Configuration
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	21 =============
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	22
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	23 To use this with [mod\_mam] add this to your config:
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	24
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	25 ``` lua
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	26 storage = {
3451 7b97abf26612 mod_storage_xmlarchive/README: Change example to match Prosodys mod_mam Kim Alvefur <zash@zash.se> parents: 3434 diff changeset	27 archive = "xmlarchive"
1940 453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	28 }
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	29 ```
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	30
1969 e63dba236a2a mod_storage_xmlarchive: Use datamanager.append_raw (had that code duplicated here) Kim Alvefur <zash@zash.se> parents: 1940 diff changeset	31 To use it with [mod\_mam\_muc] or [mod\_http\_muc\_log]:
1940 453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	32
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	33 ``` lua
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	34 storage = {
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	35 muc_log = "xmlarchive"
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	36 }
453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	37 ```
1782 29f3d6b7ad16 Import wiki pages Kim Alvefur <zash@zash.se> parents: diff changeset	38
1969 e63dba236a2a mod_storage_xmlarchive: Use datamanager.append_raw (had that code duplicated here) Kim Alvefur <zash@zash.se> parents: 1940 diff changeset	39 Refer to [Prosodys data storage documentation][doc:storage] for more
e63dba236a2a mod_storage_xmlarchive: Use datamanager.append_raw (had that code duplicated here) Kim Alvefur <zash@zash.se> parents: 1940 diff changeset	40 information.
1803 4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	41
4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	42 Note that this module does not implement the "keyval" storage method and
1940 453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	43 can't be used by anything other than archives.
1782 29f3d6b7ad16 Import wiki pages Kim Alvefur <zash@zash.se> parents: diff changeset	44
1803 4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	45 Compatibility
4d73a1a6ba68 Convert all wiki pages to Markdown Kim Alvefur <zash@zash.se> parents: 1782 diff changeset	46 =============
1782 29f3d6b7ad16 Import wiki pages Kim Alvefur <zash@zash.se> parents: diff changeset	47
2291 1eeda34d668b mod_storage_xmlarchive/README: Should work in 0.9 now Kim Alvefur <zash@zash.se> parents: 2048 diff changeset	48 ------ ---------------
1940 453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	49 0.10 Works
2291 1eeda34d668b mod_storage_xmlarchive/README: Should work in 0.9 now Kim Alvefur <zash@zash.se> parents: 2048 diff changeset	50 0.9 Should work
1940 453eee191e18 mod_storage_xmlarchive/README: Add example configuration Kim Alvefur <zash@zash.se> parents: 1803 diff changeset	51 0.8 Does not work
2291 1eeda34d668b mod_storage_xmlarchive/README: Should work in 0.9 now Kim Alvefur <zash@zash.se> parents: 2048 diff changeset	52 ------ ---------------
2815 d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	53
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	54 Conversion to or from internal storage
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	55 --------------------------------------
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	56
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	57 This module stores data in a way that overlaps with the more recent
2817 f052b62c653c mod_storage_xmlarchive/README: Minor tweak Kim Alvefur <zash@zash.se> parents: 2815 diff changeset	58 archive support in `mod_storage_internal`, meaning e.g. [mod_migrate]
2815 d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	59 will not be able to cleanly convert to or from the `xmlarchive` format.
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	60
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	61 To mitigate this, an migration command has been added to
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	62 `mod_storage_xmlarchive`:
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	63
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	64 ``` bash
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	65 prosodyctl mod_storage_xmlarchive convert $DIR internal $STORE $JID
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	66 ```
d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	67
2818 88474dd1af48 Various READMEs: s/eg/e.g.g/ Kim Alvefur <zash@zash.se> parents: 2817 diff changeset	68 Where `$DIR` is `to` or `from`, `$STORE` is e.g. `archive` or `archive2`
2815 d48d4d9ccae7 mod_storage_xmlarchive: Add a prosodyctl command for migrating to/from the internal storage format Kim Alvefur <zash@zash.se> parents: 2359 diff changeset	69 for MAM and `muc_log` for MUC logs. Finally, `$JID` is the JID of the
2855 7713cd4fff8f mod_storage_xmlarchive/README: Fix typo Kim Alvefur <zash@zash.se> parents: 2854 diff changeset	70 user or MUC room to be migrated, which can be repeated.
2854 687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	71
3459 471855552f49 mod_storage_xmlarchive/README: Add reminder about backups Kim Alvefur <zash@zash.se> parents: 3451 diff changeset	72 ::: {.alert .alert-danger}
471855552f49 mod_storage_xmlarchive/README: Add reminder about backups Kim Alvefur <zash@zash.se> parents: 3451 diff changeset	73 Since this is a destructive command, don't forget to backup your data
471855552f49 mod_storage_xmlarchive/README: Add reminder about backups Kim Alvefur <zash@zash.se> parents: 3451 diff changeset	74 first.
3460 f25b0623b276 mod_storage_xmlarchive/README: And don't have prosody running at the same time Kim Alvefur <zash@zash.se> parents: 3459 diff changeset	75
f25b0623b276 mod_storage_xmlarchive/README: And don't have prosody running at the same time Kim Alvefur <zash@zash.se> parents: 3459 diff changeset	76 Prosody should not be running while converting data.
3459 471855552f49 mod_storage_xmlarchive/README: Add reminder about backups Kim Alvefur <zash@zash.se> parents: 3451 diff changeset	77 :::
471855552f49 mod_storage_xmlarchive/README: Add reminder about backups Kim Alvefur <zash@zash.se> parents: 3451 diff changeset	78
471855552f49 mod_storage_xmlarchive/README: Add reminder about backups Kim Alvefur <zash@zash.se> parents: 3451 diff changeset	79
2854 687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	80 Data structure
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	81 ==============
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	82
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	83 Data is split in three kinds of files and messages are grouped by day.
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	84 Prosodys `util.datamanager` is used, so all special characters in these
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	85 filenames are escaped and reside under `hostname/store` in Prosodys Data
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	86 directory, commonly `/var/lib/prosody`.
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	87
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	88 `username.list`
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	89 : A list of dates in `YYYY-MM-DD` format.
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	90
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	91 `username@YYYY-MM-DD.list`
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	92 : Index containing metadata for messages stored on that day.
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	93
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	94 `username@YYYY-MM-DD.xml`
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	95 : Messages in textual XML format, separated by newlines.
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	96
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	97 This makes it fairly simple and fast to find messages by timestamp.
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	98 Queries that are not time based, but limited to a specific contact may
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	99 be expensive as potentially the entire archive will be read.
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	100
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	101 Each archive ID is of the form `YYYY-MM-DD-random`, making lookups by
687b19cad4f5 mod_storage_xmlarchive/README: Add description of how data is stored Kim Alvefur <zash@zash.se> parents: 2818 diff changeset	102 archive id just as simple as time based queries.
3434 a6722a35f35e mod_storage_xmlarchive: Expand description in README Kim Alvefur <zash@zash.se> parents: 2855 diff changeset	103
a6722a35f35e mod_storage_xmlarchive: Expand description in README Kim Alvefur <zash@zash.se> parents: 2855 diff changeset	104 ## Limitations
a6722a35f35e mod_storage_xmlarchive: Expand description in README Kim Alvefur <zash@zash.se> parents: 2855 diff changeset	105
a6722a35f35e mod_storage_xmlarchive: Expand description in README Kim Alvefur <zash@zash.se> parents: 2855 diff changeset	106 - Only XML stanzas can be stored.
a6722a35f35e mod_storage_xmlarchive: Expand description in README Kim Alvefur <zash@zash.se> parents: 2855 diff changeset	107 - The deletion method only supports removing entire days at a time.

Mercurial > prosody-modules

annotate mod_storage_xmlarchive/README.markdown @ 3673:11ebf1da416b