Mercurial > prosody-modules

annotate mod_migrate/mod_migrate.lua @ 2812:fd4a5ba12fa2

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
mod_migrate: Catch failure to use user listing API and suggest the alternative
author Kim Alvefur <zash@zash.se>
date Wed, 01 Nov 2017 22:12:09 +0100
parents 17408ddd34b0
children cd9831b27746
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1791
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
1 -- mod_migrate
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
2
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
3 local sm = require"core.storagemanager";
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
4 local um = require"core.usermanager";
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
5 local mm = require"core.modulemanager";
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
6
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
7 function module.command(arg)
1985
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
8 local host, source_stores, migrate_to, user = unpack(arg);
1791
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
9 if not migrate_to then
1813
f02f52a2ee11 mod_migrate: Add support for migrating archives
Kim Alvefur <zash@zash.se>
parents: 1812
diff changeset
10 return print("Usage: prosodyctl mod_migrate example.com <source-store>[-<store-type>] <target-driver> [users]*");
1791
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
11 end
2139
17408ddd34b0 mod_migrate: Check that the hostname is known, or yell and abort (thanks Link Mauve)
Kim Alvefur <zash@zash.se>
parents: 2064
diff changeset
12 if not prosody.hosts[host] then
17408ddd34b0 mod_migrate: Check that the hostname is known, or yell and abort (thanks Link Mauve)
Kim Alvefur <zash@zash.se>
parents: 2064
diff changeset
13 return print(("The host %q is not know by Prosody."):format(host));
17408ddd34b0 mod_migrate: Check that the hostname is known, or yell and abort (thanks Link Mauve)
Kim Alvefur <zash@zash.se>
parents: 2064
diff changeset
14 end
1791
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
15 sm.initialize_host(host);
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
16 um.initialize_host(host);
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
17 local module = module:context(host);
1985
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
18 for source_store in source_stores:gmatch("[^,]+") do
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
19 local store_type = source_store:match("%-(%a+)$");
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
20 if store_type then
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
21 source_store = source_store:sub(1, -2-#store_type);
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
22 end
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
23 local storage = module:open_store(source_store, store_type);
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
24 local target = assert(sm.load_driver(host, migrate_to));
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
25 target = assert(target:open(source_store, store_type));
1813
f02f52a2ee11 mod_migrate: Add support for migrating archives
Kim Alvefur <zash@zash.se>
parents: 1812
diff changeset
26
1985
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
27 local function migrate_user(username)
2028
530bf8ef2e55 mod_migrate: Include debug data of what store is being migrated
Kim Alvefur <zash@zash.se>
parents: 1985
diff changeset
28 module:log("info", "Migrating %s data for %s", source_store, username);
1985
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
29 local data, err = storage:get(username);
2064
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error
Kim Alvefur <zash@zash.se>
parents: 2028
diff changeset
30 if not data and err then
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error
Kim Alvefur <zash@zash.se>
parents: 2028
diff changeset
31 module:log("error", "Could not read data: %s", err);
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error
Kim Alvefur <zash@zash.se>
parents: 2028
diff changeset
32 else
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error
Kim Alvefur <zash@zash.se>
parents: 2028
diff changeset
33 local ok, err = target:set(username, data);
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error
Kim Alvefur <zash@zash.se>
parents: 2028
diff changeset
34 if not ok then
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error
Kim Alvefur <zash@zash.se>
parents: 2028
diff changeset
35 module:log("error", "Could not write data: %s", err);
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error
Kim Alvefur <zash@zash.se>
parents: 2028
diff changeset
36 end
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error
Kim Alvefur <zash@zash.se>
parents: 2028
diff changeset
37 end
1985
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
38 end
1791
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
39
1985
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
40 if store_type == "archive" then
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
41 function migrate_user(username)
2028
530bf8ef2e55 mod_migrate: Include debug data of what store is being migrated
Kim Alvefur <zash@zash.se>
parents: 1985
diff changeset
42 module:log("info", "Migrating %s archive items for %s", source_store, username);
1985
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
43 local count, errs = 0, 0;
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
44 for id, item, when, with in storage:find(username) do
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
45 local ok, err = target:append(username, id, item, when, with);
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
46 if ok then
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
47 count = count + 1;
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
48 else
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
49 module:log("warn", "Error: %s", err);
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
50 errs = errs + 1;
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
51 end
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
52 if ( count + errs ) % 100 == 0 then
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
53 module:log("info", "%d items migrated, %d errors", count, errs);
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
54 end
1813
f02f52a2ee11 mod_migrate: Add support for migrating archives
Kim Alvefur <zash@zash.se>
parents: 1812
diff changeset
55 end
1985
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
56 module:log("info", "%d items migrated, %d errors", count, errs);
1813
f02f52a2ee11 mod_migrate: Add support for migrating archives
Kim Alvefur <zash@zash.se>
parents: 1812
diff changeset
57 end
f02f52a2ee11 mod_migrate: Add support for migrating archives
Kim Alvefur <zash@zash.se>
parents: 1812
diff changeset
58 end
f02f52a2ee11 mod_migrate: Add support for migrating archives
Kim Alvefur <zash@zash.se>
parents: 1812
diff changeset
59
1985
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
60 if arg[4] then
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
61 for i = 4, #arg do
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
62 migrate_user(arg[i]);
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
63 end
7821a6986e68 mod_migrate: Support migrating multiple stores
Kim Alvefur <zash@zash.se>
parents: 1813
diff changeset
64 else
2812
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative
Kim Alvefur <zash@zash.se>
parents: 2139
diff changeset
65 xpcall(function()
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative
Kim Alvefur <zash@zash.se>
parents: 2139
diff changeset
66 for user in um.users(host) do
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative
Kim Alvefur <zash@zash.se>
parents: 2139
diff changeset
67 migrate_user(user);
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative
Kim Alvefur <zash@zash.se>
parents: 2139
diff changeset
68 end
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative
Kim Alvefur <zash@zash.se>
parents: 2139
diff changeset
69 end,
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative
Kim Alvefur <zash@zash.se>
parents: 2139
diff changeset
70 function (err)
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative
Kim Alvefur <zash@zash.se>
parents: 2139
diff changeset
71 module:log("error", "Could not list users, you'll have to supply them as arguments");
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative
Kim Alvefur <zash@zash.se>
parents: 2139
diff changeset
72 module:log("error", "The error was: %s", err);
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative
Kim Alvefur <zash@zash.se>
parents: 2139
diff changeset
73 end);
1791
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
74 end
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
75 end
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends
Kim Alvefur <zash@zash.se>
parents:
diff changeset
76 end