prosody-modules: mod_migrate/mod

annotate mod_migrate/mod_migrate.lua @ 3503:882180b459a0

mod_pubsub_post: Restructure authentication and authorization (BC) This deprecates the default "superuser" actor model and makes the default equivalent to the previous "request.id". A single actor and secret per node is supported because HTTP and WebHooks don't normally include any authorization identity. Allowing authentication bypass when no secret is given should be relatively safe when the actor is unprivileged, as will be unless explicitly configured otherwise.

author	Kim Alvefur <zash@zash.se>
date	Sat, 30 Mar 2019 21:16:13 +0100
parents	cd9831b27746
children	611ac62c5b63

rev	line source
1791 8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	1 -- mod_migrate
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	2
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	3 local sm = require"core.storagemanager";
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	4 local um = require"core.usermanager";
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	5
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	6 function module.command(arg)
2816 cd9831b27746 mod_migrate: Remove unused locals Kim Alvefur <zash@zash.se> parents: 2812 diff changeset	7 local host, source_stores, migrate_to = unpack(arg);
1791 8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	8 if not migrate_to then
1813 f02f52a2ee11 mod_migrate: Add support for migrating archives Kim Alvefur <zash@zash.se> parents: 1812 diff changeset	9 return print("Usage: prosodyctl mod_migrate example.com <source-store>[-<store-type>] <target-driver> [users]*");
1791 8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	10 end
2139 17408ddd34b0 mod_migrate: Check that the hostname is known, or yell and abort (thanks Link Mauve) Kim Alvefur <zash@zash.se> parents: 2064 diff changeset	11 if not prosody.hosts[host] then
17408ddd34b0 mod_migrate: Check that the hostname is known, or yell and abort (thanks Link Mauve) Kim Alvefur <zash@zash.se> parents: 2064 diff changeset	12 return print(("The host %q is not know by Prosody."):format(host));
17408ddd34b0 mod_migrate: Check that the hostname is known, or yell and abort (thanks Link Mauve) Kim Alvefur <zash@zash.se> parents: 2064 diff changeset	13 end
1791 8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	14 sm.initialize_host(host);
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	15 um.initialize_host(host);
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	16 local module = module:context(host);
1985 7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	17 for source_store in source_stores:gmatch("[^,]+") do
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	18 local store_type = source_store:match("%-(%a+)$");
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	19 if store_type then
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	20 source_store = source_store:sub(1, -2-#store_type);
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	21 end
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	22 local storage = module:open_store(source_store, store_type);
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	23 local target = assert(sm.load_driver(host, migrate_to));
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	24 target = assert(target:open(source_store, store_type));
1813 f02f52a2ee11 mod_migrate: Add support for migrating archives Kim Alvefur <zash@zash.se> parents: 1812 diff changeset	25
1985 7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	26 local function migrate_user(username)
2028 530bf8ef2e55 mod_migrate: Include debug data of what store is being migrated Kim Alvefur <zash@zash.se> parents: 1985 diff changeset	27 module:log("info", "Migrating %s data for %s", source_store, username);
1985 7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	28 local data, err = storage:get(username);
2064 2a82c55762a7 mod_migrate: Log read and write errors instead of raising error Kim Alvefur <zash@zash.se> parents: 2028 diff changeset	29 if not data and err then
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error Kim Alvefur <zash@zash.se> parents: 2028 diff changeset	30 module:log("error", "Could not read data: %s", err);
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error Kim Alvefur <zash@zash.se> parents: 2028 diff changeset	31 else
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error Kim Alvefur <zash@zash.se> parents: 2028 diff changeset	32 local ok, err = target:set(username, data);
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error Kim Alvefur <zash@zash.se> parents: 2028 diff changeset	33 if not ok then
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error Kim Alvefur <zash@zash.se> parents: 2028 diff changeset	34 module:log("error", "Could not write data: %s", err);
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error Kim Alvefur <zash@zash.se> parents: 2028 diff changeset	35 end
2a82c55762a7 mod_migrate: Log read and write errors instead of raising error Kim Alvefur <zash@zash.se> parents: 2028 diff changeset	36 end
1985 7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	37 end
1791 8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	38
1985 7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	39 if store_type == "archive" then
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	40 function migrate_user(username)
2028 530bf8ef2e55 mod_migrate: Include debug data of what store is being migrated Kim Alvefur <zash@zash.se> parents: 1985 diff changeset	41 module:log("info", "Migrating %s archive items for %s", source_store, username);
1985 7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	42 local count, errs = 0, 0;
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	43 for id, item, when, with in storage:find(username) do
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	44 local ok, err = target:append(username, id, item, when, with);
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	45 if ok then
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	46 count = count + 1;
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	47 else
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	48 module:log("warn", "Error: %s", err);
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	49 errs = errs + 1;
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	50 end
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	51 if ( count + errs ) % 100 == 0 then
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	52 module:log("info", "%d items migrated, %d errors", count, errs);
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	53 end
1813 f02f52a2ee11 mod_migrate: Add support for migrating archives Kim Alvefur <zash@zash.se> parents: 1812 diff changeset	54 end
1985 7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	55 module:log("info", "%d items migrated, %d errors", count, errs);
1813 f02f52a2ee11 mod_migrate: Add support for migrating archives Kim Alvefur <zash@zash.se> parents: 1812 diff changeset	56 end
f02f52a2ee11 mod_migrate: Add support for migrating archives Kim Alvefur <zash@zash.se> parents: 1812 diff changeset	57 end
f02f52a2ee11 mod_migrate: Add support for migrating archives Kim Alvefur <zash@zash.se> parents: 1812 diff changeset	58
1985 7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	59 if arg[4] then
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	60 for i = 4, #arg do
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	61 migrate_user(arg[i]);
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	62 end
7821a6986e68 mod_migrate: Support migrating multiple stores Kim Alvefur <zash@zash.se> parents: 1813 diff changeset	63 else
2812 fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative Kim Alvefur <zash@zash.se> parents: 2139 diff changeset	64 xpcall(function()
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative Kim Alvefur <zash@zash.se> parents: 2139 diff changeset	65 for user in um.users(host) do
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative Kim Alvefur <zash@zash.se> parents: 2139 diff changeset	66 migrate_user(user);
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative Kim Alvefur <zash@zash.se> parents: 2139 diff changeset	67 end
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative Kim Alvefur <zash@zash.se> parents: 2139 diff changeset	68 end,
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative Kim Alvefur <zash@zash.se> parents: 2139 diff changeset	69 function (err)
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative Kim Alvefur <zash@zash.se> parents: 2139 diff changeset	70 module:log("error", "Could not list users, you'll have to supply them as arguments");
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative Kim Alvefur <zash@zash.se> parents: 2139 diff changeset	71 module:log("error", "The error was: %s", err);
fd4a5ba12fa2 mod_migrate: Catch failure to use user listing API and suggest the alternative Kim Alvefur <zash@zash.se> parents: 2139 diff changeset	72 end);
1791 8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	73 end
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	74 end
8df071457dee mod_migrate: Provides a prosodyctl mod_migrate command for copying data between storage backends Kim Alvefur <zash@zash.se> parents: diff changeset	75 end

Mercurial > prosody-modules

annotate mod_migrate/mod_migrate.lua @ 3503:882180b459a0