Mercurial > prosody-modules

annotate mod_migrate_http_upload/mod_migrate_http_upload.lua @ 5571:ca3c2d11823c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
mod_pubsub_feeds: Track latest timestamp seen in feeds instead of last poll This should ensure that an entry that has a publish timestmap after the previously oldest post, but before the time of the last poll check, is published to the node. Previously if an entry would be skipped if it was published at 13:00 with a timestamp of 12:30, where the last poll was at 12:45. For feeds that lack a timestamp, it now looks for the first post that is not published, assuming that the feed is in reverse chronological order, then iterates back up from there.
author Kim Alvefur <zash@zash.se>
date Sun, 25 Jun 2023 16:27:55 +0200
parents f210f242cf17
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4468
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
1 -- Copyright (C) 2021 Kim Alvefur
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
2 --
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
3 -- This file is MIT licensed.
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
4
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
5 local lfs = require "lfs";
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
6 local st = require "util.stanza";
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
7 local jid = require "util.jid";
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
8 local paths = require "util.paths";
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
9 local unpack = table.unpack or _G.unpack;
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
10
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
11 function module.command(arg)
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
12 local sm = require "core.storagemanager";
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
13 local dm = sm.olddm;
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
14
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
15 local component, user_host = unpack(arg);
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
16
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
17 sm.initialize_host(component);
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
18
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
19 local new_uploads = sm.open(component, "uploads", "archive");
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
20
4470
203f0f06d766 mod_migrate_http_upload: Respect the 'http_upload_path' setting
Kim Alvefur <zash@zash.se>
parents: 4468
diff changeset
21 local legacy_storage_path = module:context(component):get_option_string("http_upload_path", paths.join(prosody.paths.data, "http_upload"));
203f0f06d766 mod_migrate_http_upload: Respect the 'http_upload_path' setting
Kim Alvefur <zash@zash.se>
parents: 4468
diff changeset
22
4468
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
23 local legacy_uploads = {};
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
24 for user in assert(dm.users(user_host, "http_upload", "list")) do
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
25 legacy_uploads[user] = dm.list_load(user, user_host, "http_upload");
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
26 end
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
27 while true do
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
28 local oldest_uploads, uploader;
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
29 for user, uploads in pairs(legacy_uploads) do
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
30 if uploads[1] and (not oldest_uploads or uploads[1].time < oldest_uploads[1].time) then
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
31 oldest_uploads, uploader = uploads, jid.join(user, user_host);
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
32 end
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
33 end
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
34 if not oldest_uploads then break end
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
35 local item = table.remove(oldest_uploads, 1);
4470
203f0f06d766 mod_migrate_http_upload: Respect the 'http_upload_path' setting
Kim Alvefur <zash@zash.se>
parents: 4468
diff changeset
36 local source_directory = paths.join(legacy_storage_path, item.dir);
4471
af7a9856950d mod_migrate_http_upload: Don't include per-upload directory twice
Kim Alvefur <zash@zash.se>
parents: 4470
diff changeset
37 local source_filename = paths.join(source_directory, item.filename);
4468
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
38 local target_filename = dm.getpath(item.dir, component, "http_file_share", "bin", true);
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
39 if not lfs.attributes(source_filename, "mode") then
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
40 print("Not migrating missing file " .. source_filename);
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
41 else
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
42 print("Moving " .. source_filename .. " to " .. target_filename .. " for " .. uploader);
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
43 local upload = st.stanza("request", {
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
44 xmlns = "urn:xmpp:http:upload:0";
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
45 filename = item.filename;
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
46 size = string.format("%d", item.size);
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
47 -- content-type not included with mod_http_upload
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
48 });
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
49 assert(new_uploads:append(nil, item.dir, upload, item.time, uploader));
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
50 assert(os.rename(source_filename, target_filename));
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
51 end
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
52 os.remove(source_directory); -- failure not fatal
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
53 end
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
54 for user, uploads in pairs(legacy_uploads) do
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
55 assert(dm.list_store(user, user_host, "http_upload", uploads));
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
56 end
4472
f210f242cf17 mod_migrate_http_upload: Remove storage path when done
Kim Alvefur <zash@zash.se>
parents: 4471
diff changeset
57 os.remove(legacy_storage_path);
4468
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
58 return 0;
5d8f9cc5c6fb mod_migrate_http_upload: Upload data converter to mod_http_file_share
Kim Alvefur <zash@zash.se>
parents:
diff changeset
59 end