prosody-modules: mod_firewall/definitions.lib.lua annotate

annotate mod_firewall/definitions.lib.lua @ 5173:460f78654864

mod_muc_rtbl: also filter messages This was a bit tricky because we don't want to run the JIDs through SHA256 on each message. Took a while to come up with this simple plan of just caching the SHA256 of the JIDs on the occupants. This will leave some dirt in the occupants after unloading the module, but that should be ok; once they cycle the room, the hashes will be gone. This is direly needed, otherwise, there is a tight race between the moderation activities and the actors joining the room.

author	Jonas Schäfer <jonas@wielicki.name>
date	Tue, 21 Feb 2023 21:37:27 +0100
parents	b88f05c878ac
children	0f943619e01a

rev	line source
2079 edec9de0220a mod_firewall: Silence warnings about unused arguments [luacheck] Kim Alvefur <zash@zash.se> parents: 1863 diff changeset	1
edec9de0220a mod_firewall: Silence warnings about unused arguments [luacheck] Kim Alvefur <zash@zash.se> parents: 1863 diff changeset	2 -- Name arguments are unused here
edec9de0220a mod_firewall: Silence warnings about unused arguments [luacheck] Kim Alvefur <zash@zash.se> parents: 1863 diff changeset	3 -- luacheck: ignore 212
999 197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	4
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	5 local definition_handlers = {};
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	6
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	7 local http = require "net.http";
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	8 local timer = require "util.timer";
1863 92602cfac751 mod_firewall: Fix missing import of util.set (used to be global) Kim Alvefur <zash@zash.se> parents: 999 diff changeset	9 local set = require"util.set";
999 197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	10 local new_throttle = require "util.throttle".create;
2586 d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	11 local hashes = require "util.hashes";
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	12 local jid = require "util.jid";
4016 b872f111b7af mod_firewall: Add option to ignore missing list files Matthew Wild <mwild1@gmail.com> parents: 3240 diff changeset	13 local lfs = require "lfs";
2128 21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	14
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	15 local multirate_cache_size = module:get_option_number("firewall_multirate_cache_limit", 1000);
999 197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	16
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	17 function definition_handlers.ZONE(zone_name, zone_members)
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	18 local zone_member_list = {};
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	19 for member in zone_members:gmatch("[^, ]+") do
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	20 zone_member_list[#zone_member_list+1] = member;
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	21 end
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	22 return set.new(zone_member_list)._items;
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	23 end
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	24
2128 21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	25 -- Helper function used by RATE handler
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	26 local function evict_only_unthrottled(name, throttle)
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	27 throttle:update();
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	28 -- Check whether the throttle is at max balance (i.e. totally safe to forget about it)
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	29 if throttle.balance < throttle.max then
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	30 -- Not safe to forget
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	31 return false;
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	32 end
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	33 end
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	34
999 197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	35 function definition_handlers.RATE(name, line)
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	36 local rate = assert(tonumber(line:match("([%d.]+)")), "Unable to parse rate");
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	37 local burst = tonumber(line:match("%(%sburst%s+([%d.]+)%s%)")) or 1;
2128 21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	38 local max_throttles = tonumber(line:match("%(%sentries%s+([%d]+)%s%)")) or multirate_cache_size;
2370 5fe483b73fd2 mod_firewall: Rate limiting: Document 'entries' and add option to allow overflowing when full Matthew Wild <mwild1@gmail.com> parents: 2131 diff changeset	39 local deny_when_full = not line:match("%(allow overflow%)");
2128 21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	40 return {
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	41 single = function ()
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	42 return new_throttle(rate*burst, burst);
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	43 end;
2859 22e11645a895 mod_firewall: Trim trailing whitespace [luacheck] Kim Alvefur <zash@zash.se> parents: 2587 diff changeset	44
2128 21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	45 multi = function ()
2370 5fe483b73fd2 mod_firewall: Rate limiting: Document 'entries' and add option to allow overflowing when full Matthew Wild <mwild1@gmail.com> parents: 2131 diff changeset	46 local cache = require "util.cache".new(max_throttles, deny_when_full and evict_only_unthrottled or nil);
2128 21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	47 return {
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	48 poll_on = function (_, key, amount)
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	49 assert(key, "no key");
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	50 local throttle = cache:get(key);
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	51 if not throttle then
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	52 throttle = new_throttle(rate*burst, burst);
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	53 if not cache:set(key, throttle) then
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	54 module:log("warn", "Multirate '%s' has hit its maximum number of active throttles (%d), denying new events", name, max_throttles);
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	55 return false;
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	56 end
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	57 end
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	58 return throttle:poll(amount);
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	59 end;
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	60 }
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	61 end;
21bc4d7cddae mod_firewall: Add support for throttling based on user-defined properties (experimental) Matthew Wild <mwild1@gmail.com> parents: 2079 diff changeset	62 };
999 197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	63 end
197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	64
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	65 local list_backends = {
2898 9fd61234b6f0 mod_firewall/definitions: Comments on LIST backends Kim Alvefur <zash@zash.se> parents: 2859 diff changeset	66 -- %LIST name: memory (limit: number)
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	67 memory = {
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	68 init = function (self, type, opts)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	69 if opts.limit then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	70 local have_cache_lib, cache_lib = pcall(require, "util.cache");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	71 if not have_cache_lib then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	72 error("In-memory lists with a size limit require Prosody 0.10");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	73 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	74 self.cache = cache_lib.new((assert(tonumber(opts.limit), "Invalid list limit")));
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	75 if not self.cache.table then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	76 error("In-memory lists with a size limit require a newer version of Prosody 0.10");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	77 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	78 self.items = self.cache:table();
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	79 else
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	80 self.items = {};
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	81 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	82 end;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	83 add = function (self, item)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	84 self.items[item] = true;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	85 end;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	86 remove = function (self, item)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	87 self.items[item] = nil;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	88 end;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	89 contains = function (self, item)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	90 return self.items[item] == true;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	91 end;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	92 };
2898 9fd61234b6f0 mod_firewall/definitions: Comments on LIST backends Kim Alvefur <zash@zash.se> parents: 2859 diff changeset	93
9fd61234b6f0 mod_firewall/definitions: Comments on LIST backends Kim Alvefur <zash@zash.se> parents: 2859 diff changeset	94 -- %LIST name: http://example.com/ (ttl: number, pattern: pat, hash: sha1)
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	95 http = {
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	96 init = function (self, url, opts)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	97 local poll_interval = assert(tonumber(opts.ttl or "3600"), "invalid ttl for <"..url.."> (expected number of seconds)");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	98 local pattern = opts.pattern or "([^\r\n]+)\r?\n";
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	99 assert(pcall(string.match, "", pattern), "invalid pattern for <"..url..">");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	100 if opts.hash then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	101 assert(opts.hash:match("^%w+$") and type(hashes[opts.hash]) == "function", "invalid hash function: "..opts.hash);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	102 self.hash_function = hashes[opts.hash];
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	103 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	104 local etag;
2522 72cbec103709 mod_firewall: Improve HTTP polling logic Matthew Wild <mwild1@gmail.com> parents: 2520 diff changeset	105 local failure_count = 0;
72cbec103709 mod_firewall: Improve HTTP polling logic Matthew Wild <mwild1@gmail.com> parents: 2520 diff changeset	106 local retry_intervals = { 60, 120, 300 };
4127 e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported Matthew Wild <mwild1@gmail.com> parents: 4017 diff changeset	107 -- By default only check the certificate if net.http supports SNI
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported Matthew Wild <mwild1@gmail.com> parents: 4017 diff changeset	108 local sni_supported = http.feature and http.features.sni;
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported Matthew Wild <mwild1@gmail.com> parents: 4017 diff changeset	109 local insecure = false;
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported Matthew Wild <mwild1@gmail.com> parents: 4017 diff changeset	110 if opts.checkcert == "never" then
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported Matthew Wild <mwild1@gmail.com> parents: 4017 diff changeset	111 insecure = true;
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported Matthew Wild <mwild1@gmail.com> parents: 4017 diff changeset	112 elseif (opts.checkcert == nil or opts.checkcert == "when-sni") and not sni_supported then
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported Matthew Wild <mwild1@gmail.com> parents: 4017 diff changeset	113 insecure = false;
e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported Matthew Wild <mwild1@gmail.com> parents: 4017 diff changeset	114 end
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	115 local function update_list()
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	116 http.request(url, {
4127 e9e10ec1b91c mod_firewall: Add checkcerts option for HTTP lists, cert verification disabled when SNI unsupported Matthew Wild <mwild1@gmail.com> parents: 4017 diff changeset	117 insecure = insecure;
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	118 headers = {
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	119 ["If-None-Match"] = etag;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	120 };
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	121 }, function (body, code, response)
2522 72cbec103709 mod_firewall: Improve HTTP polling logic Matthew Wild <mwild1@gmail.com> parents: 2520 diff changeset	122 local next_poll = poll_interval;
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	123 if code == 200 and body then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	124 etag = response.headers.etag;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	125 local items = {};
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	126 for entry in body:gmatch(pattern) do
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	127 items[entry] = true;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	128 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	129 self.items = items;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	130 module:log("debug", "Fetched updated list from <%s>", url);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	131 elseif code == 304 then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	132 module:log("debug", "List at <%s> is unchanged", url);
2522 72cbec103709 mod_firewall: Improve HTTP polling logic Matthew Wild <mwild1@gmail.com> parents: 2520 diff changeset	133 elseif code == 0 or (code >= 400 and code <=599) then
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	134 module:log("warn", "Failed to fetch list from <%s>: %d %s", url, code, tostring(body));
2522 72cbec103709 mod_firewall: Improve HTTP polling logic Matthew Wild <mwild1@gmail.com> parents: 2520 diff changeset	135 failure_count = failure_count + 1;
72cbec103709 mod_firewall: Improve HTTP polling logic Matthew Wild <mwild1@gmail.com> parents: 2520 diff changeset	136 next_poll = retry_intervals[failure_count] or retry_intervals[#retry_intervals];
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	137 end
2522 72cbec103709 mod_firewall: Improve HTTP polling logic Matthew Wild <mwild1@gmail.com> parents: 2520 diff changeset	138 if next_poll > 0 then
72cbec103709 mod_firewall: Improve HTTP polling logic Matthew Wild <mwild1@gmail.com> parents: 2520 diff changeset	139 timer.add_task(next_poll+math.random(0, 60), update_list);
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	140 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	141 end);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	142 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	143 update_list();
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	144 end;
2530 84e103fd8039 mod_firewall: Add dummy add/remove methods for HTTP lists Matthew Wild <mwild1@gmail.com> parents: 2528 diff changeset	145 add = function ()
84e103fd8039 mod_firewall: Add dummy add/remove methods for HTTP lists Matthew Wild <mwild1@gmail.com> parents: 2528 diff changeset	146 end;
84e103fd8039 mod_firewall: Add dummy add/remove methods for HTTP lists Matthew Wild <mwild1@gmail.com> parents: 2528 diff changeset	147 remove = function ()
84e103fd8039 mod_firewall: Add dummy add/remove methods for HTTP lists Matthew Wild <mwild1@gmail.com> parents: 2528 diff changeset	148 end;
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	149 contains = function (self, item)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	150 if self.hash_function then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	151 item = self.hash_function(item);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	152 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	153 return self.items and self.items[item] == true;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	154 end;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	155 };
2898 9fd61234b6f0 mod_firewall/definitions: Comments on LIST backends Kim Alvefur <zash@zash.se> parents: 2859 diff changeset	156
9fd61234b6f0 mod_firewall/definitions: Comments on LIST backends Kim Alvefur <zash@zash.se> parents: 2859 diff changeset	157 -- %LIST: file:/path/to/file
2532 2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	158 file = {
2535 b85d88737a32 mod_firewall: Fix file backend init code Matthew Wild <mwild1@gmail.com> parents: 2532 diff changeset	159 init = function (self, file_spec, opts)
4017 015452258952 mod_firewall: Ensure file lists are always initialized empty Matthew Wild <mwild1@gmail.com> parents: 4016 diff changeset	160 local n, items = 0, {};
015452258952 mod_firewall: Ensure file lists are always initialized empty Matthew Wild <mwild1@gmail.com> parents: 4016 diff changeset	161 self.items = items;
2535 b85d88737a32 mod_firewall: Fix file backend init code Matthew Wild <mwild1@gmail.com> parents: 2532 diff changeset	162 local filename = file_spec:gsub("^file:", "");
4016 b872f111b7af mod_firewall: Add option to ignore missing list files Matthew Wild <mwild1@gmail.com> parents: 3240 diff changeset	163 if opts.missing == "ignore" and not lfs.attributes(filename, "mode") then
b872f111b7af mod_firewall: Add option to ignore missing list files Matthew Wild <mwild1@gmail.com> parents: 3240 diff changeset	164 module:log("debug", "Ignoring missing list file: %s", filename);
b872f111b7af mod_firewall: Add option to ignore missing list files Matthew Wild <mwild1@gmail.com> parents: 3240 diff changeset	165 return;
b872f111b7af mod_firewall: Add option to ignore missing list files Matthew Wild <mwild1@gmail.com> parents: 3240 diff changeset	166 end
2532 2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	167 local file, err = io.open(filename);
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	168 if not file then
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	169 module:log("warn", "Failed to open list from %s: %s", filename, err);
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	170 return;
3240 c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded Matthew Wild <mwild1@gmail.com> parents: 2898 diff changeset	171 else
c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded Matthew Wild <mwild1@gmail.com> parents: 2898 diff changeset	172 for line in file:lines() do
c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded Matthew Wild <mwild1@gmail.com> parents: 2898 diff changeset	173 if not items[line] then
c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded Matthew Wild <mwild1@gmail.com> parents: 2898 diff changeset	174 n = n + 1;
c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded Matthew Wild <mwild1@gmail.com> parents: 2898 diff changeset	175 items[line] = true;
c30f2cfe9f15 mod_firewall: Assume empty list if file could not be loaded Matthew Wild <mwild1@gmail.com> parents: 2898 diff changeset	176 end
2536 22a271641c29 mod_firewall: Improve debug logging for LIST file backend Matthew Wild <mwild1@gmail.com> parents: 2535 diff changeset	177 end
2532 2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	178 end
2536 22a271641c29 mod_firewall: Improve debug logging for LIST file backend Matthew Wild <mwild1@gmail.com> parents: 2535 diff changeset	179 module:log("debug", "Loaded %d items from %s", n, filename);
2532 2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	180 end;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	181 add = function (self, item)
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	182 self.items[item] = true;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	183 end;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	184 remove = function (self, item)
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	185 self.items[item] = nil;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	186 end;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	187 contains = function (self, item)
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	188 return self.items and self.items[item] == true;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	189 end;
2ddb74805f91 mod_firewall: Add 'file' backend for lists (read-only atm) Matthew Wild <mwild1@gmail.com> parents: 2530 diff changeset	190 };
4512 b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	191
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	192 -- %LIST: pubsub:pubsub.example.com/node
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	193 -- TODO or the actual URI scheme? Bit overkill maybe?
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	194 -- TODO Publish items back to the service?
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	195 -- Step 1: Receiving pubsub events and storing them in the list
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	196 -- We'll start by using only the item id.
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	197 -- TODO Invent some custom schema for this? Needed for just a set of strings?
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	198 pubsubitemid = {
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	199 init = function(self, pubsub_spec, opts)
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	200 local service_addr, node = pubsub_spec:match("^([^/])/(.)");
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	201 module:depends("pubsub_subscription");
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	202 module:add_item("pubsub-subscription", {
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	203 service = service_addr;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	204 node = node;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	205 on_subscribed = function ()
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	206 self.items = {};
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	207 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	208 on_item = function (event)
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	209 self:add(event.item.attr.id);
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	210 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	211 on_retract = function (event)
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	212 self:remove(event.item.attr.id);
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	213 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	214 on_purge = function ()
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	215 self.items = {};
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	216 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	217 on_unsubscribed = function ()
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	218 self.items = nil;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	219 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	220 on_delete= function ()
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	221 self.items = nil;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	222 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	223 });
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	224 -- TODO Initial fetch? Or should mod_pubsub_subscription do this?
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	225 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	226 add = function (self, item)
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	227 if self.items then
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	228 self.items[item] = true;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	229 end
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	230 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	231 remove = function (self, item)
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	232 if self.items then
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	233 self.items[item] = nil;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	234 end
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	235 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	236 contains = function (self, item)
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	237 return self.items and self.items[item] == true;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	238 end;
b88f05c878ac mod_firewall: Add basic LIST backend for receiving items from PubSub Kim Alvefur <zash@zash.se> parents: 4127 diff changeset	239 };
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	240 };
2523 a3a18d09ae8a mod_firewall: Also handle HTTPS for lists Matthew Wild <mwild1@gmail.com> parents: 2522 diff changeset	241 list_backends.https = list_backends.http;
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	242
2586 d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	243 local normalize_functions = {
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	244 upper = string.upper, lower = string.lower;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	245 md5 = hashes.md5, sha1 = hashes.sha1, sha256 = hashes.sha256;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	246 prep = jid.prep, bare = jid.bare;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	247 };
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	248
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	249 local function wrap_list_method(list_method, filter)
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	250 return function (self, item)
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	251 return list_method(self, filter(item));
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	252 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	253 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	254
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	255 local function create_list(list_backend, list_def, opts)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	256 if not list_backends[list_backend] then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	257 error("Unknown list type '"..list_backend.."'", 0);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	258 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	259 local list = setmetatable({}, { __index = list_backends[list_backend] });
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	260 if list.init then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	261 list:init(list_def, opts);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	262 end
2586 d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	263 if opts.filter then
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	264 local filters = {};
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	265 for func_name in opts.filter:gmatch("[%w_]+") do
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	266 if func_name == "log" then
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	267 table.insert(filters, function (s)
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	268 --print("&&&&&", s);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	269 module:log("debug", "Checking list <%s> for: %s", list_def, s);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	270 return s;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	271 end);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	272 else
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	273 assert(normalize_functions[func_name], "Unknown list filter: "..func_name);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	274 table.insert(filters, normalize_functions[func_name]);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	275 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	276 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	277
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	278 local filter;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	279 local n = #filters;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	280 if n == 1 then
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	281 filter = filters[1];
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	282 else
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	283 function filter(s)
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	284 for i = 1, n do
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	285 s = filters[i](s or "");
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	286 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	287 return s;
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	288 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	289 end
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	290
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	291 list.add = wrap_list_method(list.add, filter);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	292 list.remove = wrap_list_method(list.remove, filter);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	293 list.contains = wrap_list_method(list.contains, filter);
d28e434cb5fd mod_firewall: Support filters for normalizing items before checking for them in lists Matthew Wild <mwild1@gmail.com> parents: 2536 diff changeset	294 end
2520 c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	295 return list;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	296 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	297
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	298 --[[
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	299 %LIST spammers: memory (source: /etc/spammers.txt)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	300
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	301 %LIST spammers: memory (source: /etc/spammers.txt)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	302
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	303
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	304 %LIST spammers: http://example.com/blacklist.txt
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	305 ]]
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	306
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	307 function definition_handlers.LIST(list_name, list_definition)
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	308 local list_backend = list_definition:match("^%w+");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	309 local opts = {};
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	310 local opt_string = list_definition:match("^%S+%s+%((.+)%)");
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	311 if opt_string then
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	312 for opt_k, opt_v in opt_string:gmatch("(%w+): ?([^,]+)") do
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	313 opts[opt_k] = opt_v;
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	314 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	315 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	316 return create_list(list_backend, list_definition:match("^%S+"), opts);
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	317 end
c6fd8975704b mod_firewall: Initial support for lists, in-memory and HTTP Matthew Wild <mwild1@gmail.com> parents: 2370 diff changeset	318
2528 44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	319 function definition_handlers.PATTERN(name, pattern)
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	320 local ok, err = pcall(string.match, "", pattern);
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	321 if not ok then
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	322 error("Invalid pattern '"..name.."': "..err);
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	323 end
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	324 return pattern;
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	325 end
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	326
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	327 function definition_handlers.SEARCH(name, pattern)
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	328 return pattern;
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	329 end
44a71584521d mod_firewall: Add SEARCH, PATTERN definitions and SCAN condition to check tokenized stanza:find() against a list Matthew Wild <mwild1@gmail.com> parents: 2523 diff changeset	330
999 197af8440ffb mod_firewall: Make defining objects generic (currently zones and rate limits), so more can easily be added. Also a syntax change... definition lines must begin with % Matthew Wild <mwild1@gmail.com> parents: diff changeset	331 return definition_handlers;

Mercurial > prosody-modules

annotate mod_firewall/definitions.lib.lua @ 5173:460f78654864