sat_docs: scripts/association/update

annotate scripts/association/update_garradin.py @ 52:4d924e65a4ba

compte-rendu de l'AG ordinaire du 24 aout 2015 à Sankt Augustin

author	souliane <souliane@mailoo.org>
date	Sun, 04 Oct 2015 11:21:02 +0200
parents	c34633e3f56a
children

rev	line source
51 c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	1 #!/usr/bin/python
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	2 # -- coding: utf-8 --
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	3
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	4 # Scripts to update Garradin database from various CSV inputs.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	5 # Copyright (C) 2015 Adrien Cossa <souliane@mailoo.org>
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	6
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	7 # This program is free software: you can redistribute it and/or modify
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	8 # it under the terms of the GNU Affero General Public License as published by
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	9 # the Free Software Foundation, either version 3 of the License, or
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	10 # (at your option) any later version.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	11
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	12 # This program is distributed in the hope that it will be useful,
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	13 # but WITHOUT ANY WARRANTY; without even the implied warranty of
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	14 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	15 # GNU Affero General Public License for more details.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	16
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	17 # You should have received a copy of the GNU Affero General Public License
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	18 # along with this program. If not, see <http://www.gnu.org/licenses/>.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	19
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	20 import sqlite3
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	21 import csv, unicodecsv
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	22 import os
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	23 import os.path
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	24 import tempfile
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	25 from glob import glob
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	26 from time import strftime
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	27 from datetime import datetime
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	28 from collections import OrderedDict
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	29
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	30
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	31 ### PATHS AND OTHER STUFF TO CONFIGURE ###
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	32
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	33
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	34 DB_FILE = u"./association.sqlite"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	35 INPUT_PATH = u"../../../sat_private"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	36
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	37 APAYER_FILES = (u"apayer", u"SALUT-A-TOI_*.csv")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	38 CM_FILES = (u"credit_mutuel/dernieres_operations", "_.csv")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	39 DJANGO_FILES = (u"django", u"sat_website_subscriptions.csv")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	40
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	41 LOG_PATH = u"./OUTPUT"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	42 INFO, WARNING = 1, 2
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	43 LOG_LEVELS = (INFO, WARNING)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	44
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	45
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	46 def dirtyGetDateOfFile(path):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	47 """Extract the date at the end of the filename, just before the extension.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	48
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	49 @return: datetime
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	50 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	51 # FIXME: there should be a better way to do that
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	52 token = os.path.basename(path)[-12:-4]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	53 return datetime.strptime(token, "%Y%m%d")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	54
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	55
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	56 ### !!! DO NOT MODIFY BELOW THIS LINE !!! ###
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	57
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	58
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	59 ### DATE AND TIME ###
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	60
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	61 DATE_ORIGIN = u"1970-01-01"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	62 DATE_FORMAT = "%Y-%m-%d"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	63 TODAY = strftime(DATE_FORMAT)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	64
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	65
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	66 ### RETRIEVE DATABASE INFOS ###
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	67
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	68
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	69 class GarradinDB(object):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	70
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	71 TABLES = ("membres", "cotisations", "cotisations_membres")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	72
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	73 def __init__(self, cursor):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	74 self.cursor = cursor
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	75
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	76 def getTableInfos(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	77 infos = {}
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	78 for table in self.TABLES:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	79 cursor.execute("PRAGMA table_info(%s)" % table)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	80 rows = cursor.fetchall()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	81 infos[table] = OrderedDict()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	82 for row in rows:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	83 # don't use {row[1]: row[2]...} : it messes the order
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	84 infos[table][row[1]] = row[2] # field name: field type
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	85 return infos
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	86
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	87 def getLastAutoUpdate(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	88 """Get the datetime of the last database automatic update.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	89
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	90 @return: datetime
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	91 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	92 self.cursor.execute("SELECT valeur FROM config WHERE cle='last_auto_update'")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	93 row = self.cursor.fetchone()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	94 try:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	95 date = row[0]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	96 except TypeError:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	97 date = DATE_ORIGIN
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	98 return datetime.strptime(date, DATE_FORMAT)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	99
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	100 def setLastAutoUpdate(self, date=TODAY):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	101 """Set to date of the last database automatic update to today."""
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	102 self.cursor.execute("REPLACE INTO config VALUES ('last_auto_update', ?)", (date,))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	103
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	104
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	105 con = sqlite3.connect(DB_FILE)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	106 with con:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	107 cursor = con.cursor()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	108 db = GarradinDB(cursor)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	109 TABLE_INFOS = db.getTableInfos()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	110 LAST_AUTO_UPDATE = db.getLastAutoUpdate()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	111
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	112
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	113 ### LOGGERS ###
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	114
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	115
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	116 class Logger(object):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	117
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	118 def __init__(self, path):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	119 if not os.path.isdir(path):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	120 os.mkdir(path)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	121 basename = "update_garradin_{day}_%s.log".format(day=strftime("%Y%m%d"))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	122 self.info_file = os.path.join(path, basename % "info")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	123 self.warning_file = os.path.join(path, basename % "warning")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	124
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	125 def write(self, file, msg):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	126 with open(file, 'a') as file:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	127 file.write((u"[%s] %s\n" % (strftime("%d/%m/%Y@%H:%m:%S"), msg)).encode("utf-8"))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	128
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	129 def info(self, msg):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	130 self.write(self.info_file, msg)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	131 if INFO in LOG_LEVELS:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	132 print msg
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	133
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	134 def warning(self, msg):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	135 self.write(self.warning_file, msg)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	136 if WARNING in LOG_LEVELS:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	137 print msg
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	138
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	139 logger = Logger(LOG_PATH)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	140
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	141
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	142 ### CSV DIALECTS ###
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	143
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	144
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	145 # FIXME: Python CSV doesn't handle CSV file starting with a BOM
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	146
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	147 class my_dialect(csv.excel):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	148 """Describe my properties for CSV files."""
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	149 delimiter = ';'
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	150 csv.register_dialect("my_dialect", my_dialect)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	151
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	152
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	153 ### GENERIC ENTRIES' CLASSES ###
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	154
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	155
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	156 class Entry(OrderedDict):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	157
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	158 TYPE_INTEGER = u"INTEGER"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	159 TYPE_TEXT = u"TEXT"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	160
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	161 ID_UNIQUE_KEYS = ("FIELD_NAME", "FIELD_SURNAME", "FIELD_EMAIL") # define unique members
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	162 ID_EQUAL_KEYS = ID_UNIQUE_KEYS # define equality, not even needed to update the database
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	163
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	164 FIELD_NAME = u"prenom"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	165 FIELD_SURNAME = u"nom"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	166 FIELD_EMAIL = u"email"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	167
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	168 FIELDS = [FIELD_NAME, FIELD_SURNAME, FIELD_EMAIL]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	169
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	170 MATCH_NONE = 0
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	171 MATCH_INCOMPLETE = 1
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	172 MATCH_UNIQUE = 2
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	173 MATCH_EQUAL = 3
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	174
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	175 matched = MATCH_NONE
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	176
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	177 def __init__(self, entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	178 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	179
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	180 @param entry(dict or row): entry data
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	181 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	182 if isinstance(entry, dict):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	183 data = OrderedDict({key: value for key, value in entry.iteritems() if key in self.FIELDS})
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	184 if hasattr(entry, "identityDict"):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	185 data.update(entry.identityDict(self))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	186 else:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	187 data = self.row2dict(entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	188 OrderedDict.__init__(self, data)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	189
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	190 def __iter__(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	191 # XXX: use the order of self.FIELDS and ignore OrderedDict internal order
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	192 for field in self.FIELDS:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	193 if field in self:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	194 yield field
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	195
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	196 def row2dict(self, row):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	197 """Return a dict representation of a row.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	198
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	199 @param row (list[unicode]): a row of information
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	200 @return: OrderedDict {unicode: unicode} with:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	201 - key: field name
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	202 - value: field value
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	203 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	204 data = OrderedDict()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	205 index = 0
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	206 for field in self.FIELDS:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	207 if row[index]:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	208 data[field] = row[index]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	209 index += 1
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	210 return data
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	211
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	212 def identityDict(self, cls=None):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	213 """Return a dict representing a user identity.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	214
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	215 @param cls (class): if specified, the result dict will use cls's
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	216 identity fields for its own keys (instead of self's fields).
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	217 @return: dict{unicode: unicode}
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	218 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	219 if cls is None:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	220 cls = self
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	221 return {getattr(cls, key): self[getattr(self, key)] for key in self.ID_EQUAL_KEYS}
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	222
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	223 def identityMatches(self, entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	224 """Return a dict compiling the identity matches with another entry.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	225
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	226 @param entry (Entry): another entry
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	227 @return: dict{unicode: bool}
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	228 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	229 return {key: self[getattr(self, key)] == entry[getattr(entry, key)] for key in self.ID_EQUAL_KEYS}
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	230
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	231 def identityMatch(self, entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	232 """Return a value representing the matching level with another entry.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	233
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	234 @param entry (Entry): another entry
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	235 @return: int in (Entry.MATCH_NONE, Entry.MATCH_INCOMPLETE, Entry.MATCH_UNIQUE, Entry.MATCH_EQUAL)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	236 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	237 matches = self.identityMatches(entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	238 if len([match for match in matches.values() if not match]) == 0:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	239 return self.MATCH_EQUAL
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	240 match_name = matches["FIELD_NAME"] and matches["FIELD_SURNAME"]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	241 if match_name and matches["FIELD_EMAIL"]:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	242 return self.MATCH_UNIQUE
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	243 if match_name or matches["FIELD_EMAIL"]:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	244 return self.MATCH_INCOMPLETE
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	245 return self.MATCH_NONE
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	246
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	247 def identityToString(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	248 """Return a human-readable string representing a user identity.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	249
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	250 @return: unicode
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	251 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	252 return "%s %s <%s>" % (self[self.FIELD_NAME], self[self.FIELD_SURNAME], self[self.FIELD_EMAIL])
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	253
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	254 def valuesToString(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	255 """List the values.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	256
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	257 @return: unicode
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	258 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	259 return u", ".join(self.values())
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	260
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	261 def fields(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	262 """List the fields to which a value is associated.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	263
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	264 @return: list[unicode]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	265 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	266 return [field for field in self.FIELDS if field in self]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	267
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	268 def fieldsToString(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	269 """List the fields to which a value is associated.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	270
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	271 @return: unicode
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	272 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	273 return u", ".join(self.fields())
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	274
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	275 def changeset(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	276 """Return the list of fields and values as needed to update the database.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	277
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	278 @return: tuple(unicode, list)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	279 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	280 changes = ['%s=?' % field for field in self.fields()]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	281 return (u", ".join(changes), self.values())
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	282
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	283
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	284 class GarradinDBEntry(Entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	285
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	286 def __init__(self, cursor, entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	287 self.cursor = cursor
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	288 Entry.__init__(self, entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	289
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	290 def insertOrReplace(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	291 """Insert or replace the entry in the garradin database."""
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	292 fields = self.fieldsToString()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	293 args = self.values()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	294 placeholders = ",".join([u"?" for arg in args])
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	295 # XXX: "INSERT OR REPLACE" is needed for updating lastrowid, "REPLACE" is not enough
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	296 request = "INSERT OR REPLACE INTO %s (%s) VALUES (%s)" % (self.TABLE, fields, placeholders)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	297 self.cursor.execute(request, args)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	298
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	299
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	300 class CSVEntry(Entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	301
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	302 def date(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	303 """Return the date associated with this entry.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	304
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	305 @return: datetime
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	306 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	307 try:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	308 format = self.DATE_FORMAT
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	309 except AttributeError:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	310 format = DATE_FORMAT
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	311 return datetime.strptime(self[self.FIELD_DATE], format)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	312
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	313
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	314 ### MEMBERS ENTRIES ###
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	315
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	316
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	317 class GarradinDBMember(GarradinDBEntry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	318
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	319 TABLE = u"membres"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	320 FIELDS = TABLE_INFOS[TABLE].keys()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	321
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	322 FIELD_ID_CATEGORY = u"id_categorie"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	323 FIELD_PASSWORD = u"passe"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	324 FIELD_AUTO_UPDATE = u"date_maj_auto"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	325 FIELD_ADDRESS = u"adresse"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	326
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	327 ID_EQUAL_KEYS = Entry.ID_UNIQUE_KEYS + ("FIELD_ADDRESS",)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	328
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	329 def __init__(self, cursor, entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	330 GarradinDBEntry.__init__(self, cursor, entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	331 try:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	332 self.pop(self.FIELD_PASSWORD)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	333 except KeyError:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	334 pass
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	335
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	336 if hasattr(entry, "address"):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	337 self[self.FIELD_ADDRESS] = entry.address()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	338
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	339 def setAutoUpdate(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	340 """Update the value of the field self.FIELD_AUTO_UPDATE."""
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	341 self[self.FIELD_AUTO_UPDATE] = TODAY
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	342
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	343 def values(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	344 """List the values.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	345
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	346 @return: list
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	347 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	348 values = []
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	349 for field, value in self.iteritems():
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	350 if TABLE_INFOS[self.TABLE].get(field, self.TYPE_TEXT) == self.TYPE_TEXT:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	351 value = u"%s" % value.replace('\r', '').replace('\n', '\\n')
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	352 values.append(unicode(value))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	353 return values
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	354
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	355
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	356 class GarradinDBMembership(GarradinDBEntry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	357
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	358 TABLE = u"cotisations_membres"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	359 FIELDS = TABLE_INFOS[TABLE].keys()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	360
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	361 FIELD_MEMBER = u"id_membre"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	362 FIELD_MEMBERSHIP = u"id_cotisation"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	363 FIELD_DATE = u"date"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	364
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	365
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	366 class GarradinDBMembershipType(GarradinDBEntry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	367
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	368 TABLE = u"cotisations"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	369 FIELDS = TABLE_INFOS[TABLE].keys()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	370
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	371 FIELD_AMOUNT = u"montant"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	372
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	373
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	374 class GarradinCSVMember(CSVEntry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	375 """Like GarradinDBMember but with a "categorie" field in second position."""
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	376
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	377 FIELD_ID_CATEGORY = u"id_categorie"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	378 FIELD_CATEGORY = u"categorie"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	379 FIELD_ADDRESS = u"adresse"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	380
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	381 FIELDS = GarradinDBMember.FIELDS[:1] + [FIELD_CATEGORY] + GarradinDBMember.FIELDS[1:]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	382
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	383 def __init__(self, cursor, db_member):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	384 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	385
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	386 @param db_member (GarradinDBMember)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	387 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	388 CSVEntry.__init__(self, db_member)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	389
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	390 if self.FIELD_ID_CATEGORY not in self:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	391 self[self.FIELD_ID_CATEGORY] = 1
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	392 if self.FIELD_CATEGORY not in self:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	393 cursor.execute("SELECT nom FROM membres_categories WHERE id=?", (self[self.FIELD_ID_CATEGORY],))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	394 rows = cursor.fetchone()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	395 self[self.FIELD_CATEGORY] = rows[0]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	396
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	397
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	398 class ApayerCSVEntry(CSVEntry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	399
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	400 FIELD_NAME = u"Prenom"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	401 FIELD_SURNAME = u"Nom"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	402 FIELD_EMAIL = u"Courriel"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	403 FIELD_AMOUNT = u"Montant"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	404 FIELD_DATE = u"DatePaiement"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	405 DATE_FORMAT = u"%d/%m/%y"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	406
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	407 # XXX: there's a BOM at the beginning of the CSV file
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	408 FIELDS = ["Mnemonique", "TPE", "DatePaiement", "ObjetPaiement",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	409 "AutreObjet", "Montant", "Devise", "Reference",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	410 "Commentaire", "ReferencBancaire", "NumeroAutorisation",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	411 "Nom", "Prenom", "Adresse", "CodePostal", "Ville", "Courriel",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	412 "Etat", "MotifRefus", "Cvx", "Vld", "Brand", "Status3DS"]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	413
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	414 FILTERED_OUT = ("", u"Mnemonique", "TPE", "Devise", "ReferencBancaire",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	415 "NumeroAutorisation", "Cvx", "Vld", "Brand", "Status3DS")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	416
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	417 def address(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	418 return u"\n".join([self["Adresse"], self["CodePostal"], self["Ville"]])
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	419
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	420
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	421 class DjangoCSVEntry(CSVEntry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	422
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	423 FIELD_NAME = u"prenom"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	424 FIELD_SURNAME = u"nom"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	425 FIELD_EMAIL = u"courriel"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	426 FIELD_AMOUNT = u"montant"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	427 FIELD_DATE = u"date"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	428
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	429 FIELDS = ["date", "prenom", "nom", "adresse", "courriel", "jid", "montant",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	430 "moyen_paiement", "reference", "commentaire", "lettre_info",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	431 "langue"]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	432
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	433
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	434
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	435 ### ENTRIES FOR THE FINANCES ###
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	436
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	437
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	438 class GarradinDBOperation(GarradinDBEntry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	439
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	440 TABLE = u"compta_journal"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	441
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	442
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	443 class GarradinCSVOperation(CSVEntry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	444
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	445 FIELDS = (u"Numéro mouvement", u"Date", u"Type de mouvement", u"Catégorie",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	446 u"Libellé", u"Montant", u"Compte de débit - numéro",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	447 u"Compte de débit - libellé", u"Compte de crédit - numéro",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	448 u"Compte de crédit - libellé", u"Moyen de paiement",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	449 u"Numéro de chèque", u"Numéro de pièce", u"Remarques")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	450
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	451
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	452 class CmCSVOperation(CSVEntry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	453
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	454 FIELD_DATE = u"Date d'opération"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	455 DATE_FORMAT = u"%d/%m/%Y"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	456
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	457 FIELDS = (FIELD_DATE, u"Date de valeur", u"Débit", u"Crédit",
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	458 u"Libellé", u"Solde")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	459
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	460
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	461 ### CLASSES FOR THE IMPORTS ###
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	462
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	463
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	464 class EntryMatcher(object):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	465
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	466 def __init__(self, cursor):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	467 self.cursor = cursor
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	468
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	469 def processEntryForCSVExport(self, entry, context):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	470 """Return the processed entry ready to be exported to CSV,
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	471 or None if the entry is conflicting with another one already existing.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	472
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	473 @param entry (Entry): an Entry instance
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	474 @param context (unicode): context indication for the user
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	475 @return: GarradinCSVMember or None
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	476 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	477 # this method is not optimised but it's sensible... don't mess with it.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	478 tmp_entry = GarradinDBMember(self.cursor, entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	479 base_entry = self.getMatchingEntry(tmp_entry, context)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	480 if base_entry is None:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	481 return None
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	482 base_entry.update(tmp_entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	483
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	484 new_entry = GarradinCSVMember(self.cursor, base_entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	485 new_entry.matched = base_entry.matched
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	486 return new_entry
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	487
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	488 def getMatchingEntry(self, new_entry, context):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	489 """Try to get a matching entry, the behavior depends of the found match:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	490
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	491 - no match at all: returns a new empty entry
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	492 - incomplete match: returns None and write a warning in the logs
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	493 - unique or equal match: returns the matching database entry
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	494
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	495 @param new_entry (GarradinDBMember): entry to be matched
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	496 @param context (unicode): context indication for the user
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	497 @return: GarradinDBMember or None:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	498 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	499 self.cursor.execute("SELECT * FROM %s" % new_entry.TABLE)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	500 rows = self.cursor.fetchall()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	501
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	502 for row in rows:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	503 row = list(row)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	504 current_entry = GarradinDBMember(self, row)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	505 match = current_entry.identityMatch(new_entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	506 if match == Entry.MATCH_NONE:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	507 continue # try to match next row
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	508
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	509 current_entry.matched = match
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	510
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	511 if match == Entry.MATCH_EQUAL:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	512 return current_entry # we still need to update the membership
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	513 if match == Entry.MATCH_INCOMPLETE:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	514 log = logger.warning
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	515 desc = (u"INCOMPLETE", u"A MANUAL check and process is required!")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	516 elif match == Entry.MATCH_UNIQUE:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	517 log = logger.info
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	518 desc = (u"unique", u"The database entry will be updated.")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	519 log(u"A %s match has been found between these two entries from \"%s\" and garradin's \"%s\" table:" % (desc[0], context, current_entry.TABLE))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	520 log(u" - %s" % new_entry.valuesToString())
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	521 log(u" - %s" % current_entry.valuesToString())
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	522 log(" --> %s" % desc[1])
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	523 return current_entry if match == Entry.MATCH_UNIQUE else None
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	524
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	525 return GarradinDBMember(self, {})
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	526
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	527 class MembershipManager(object):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	528 def __init__(self, cursor):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	529 self.cursor = cursor
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	530 self.memberships = {}
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	531
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	532 def setCache(self, email, amount, date):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	533 """Write a membership in cache for later database update.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	534
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	535 @param email (unicode): email adress of the member
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	536 @param amount (int): amount of the membership fee
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	537 @param date (datetime): date of the subscription
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	538 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	539 self.memberships[email] = (amount, date)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	540
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	541 def getMembershipAmount(self, membership_id):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	542 """Return a membership amount.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	543
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	544 @param membership_id (int): membership ID in the "cotisations" table
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	545 @return: int
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	546 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	547 self.cursor.execute("SELECT %s FROM %s WHERE id=?" % (GarradinDBMembershipType.FIELD_AMOUNT, GarradinDBMembershipType.TABLE), (membership_id,))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	548 return int(self.cursor.fetchone()[0])
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	549
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	550 def getMemberIdentity(self, member_id):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	551 """Return a human-readable identity.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	552
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	553 @param member_id (int): member ID in the "membres" table
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	554 @return: unicode
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	555 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	556 self.cursor.execute("SELECT * FROM %s WHERE id=?" % GarradinDBMember.TABLE, (member_id,))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	557 return GarradinDBMember(self, self.cursor.fetchone()).identityToString()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	558
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	559 def getMembershipType(self, amount):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	560 """Return the suitable membership type ID and amount for the given amount.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	561
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	562 @param amount (int): payment amount
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	563 @return: (int, int)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	564 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	565 request = "SELECT id, MAX({field}) FROM {table} WHERE {field} <= ?".format(field=GarradinDBMembershipType.FIELD_AMOUNT, table=GarradinDBMembershipType.TABLE)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	566 self.cursor.execute(request, (amount,))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	567 return self.cursor.fetchone()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	568
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	569 def updateMembershipFromCache(self, member_id, email):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	570 """Look in the cache to update the membership fee of a member.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	571
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	572 @param member_id (int): member ID in the "membres" table
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	573 @param email(unicode): member email
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	574 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	575 membership = self.memberships.pop(email, None)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	576 if membership:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	577 amount, date = membership
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	578 self.updateMembership(member_id, amount, date)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	579
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	580 def updateMembership(self, member_id, amount, date_payment):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	581 """Update the membership fee of a member.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	582
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	583 @param member_id (int): member ID in the "membres" table
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	584 @param amount (float): amount of the membership fee
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	585 @param date_payment (unicode): date of the payment
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	586 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	587 member = self.getMemberIdentity(member_id)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	588 self.cursor.execute("SELECT * FROM %s WHERE id_membre=?" % GarradinDBMembership.TABLE, (member_id,))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	589 rows = self.cursor.fetchall()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	590 entries = [GarradinDBMembership(self.cursor, row) for row in rows]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	591 entries.sort(key=lambda entry: entry[entry.FIELD_DATE])
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	592
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	593 # first we check for inconsistency
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	594 last_date, last_amount = datetime.strptime(DATE_ORIGIN, DATE_FORMAT), 0
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	595 for entry in entries:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	596 current_date = datetime.strptime(entry[entry.FIELD_DATE], DATE_FORMAT)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	597 current_amount = self.getMembershipAmount(entry[entry.FIELD_MEMBERSHIP])
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	598 if (current_date - last_date).days < 365:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	599 logger.warning(u"At least two memberships within less than one year have been registered for %s:" % member)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	600 logger.warning(u" - € %d on %s" % (last_amount, last_date))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	601 logger.warning(u" - € %d on %s" % (current_amount, current_date))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	602 logger.warning(u" --> Please fix it MANUALLY!")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	603 last_date, last_amount = current_date, current_amount
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	604
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	605 if (date_payment - last_date).days < 365:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	606 amount += last_amount # cumulate two payments
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	607 else:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	608 data = {GarradinDBMembership.FIELD_MEMBER: member_id,
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	609 GarradinDBMembership.FIELD_DATE: date_payment}
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	610 entry = GarradinDBMembership(self.cursor, data)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	611
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	612 m_type_id, m_type_amount = self.getMembershipType(amount)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	613 delta = amount - m_type_amount
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	614 if delta:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	615 logger.warning(u"We received € %.2f from %s: a membership fee of € %d will be registered." % (amount, member, m_type_amount))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	616 logger.warning(u" --> Please MANUALLY register a donation of € %.2f." % delta)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	617 entry[entry.FIELD_MEMBERSHIP] = m_type_id
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	618 entry.insertOrReplace()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	619
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	620 class MembersImporter(object):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	621
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	622 def __init__(self, cursor):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	623 self.cursor = cursor
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	624 self.fails = []
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	625 self.membership_manager = MembershipManager(cursor)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	626
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	627 def fixRowFromLeadingEqualSign(self, row):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	628 """Fix leading equal sign in CSV field since it is not handled by Python.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	629
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	630 @param data (dict{unicode: unicode}): row data
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	631 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	632 for field, value in row.iteritems():
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	633 if value.startswith('="') and value.endswith('"'):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	634 row[field] = value[2:-1]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	635
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	636 def importFromCSV(self, input_csv, output_csv=None):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	637 """Import members from a CSV to garradin.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	638
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	639 @param input_csv (unicode): path to the input CSV file
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	640 @param output_csv (unicode): path to store intermediate Garradin CSV
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	641 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	642 tmp_csv = output_csv if output_csv else tempfile.mkstemp()[1]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	643 matcher = EntryMatcher(self.cursor)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	644 with open(input_csv, 'r') as csv_file:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	645 reader = unicodecsv.DictReader(csv_file, dialect="my_dialect")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	646 logger.info("Processing %s..." % input_csv)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	647 with open(tmp_csv, 'w') as tmp_file:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	648 writer = unicodecsv.DictWriter(tmp_file, GarradinCSVMember.FIELDS, dialect="my_dialect")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	649 fails = []
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	650 for row in reader:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	651 input_entry = self.CSV_ENTRY_CLASS(row)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	652
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	653 date = input_entry.date()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	654 if (LAST_AUTO_UPDATE - date).days > self.AGE_LIMIT:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	655 continue # skip older entries
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	656
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	657 self.fixRowFromLeadingEqualSign(input_entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	658 if not self.checkEntry(input_entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	659 self.fails.append(input_entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	660 continue
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	661 entry = matcher.processEntryForCSVExport(input_entry, self.CONTEXT)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	662 if not entry:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	663 continue
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	664 if entry.matched != Entry.MATCH_EQUAL:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	665 writer.writerow(entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	666
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	667 self.extraProcessEntry(input_entry, entry)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	668 self.printFails()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	669
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	670 self.importFromGarradinCSV(tmp_csv)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	671 if not output_csv:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	672 os.remove(tmp_csv)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	673
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	674 def checkEntry(self, input_entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	675 return True
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	676
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	677 def printFails(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	678 pass
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	679
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	680 def extraProcessEntry(self, input_entry, entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	681 pass
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	682
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	683 def importFromGarradinCSV(self, csv_file):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	684 """Import a garradin CSV to the database.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	685
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	686 @param csv_file (unicode): file to import
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	687 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	688 with open(csv_file, 'r') as file:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	689 reader = unicodecsv.reader(file, dialect="my_dialect")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	690 for row in reader:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	691 row.pop(1) # remove the category name
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	692 db_entry = GarradinDBMember(self.cursor, row)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	693 db_entry.setAutoUpdate()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	694 db_entry.insertOrReplace()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	695
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	696 self.membership_manager.updateMembershipFromCache(self.cursor.lastrowid, db_entry[db_entry.FIELD_EMAIL])
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	697
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	698 action = "updated" if db_entry.get('id', None) else "added"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	699 logger.warning("The member %s has been %s." % (db_entry.identityToString(), action))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	700
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	701
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	702 class ApayerImporter(MembersImporter):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	703
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	704 CSV_ENTRY_CLASS = ApayerCSVEntry
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	705
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	706 # since we receive the CSV every Sunday, operations can be one week old
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	707 # without having been processed yet, even if you do a daily auto update
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	708 AGE_LIMIT = 14
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	709
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	710 CONTEXT = u"apayer"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	711
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	712 STATE_ACCEPTED = u"Paiement accepté (payé)"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	713
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	714 def checkEntry(self, input_entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	715 return input_entry["Etat"] == self.STATE_ACCEPTED
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	716
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	717 def printFails(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	718 if self.fails:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	719 logger.warning(u"The following \"apayer\" operations have NOT been completed, you MAY want to contact the users:")
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	720 for row in self.fails:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	721 fields = ["%s: %s" % (key, value) for key, value in row.iteritems() if key not in ApayerCSVEntry.FILTERED_OUT and value]
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	722 logger.warning(u" - %s" % ", ".join(fields))
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	723
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	724 def extraProcessEntry(self, input_entry, entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	725 # update membership fee
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	726 date = input_entry.date()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	727 amount = float(input_entry[input_entry.FIELD_AMOUNT])
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	728 if entry.get("id", None):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	729 self.membership_manager.updateMembership(entry["id"], amount, date)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	730 else: # we don't know the member ID yet
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	731 self.membership_manager.setCache(entry[entry.FIELD_EMAIL], amount, date)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	732
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	733
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	734 class DjangoImporter(MembersImporter):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	735
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	736 CSV_ENTRY_CLASS = DjangoCSVEntry
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	737
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	738 # if you do a daily auto update, this file is processed everyday
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	739 AGE_LIMIT = 2
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	740
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	741 CONTEXT = u"django form"
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	742
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	743 def extraProcessEntry(self, input_entry, entry):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	744 date = input_entry.date()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	745 if not entry.get("id", None): # new user
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	746 self.membership_manager.setCache(entry[entry.FIELD_EMAIL], 0, date)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	747
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	748
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	749 class CmImporter(object):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	750
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	751 AGE_LIMIT = 60
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	752
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	753 def importFromCSV(self, csv_file):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	754 """Import financial operation from credit mutuel CSV to garradin.
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	755
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	756 @param csv_file (unicode): path to django form CSV file
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	757 """
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	758 raise NotImplementedError # TODO: update database with entry
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	759
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	760
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	761 ### MAIN CLASS ###
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	762
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	763
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	764 class GarradinUpdater(object):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	765
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	766 def __init__(self, cursor):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	767 self.cursor = cursor
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	768
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	769 def importFromApayer(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	770 """Import CSV files from apayer"""
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	771 for apayer_csv in glob(os.path.join(INPUT_PATH, *APAYER_FILES)):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	772 if dirtyGetDateOfFile(apayer_csv) >= LAST_AUTO_UPDATE:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	773 ApayerImporter(self.cursor).importFromCSV(apayer_csv)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	774
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	775 def importFromDjango(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	776 """Import all Django CSV files"""
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	777 # FIXME: import JID, subscription to mailing list and comment
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	778 for django_csv in glob(os.path.join(INPUT_PATH, *DJANGO_FILES)):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	779 DjangoImporter(self.cursor).importFromCSV(django_csv)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	780
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	781 def importFromCm(self):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	782 """Import all credit mutuel CSV files"""
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	783 for cm_csv in glob(os.path.join(INPUT_PATH, *CM_FILES)):
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	784 if dirtyGetDateOfFile(cm_csv) >= LAST_AUTO_UPDATE:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	785 CmImporter(self.cursor).importFromCSV(cm_csv)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	786
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	787
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	788 con = sqlite3.connect(DB_FILE)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	789 with con:
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	790 cursor = con.cursor()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	791 updater = GarradinUpdater(cursor)
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	792 updater.importFromApayer()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	793 updater.importFromDjango()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	794 #updater.importFromCm()
c34633e3f56a add a script to semi-automatically update garradin database (work in progress) souliane <souliane@mailoo.org> parents: diff changeset	795 GarradinDB(cursor).setLastAutoUpdate("2015-06-01")

Mercurial > sat_docs

annotate scripts/association/update_garradin.py @ 52:4d924e65a4ba