sat_docs: flatpak/build_manifest.py annotate

annotate flatpak/build_manifest.py @ 151:6f8c849daf2a

flatpak (build_manifest): updated org.freedesktop.Platform runtime version: manifests were using an old 1.6 version of the runtime, this patch fixes them by using the current 18.8 one. Thanks to that, the main_loop.patch which was made because of an outdated version of GLib (see https://github.com/flatpak/flatpak/issues/1899) is not needed anymore and has been removed. Some useless extensions have also been removed from default manifest.

author	Goffi <goffi@goffi.org>
date	Wed, 26 Jun 2019 20:49:31 +0200
parents	10adfa3f9552
children	df75b62e653e

rev	line source
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1 #!/usr/bin/env python3
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	2
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	3 import tempfile
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	4 import subprocess
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	5 from pathlib import Path
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	6 from typing import List
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	7 from dataclasses import dataclass
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	8 import hashlib
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	9 from ftplib import FTP
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	10 from urllib.parse import urlparse
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	11 from textwrap import dedent
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	12 import sys
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	13 import os
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	14 import json
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	15 import time
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	16 import argparse
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	17 import shutil
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	18 from packaging.version import parse as parse_version
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	19 import requests
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	20 from lxml import etree
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	21
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	22
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	23 CACHE_LIMIT = 3600 * 24
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	24 PYTHON_DEP_OVERRIDE = {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	25 "lxml": {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	26 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	27 "python2 ./setup.py install --prefix=${FLATPAK_DEST} --optimize=1"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	28 ],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	29 "build-options": {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	30 "env": {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	31 "XSLT_CONFIG": "pkg-config libxslt"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	32 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	33 },
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	34 },
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	35 "dbus-python": {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	36 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	37 "python2 setup.py build",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	38 "python2 setup.py install --prefix=${FLATPAK_DEST}",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	39 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	40 },
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	41 "kivy": {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	42 "post-install": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	43 # this file is not copied normally when installing with
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	44 # `python2 setup.py install`.
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	45 # TODO: report upstream
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	46 "cp kivy/setupconfig.py /app/lib/python2.7/site-packages/Kivy-*.egg/kivy/"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	47 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	48 },
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	49 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	50 PYTHON_DEP_REQUIREMENTS_UPDATE = {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	51 # service-identity is not seen as a twisted requirement, so it's sometimes misplaced
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	52 'twisted': ['service-identity'],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	53 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	54 PYTHON_SETUP_REQUIREMENTS = [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	55 'setuptools', # to have an up-to-date version
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	56 'setuptools_scm',
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	57 'docutils', # needed my m2r
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	58 'mistune', # needed my m2r
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	59 'm2r', # needed by automat
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	60 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	61 DEFAULT_MANIFEST = {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	62 "runtime": "org.freedesktop.Platform",
151 6f8c849daf2a flatpak (build_manifest): updated org.freedesktop.Platform runtime version: Goffi <goffi@goffi.org> parents: 150 diff changeset	63 "runtime-version": "18.08",
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	64 "sdk": "org.freedesktop.Sdk",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	65 "command": "sat_wrapper",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	66 "finish-args": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	67 "--socket=session-bus",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	68 "--share=network",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	69 "--filesystem=home"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	70 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	71 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	72 SHOW_REQUIRES_HEADER = 'Requires: '
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	73 SETTINGS_KEY = '_build_settings'
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	74 APPDATA_RELEASE_DEV_TEXT = dedent("""\
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	75 This is a development version, used as a preview.
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	76 Please note that it is incomplete and it probably contains bugs.
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	77 """)
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	78 OVERWRITE_WARNING = "{} already exists, do you want to overwrite it (y/N)? "
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	79
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	80
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	81 @dataclass
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	82 class Package:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	83 name: str
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	84 version: str
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	85 hash_: str
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	86 url: str
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	87 requirements: List[str]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	88
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	89 def __hash__(self):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	90 return hash(self.name)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	91
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	92
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	93 def print_step(step):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	94 print()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	95 print("┌" + "─" * (len(step) + 2) + "┐")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	96 print("│ {} │".format(step))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	97 print("└" + "─" * (len(step) + 2) + "┘")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	98 print()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	99
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	100
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	101 def parse_args():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	102 parser = argparse.ArgumentParser(
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	103 description='Flatpak manifest builder for Salut à Toi')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	104
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	105 build_group = parser.add_argument_group('building', 'options used to generate the manifest')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	106 export_group = parser.add_argument_group('export', 'otions used to building files')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	107
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	108 # build group
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	109 build_group.add_argument('-f', '--force', action="store_true",
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	110 help="force overwritting of existing manifest and appdata file")
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	111 build_group.add_argument('--ignore-cache', action='append', default=[],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	112 help='ignore the cache of this step ("all" to ignore all caches)')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	113 build_group.add_argument(
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	114 '--deps-dir',
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	115 help="use this directory to build_group python dependencies (it won't be deleted at "
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	116 "the end, useful when you want to re-use it and avoir re-downloading)")
140 e23e414987d7 flatpak (build_manifest): --no-binary is currently disabled as version parsing doesn't handle yet wheel packages. Goffi <goffi@goffi.org> parents: 139 diff changeset	117 # build_group.add_argument('--no-binary', help="don't use binary packages")
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	118
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	119 # export group
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	120 export_group.add_argument('-s', '--symlink', choices=['no', 'all', 'cache'],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	121 default='no',
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	122 help='"no" to copy all files, "all" to symlink all files, "cache" to copy files '
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	123 'and symlink only cache (default: "no")')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	124 export_group.add_argument('-F', '--force-export', action='store_true',
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	125 help='force overwritting of existing files/symlinks when exporting (DEFAULT: '
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	126 'existing files are skipped)')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	127 export_group.add_argument('-e', '--export', type=Path,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	128 help="export build files to this directory (DEFAULT: don't export files)")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	129
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	130 # common
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	131 parser.add_argument('name', type=str, help="name of the package to build_group")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	132 parser.add_argument('version', type=str, help="version of the package to build_group")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	133
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	134 args = parser.parse_args()
140 e23e414987d7 flatpak (build_manifest): --no-binary is currently disabled as version parsing doesn't handle yet wheel packages. Goffi <goffi@goffi.org> parents: 139 diff changeset	135 # FIXME: no_binary is currently not managed because version parser doesn't handle
e23e414987d7 flatpak (build_manifest): --no-binary is currently disabled as version parsing doesn't handle yet wheel packages. Goffi <goffi@goffi.org> parents: 139 diff changeset	136 # wheels
e23e414987d7 flatpak (build_manifest): --no-binary is currently disabled as version parsing doesn't handle yet wheel packages. Goffi <goffi@goffi.org> parents: 139 diff changeset	137 args.no_binary = True
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	138
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	139 if 'all' in args.ignore_cache and len(args.ignore_cache) != 1:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	140 parser.error('"all" cannot be used with step names in --ignore-cache, '
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	141 'use it alone')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	142
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	143 if args.export is not None and not args.export.is_dir():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	144 parser.error(f"{args.export} is not a directory!")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	145
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	146 return args
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	147
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	148
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	149 ## useful methods ##
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	150
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	151 def get_cache(name):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	152 """Retrieve cache for a step
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	153
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	154 cache will be ignored if too old, or explicitly ignored by user
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	155 @param name(str): name of the step
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	156 @return (object): cached data
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	157 """
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	158 # name of the cache without prefix, mainly used for python_deps
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	159 shortname = name.split('__', 1)[0]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	160
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	161 if shortname in args.ignore_cache or 'all' in args.ignore_cache:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	162 print(f"ignoring cache for {shortname}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	163 args.ignore_cache.remove(shortname)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	164 return None
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	165 try:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	166 cache_name = cache[name]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	167 cache_updated = cache_name['updated']
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	168 except KeyError:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	169 return None
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	170 if time.time() - cache_updated > CACHE_LIMIT:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	171 print(f"cache for {name} is too old, we won't use it")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	172 return None
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	173 print(f"cache found for {shortname}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	174 return cache_name['data']
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	175
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	176
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	177 def set_cache(name, data):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	178 """Update cache for a step
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	179
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	180 @param name(str): name of the step
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	181 @param data(object): data to cache
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	182 """
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	183 cache_name = cache.setdefault(name, {})
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	184 cache_name['updated'] = time.time()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	185 cache_name['data'] = data
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	186 with cache_path.open('w') as f:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	187 json.dump(cache, f)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	188
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	189
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	190 def get_python_package(package_name, step_name=None, step_message=None, with_pre=False,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	191 version=None):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	192 """Generate module for a Python package
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	193
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	194 @param package_name(str, list[str]): name of the Python package
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	195 use list of str to get several packages at once
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	196 @param step_name(None, str): name of the step, None to use package_name
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	197 @param step_message(None, str): message of the step (None to use default one)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	198 use empty string to disable it
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	199 @param with_pre(bool): if True retrieve also pre-releases
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	200 @param version(str, None): indicate the version of the package to download
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	201 if None, the most recent version compativle with `with_pre` will be used
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	202 """
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	203 single = isinstance(package_name, str)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	204 if step_name is None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	205 step_name = package_name if single else ' - '.join(package_name)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	206 if step_message != "":
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	207 print_step(step_message or f"retrieving latest version of {package_name}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	208 cache = get_cache(step_name)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	209 if cache is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	210 return cache
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	211 package_names = [package_name] if single else package_name
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	212 data = []
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	213 for name in package_names:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	214 r = requests.get(f"https://pypi.org/pypi/{name}/json")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	215 r.raise_for_status()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	216
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	217 releases_data = []
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	218 for version_raw, release_data in r.json()["releases"].items():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	219 if version is not None and version != version_raw:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	220 continue
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	221 release_version = parse_version(version_raw)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	222 if release_version.is_prerelease and not with_pre:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	223 continue
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	224 releases_data.append((release_version, release_data))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	225
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	226 # we sort releases by version to be sure to have latests one
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	227 releases_data.sort(key=lambda d: d[0])
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	228
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	229 try:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	230 release_version, release_data = releases_data[-1]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	231 package = next(r for r in release_data if r["packagetype"] == "sdist")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	232 except (IndexError, StopIteration):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	233 raise RuntimeError(f"Can't find a matching package for {name}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	234
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	235 print(f"{name} {release_version} will be used")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	236 dep_data = {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	237 "name": name,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	238 "buildsystem": "simple",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	239 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	240 "python2 setup.py install --prefix=${FLATPAK_DEST}"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	241 ],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	242 "sources": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	243 {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	244 "type": "archive",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	245 "url": package["url"],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	246 "sha256": package["digests"]["sha256"],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	247 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	248 ],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	249 "ensure-writable": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	250 "/lib/python2.7/site-packages/easy-install.pth",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	251 "/lib/python2.7/site-packages/setuptools.pth",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	252 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	253 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	254 data.append(dep_data)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	255
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	256 set_cache(step_name, data)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	257 return data
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	258
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	259
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	260 def file_upload(filename, dest="/app/bin", src=None, replace=None, use_install=False):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	261 """Generate manifest modules to upload a local file
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	262
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	263 @param filename(str): name of the local file to upload
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	264 Note that it will be overwritted if replace is used
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	265 @param dest(str, Path): path where the file must be installed in the container
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	266 @param src(str, None): when replace is used, name of the source file
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	267 @param replace(dict, None): mapping of text to replace
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	268 Use when file must be dynamicly adapted to the container, note that src must be
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	269 set to the name of the template file if this is used.
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	270 e.g. {'##COMMAND##', 'cagou'} will replace all '##COMMAND##' in filename
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	271 by 'cagou'
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	272 @param use_install(bool): if True, install file with `install src dest`
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	273 else, use `mv src dest`
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	274 @return (list(dict)): modules to add to the manifest
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	275 """
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	276 assert isinstance(filename, str) and '/' not in filename
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	277 print_step(f"generating {filename} module")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	278 # no cache for this one, to be sure to have always latest version
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	279 filename = Path(filename)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	280 if src is None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	281 file_to_test = filename
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	282 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	283 src = Path(src)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	284 file_to_test = src
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	285 if not file_to_test.exists():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	286 raise RuntimeError(
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	287 f"{file_to_test} is missing, it is needed to build the manifest!")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	288
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	289 if replace:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	290 if src is None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	291 raise ValueError(
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	292 '"src" must be set to name of template file if replace is used')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	293 print(f'doing replacement in template "{src}" to dest "{filename}"')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	294 # there are strings to replace, we read file, do the changes and dump the
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	295 # result in <filename>
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	296 with open(src) as f:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	297 buff = f.read()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	298
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	299 for old, new in replace.items():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	300 buff = buff.replace(old, new)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	301
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	302 with open(filename, 'w') as f:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	303 f.write(buff)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	304 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	305 if src is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	306 raise ValueError('"src" must not be used if replace is not used')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	307
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	308 with filename.open('rb') as f:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	309 hash_ = hashlib.sha256(f.read()).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	310
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	311 dest = Path(dest)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	312
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	313 dep_data = {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	314 "name": str(filename),
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	315 "buildsystem": "simple",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	316 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	317 ],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	318 "sources": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	319 {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	320 "type": "file",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	321 "path": str(filename),
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	322 "sha256": hash_,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	323 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	324 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	325 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	326
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	327 build_commands = dep_data['build-commands']
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	328 if use_install:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	329 build_commands.append(f"install -Dv {filename} {dest}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	330 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	331 if dest.as_posix() not in ('/app', '/app/bin'):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	332 # just in case the destination directory doesn't exist
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	333 build_commands.append(f"mkdir -p {dest.parent}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	334 build_commands.append(f"mv -v {filename} {dest}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	335
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	336 return [dep_data]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	337
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	338
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	339 def file_from_url(url, dest=None, step_name=None, step_message=None):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	340 """Generate manifest modules for a file either on a http(s) url or local
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	341
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	342 @param url(str): url of the file to use, or local path
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	343 if it starts with http, url will be used, else file_upload(url) will be used
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	344 @param dest(str, None): path were the file should be copied
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	345 """
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	346 parsed = urlparse(url)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	347 if not parsed.scheme:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	348 return file_upload(url)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	349 if not parsed.scheme.startswith('http'):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	350 raise NotImplementedError (
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	351 f'we can\'t use the URL "{url}", the scheme "{parsed.scheme}" is not managed')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	352
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	353 filepath = Path(parsed.path)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	354 stem = filepath.stem
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	355
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	356 if step_name is None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	357 # we use this name to easily ignore cache (with stem) while avoiding
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	358 # conflict if we have 2 URLs with the same stem
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	359 step_name = f"{stem}__{url}"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	360
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	361 if step_message is None:
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	362 step_message = f"generating module for {stem}"
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	363
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	364 print_step(step_message)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	365 cache = get_cache(step_name)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	366 if cache is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	367 return cache
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	368
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	369 r = requests.get(url)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	370 r.raise_for_status()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	371 file_hash = hashlib.sha256(r.content).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	372
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	373 dep_data = {"name": stem}
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	374
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	375 if dest is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	376 dest = Path(dest)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	377 dep_data.update({
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	378 'buildsystem': 'simple',
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	379 'build-commands':[
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	380 f'mkdir -p {dest.parent}',
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	381 f'mv "{filepath.name}" "{dest}"',
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	382 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	383 })
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	384
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	385 dep_data['sources'] = [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	386 {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	387 "type": "file",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	388 "url": url,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	389 "sha256": file_hash,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	390 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	391 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	392
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	393 data = [dep_data]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	394 set_cache(step_name, data)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	395 return data
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	396
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	397
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	398 def get_requirements(pip, package_name):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	399 """Retrieve requirements for an installed python package
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	400
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	401 @param pip(Path): path to pip executable to use
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	402 package must be installed in the environment of this pip
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	403 @param package_name(str): name of the package to retrieve
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	404 @return(list[str]): found requirements
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	405 """
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	406 show_cplted = subprocess.run([pip, "show", package_name],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	407 capture_output=True, text=True)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	408 show_cplted.check_returncode()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	409
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	410 lines = show_cplted.stdout.split('\n')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	411 requirement_raw = next(l for l in lines if l.startswith(SHOW_REQUIRES_HEADER))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	412 requirement_raw = requirement_raw[len(SHOW_REQUIRES_HEADER):]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	413 requirements = [canonical(p) for p in requirement_raw.split(',') if p.strip()]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	414 requirements_update = [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	415 canonical(r) for r in PYTHON_DEP_REQUIREMENTS_UPDATE.get(package_name, [])]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	416 new_requirements = set(requirements_update).difference(requirements)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	417 if new_requirements:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	418 print("adding extra requirements to {}: {}".format(
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	419 package_name, ', '.join(new_requirements)))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	420 requirements.extend(new_requirements)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	421 return requirements
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	422
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	423
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	424 def resolve_requirements(package, deps_map, deps, indent=0):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	425 """Recursively resolve requirements
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	426
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	427 @param package(Package): retrieve dependencies of this package
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	428 @param deps_map(dict): map from dependency name to Package instance
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	429 @param deps(list[package]): ordered dependencies
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	430 this list is updated in place
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	431 @param indent(int): use internally to print dependencies tree
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	432 """
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	433 if package in deps:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	434 return
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	435 print(" " * indent + package.name)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	436 reqs_data = [deps_map[r] for r in package.requirements]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	437
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	438 for data in reqs_data:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	439 resolve_requirements(data, deps_map, deps, indent+1)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	440
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	441 deps.append(package)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	442
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	443
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	444 def get_hg_id_date(path):
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	445 """Get short identifier and date of current commit from given Mercurial repository
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	446
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	447 version is retrieve with `hg id`, a "+" is appended after shortrev if it has
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	448 been modified.
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	449 @param path(str, Path): path to the repository
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	450 @return(tuple(str, date)): found revision + iso date
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	451 """
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	452 hg_cplted = subprocess.run(
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	453 ["hg", "id", "--template", "{id\|short}{dirty}\n{date\|isodate}", path],
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	454 capture_output=True, text=True)
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	455 hg_cplted.check_returncode()
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	456 return hg_cplted.stdout.split('\n')
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	457
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	458
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	459 def get_cache_dir():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	460 """Return path to directory to use for cache"""
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	461 return Path(f"cache_{app_id}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	462
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	463
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	464 def canonical(name):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	465 """Get canonical name of a package"""
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	466 return name.lower().replace('_', '-').strip()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	467
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	468
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	469 modules = []
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	470 cache_path = Path.home() / Path('.cache/sat_flatpak_cache.json')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	471 if not cache_path.exists():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	472 cache = {}
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	473 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	474 with cache_path.open() as f:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	475 cache = json.load(f)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	476
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	477
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	478 ## steps ##
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	479
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	480 def get_libxslt():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	481 """Generate manifest module to install libxslt (needed for lxml)"""
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	482 step_name = 'libxslt'
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	483 print_step("retrieving latest version of libxslt")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	484 cache = get_cache(step_name)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	485 if cache is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	486 return cache
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	487
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	488 ftp = FTP("xmlsoft.org")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	489 ftp.login()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	490 ftp.cwd("libxml2")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	491 libxslt_archives = [l for l in ftp.nlst() if 'xslt' in l
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	492 and l.endswith('tar.gz')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	493 and 'git' not in l
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	494 and 'rc' not in l]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	495 latest_libxslt = libxslt_archives[-1]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	496 print(f"latest libxslt found: {latest_libxslt}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	497
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	498 with tempfile.TemporaryDirectory() as tmp_dirname:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	499 tmp_dir = Path(tmp_dirname)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	500 file_path = tmp_dir / latest_libxslt
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	501 with file_path.open('wb+') as f:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	502 ftp.retrbinary('RETR ' + latest_libxslt, f.write)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	503 f.seek(0)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	504 libxslt_hash = hashlib.sha256(f.read()).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	505
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	506 ftp.quit()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	507
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	508 print(f"libxstl hash: {libxslt_hash}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	509
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	510 data = [{
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	511 "name": "libxslt",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	512 "sources": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	513 {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	514 "type": "archive",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	515 "url": f"ftp://xmlsoft.org/libxml2/{latest_libxslt}",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	516 "sha256": libxslt_hash,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	517 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	518 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	519 }]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	520 set_cache(step_name, data)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	521 return data
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	522
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	523
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	524 def get_python_deps():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	525 """Generate manifest modules for python dependencies of main package"""
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	526 step_name = f'python_deps__{app_id}'
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	527 print_step("retrieving python dependencies")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	528 cache = get_cache(step_name)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	529 if cache is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	530 return cache
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	531
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	532 with tempfile.TemporaryDirectory() as tmp_dirname:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	533 if args.deps_dir is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	534 # we ignore the created temporary directory is we have an other one specified
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	535 tmp_dirname = args.deps_dir
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	536 tmp_dir = Path(tmp_dirname)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	537 env_dir = tmp_dir / 'env'
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	538 pip = env_dir / 'bin' / 'pip'
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	539 download_dir = tmp_dir / 'archives'
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	540 if download_dir.exists() and env_dir.exists():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	541 print("dependencies are already downloaded and packages are already "
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	542 "installed")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	543 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	544 download_dir.mkdir()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	545 print(f"working in temporary directory {tmp_dirname}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	546 venv_cplted = subprocess.run(["/usr/bin/env", "virtualenv2", env_dir])
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	547 venv_cplted.check_returncode()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	548 print("\ndownloading packages")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	549 command_args = [pip, "download", "-d", download_dir,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	550 "--progress-bar", "emoji"]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	551 if args.no_binary:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	552 command_args.extend(["--no-binary", ":all:"])
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	553 command_args.append(main_package_source)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	554 down_cplted = subprocess.run(command_args)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	555 down_cplted.check_returncode()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	556 print("\ninstalling package\n")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	557 # we install package to have a good map of requirements, pypi metadata
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	558 # are incomplete. Packages should be retrieved from pip cache
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	559 inst_cplted = subprocess.run([pip, "install", main_package_source])
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	560 inst_cplted.check_returncode()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	561
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	562 print_step("analyzing python dependencies")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	563 deps_map = {}
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	564
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	565 for archive_path in download_dir.iterdir():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	566 name, right_part = archive_path.name.rsplit('-', 1)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	567 name_canonical = canonical(name)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	568 if right_part.endswith('.tar.gz'):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	569 version = right_part[:-7]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	570 elif right_part.endswith('.tar.bz2'):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	571 version = right_part[:-8]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	572 elif right_part.endswith('.zip'):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	573 version = right_part[:-4]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	574 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	575 raise ValueError(
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	576 f"Archive are supposed to be .tar.gz, .tar.bz2 or .zip archives, but "
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	577 f"file found is {archive_path.name}, did something change on Pypy?")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	578 with open(archive_path, "rb") as f:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	579 dep_hash = hashlib.sha256(f.read()).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	580 r = requests.get(f"https://pypi.org/pypi/{name}/{version}/json")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	581 r.raise_for_status()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	582 dep_json = r.json()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	583 release_json = dep_json["releases"][version]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	584 try:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	585 version_json = next(v for v in release_json
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	586 if v['digests']['sha256'] == dep_hash)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	587 except IndexError:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	588 raise ValueError(f"Can't find the version we downloaded for {name}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	589
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	590 requirements = get_requirements(pip, name_canonical)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	591
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	592 dep = Package(name=name_canonical,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	593 version=version,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	594 hash_=dep_hash,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	595 url=version_json['url'],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	596 requirements=requirements,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	597 )
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	598
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	599 deps_map[name_canonical] = dep
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	600 print(f"found dependency: {dep.name} {dep.version}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	601
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	602 print_step("ordering dependencies")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	603
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	604 requirements = get_requirements(pip, main_package)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	605 main_pack = Package(name=main_package,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	606 version=args.version,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	607 hash_="",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	608 url="",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	609 requirements=requirements,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	610 )
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	611
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	612 deps = []
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	613 print("resolving requirements\n")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	614 resolve_requirements(main_pack, deps_map, deps)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	615 missing_deps = set(deps_map.values()).difference(deps)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	616 if missing_deps:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	617 print("\n/!\\ some dependencies are missing this should not happen! "
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	618 "Adding them\n")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	619 print("additional requirements")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	620 for pack in missing_deps:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	621 resolve_requirements(pack, deps_map, deps)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	622
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	623 # we remove deps already installed with PYTHON_SETUP_REQUIREMENTS
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	624 for name in PYTHON_SETUP_REQUIREMENTS:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	625 package_data = deps_map.get(name)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	626 if package_data is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	627 deps.remove(package_data)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	628 print(f"removed {name} which is already a setup requirement")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	629
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	630 # main package is installed at the end
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	631 deps.remove(main_pack)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	632
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	633 print("\npackages are now ordered: {}".format(", ".join(d.name for d in deps)))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	634
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	635 print("\nwe now generate modules for python dependencies")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	636 data = []
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	637 version_force = {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	638 canonical(n): v
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	639 for n,v in build_settings.get('python_deps_version_force', {}).items()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	640 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	641
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	642 for dep in deps:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	643 version_forced = version_force.get(dep.name)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	644 if version_forced == 'pre-release':
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	645 print(f"using pre-release version for {dep.name} as requested in build "
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	646 "settings")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	647 dep_data = get_python_package(dep.name, step_message='', with_pre=True)[0]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	648 elif version_forced is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	649 print(f"using version {version_forced} for {dep.name} as requested in "
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	650 "build settings")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	651 dep_data = get_python_package(dep.name, step_message='',
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	652 version=version_forced)[0]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	653 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	654 dep_data = {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	655 "name": dep.name,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	656 "buildsystem": "simple",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	657 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	658 "python2 setup.py install --prefix=${FLATPAK_DEST}"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	659 ],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	660 "sources": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	661 {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	662 "type": "archive",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	663 "url": dep.url,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	664 "sha256": dep.hash_,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	665 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	666 ],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	667 "ensure-writable": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	668 "/lib/python2.7/site-packages/easy-install.pth",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	669 "/lib/python2.7/site-packages/setuptools.pth",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	670 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	671 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	672
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	673 if dep.name in PYTHON_DEP_OVERRIDE:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	674 print(f"applying override for {dep.name}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	675 override = PYTHON_DEP_OVERRIDE[dep.name]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	676
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	677 # key suffixed with "_extend" won't replace the data
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	678 for key in list(override.keys()):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	679 if key.endswith('_extend'):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	680 real_key = key[:-7]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	681 extend_data = override.pop(key)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	682 if real_key == 'sources':
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	683 for item_data in extend_data:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	684 # we look for missing hashes and add them
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	685 if (item_data.get('type') in ('file', 'patch')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	686 and 'sha256' not in item_data):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	687 with open(item_data['path'], 'rb') as f:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	688 hash_ = hashlib.sha256(f.read()).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	689 item_data['sha256'] = hash_
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	690
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	691 dep_data.setdefault(real_key, []).extend(extend_data)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	692
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	693 dep_data.update(override)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	694
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	695 data.append(dep_data)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	696
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	697 set_cache(step_name, data)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	698 return data
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	699
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	700
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	701 def cache_from_repos():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	702 """Get and cache locally a repository and returns cache path"""
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	703 print_step("retrieving code from repository")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	704 dev_repos = build_settings['dev_repos']
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	705 repos_type = dev_repos.get('type', 'hg')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	706 if repos_type != 'hg':
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	707 raise NotImplementedError("only Mercurial is managed so far")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	708 url = dev_repos['url']
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	709 dest_path = get_cache_dir() / main_package
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	710 dest_path.mkdir(parents=True, exist_ok=True)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	711 if (dest_path / '.hg').is_dir():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	712 print("code is already in cache")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	713 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	714 try:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	715 print(f"retrieving code from repository {url} into {dest_path}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	716 hg_cplted = subprocess.run(["hg", "clone", url, dest_path])
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	717 hg_cplted.check_returncode()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	718 except Exception as e:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	719 shutil.rmtree(dest_path)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	720 raise e
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	721 return dest_path
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	722
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	723
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	724 def get_repos_module():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	725 """Generate manifest module for the repository"""
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	726 dep_data = {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	727 "name": main_package,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	728 "buildsystem": "simple",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	729 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	730 "python2 setup.py install --prefix=${FLATPAK_DEST}"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	731 ],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	732 "sources": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	733 {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	734 "type": "dir",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	735 "path": str(main_package_source)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	736 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	737 ],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	738 "ensure-writable": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	739 "/lib/python2.7/site-packages/easy-install.pth",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	740 "/lib/python2.7/site-packages/setuptools.pth",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	741 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	742 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	743 return [dep_data]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	744
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	745
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	746 def get_sat_media():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	747 """Generate module for last version of sat_media available on the FTP"""
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	748 step_name = 'sat_media'
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	749 print_step("retrieving latest version of sat_media")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	750 cache = get_cache(step_name)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	751 if cache is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	752 return cache
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	753 url = "https://ftp.goffi.org/sat_media/sat_media.tar.bz2"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	754 r = requests.get(url)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	755 r.raise_for_status()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	756 hash_ = hashlib.sha256(r.content).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	757 dep_data = {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	758 "name": "sat-media",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	759 "buildsystem": "simple",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	760 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	761 "cp -vr . ${FLATPAK_DEST}/share/sat-media"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	762 ],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	763 "sources": [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	764 {
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	765 "type": "archive",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	766 "url": url,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	767 "sha256": hash_,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	768 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	769 ],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	770 }
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	771
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	772 data = [dep_data]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	773 set_cache(step_name, data)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	774 return data
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	775
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	776
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	777 def get_icon():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	778 icon = build_settings.get('icon')
145 9201caf7b2a8 flatpak (build_manifest): don't crash if icon is not specified Goffi <goffi@goffi.org> parents: 140 diff changeset	779 if icon is None:
9201caf7b2a8 flatpak (build_manifest): don't crash if icon is not specified Goffi <goffi@goffi.org> parents: 140 diff changeset	780 return []
9201caf7b2a8 flatpak (build_manifest): don't crash if icon is not specified Goffi <goffi@goffi.org> parents: 140 diff changeset	781 else:
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	782 if isinstance(icon, str):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	783 icon = {'url': icon}
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	784 icon_path = Path(urlparse(icon['url']).path)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	785 suffix = icon_path.suffix[1:]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	786 if suffix not in ('svg', 'png'):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	787 raise ValueError("invalid icon, you must use a SVG or PNG image!")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	788 if 'size' not in icon:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	789 if suffix == 'svg':
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	790 icon['size'] = 'scalable'
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	791 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	792 raise ValueError('icon size is not specified, please add a "size" key')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	793
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	794 dest_path = f"/app/share/icons/hicolor/{icon['size']}/apps/{app_id}.{suffix}"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	795
145 9201caf7b2a8 flatpak (build_manifest): don't crash if icon is not specified Goffi <goffi@goffi.org> parents: 140 diff changeset	796 data = file_from_url(
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	797 url = icon['url'],
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	798 dest = dest_path,
150 10adfa3f9552 flatpak (build_manifest): icon names depends of app_id, so set cache to be dependant of app_id Goffi <goffi@goffi.org> parents: 146 diff changeset	799 step_name = f"icon__{app_id}",
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	800 step_message = "retrieving application icon",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	801 )
145 9201caf7b2a8 flatpak (build_manifest): don't crash if icon is not specified Goffi <goffi@goffi.org> parents: 140 diff changeset	802 data[0]['name'] = 'icon'
9201caf7b2a8 flatpak (build_manifest): don't crash if icon is not specified Goffi <goffi@goffi.org> parents: 140 diff changeset	803 return data
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	804
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	805
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	806 def generate_appdata_from_template(template_file):
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	807 appdata_file = Path(f"{app_id}.appdata.xml")
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	808 if appdata_file.exists() and not args.force:
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	809 confirm = input(OVERWRITE_WARNING.format(appdata_file))
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	810 if confirm != 'y':
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	811 print("manifest building cancelled")
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	812 sys.exit(0)
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	813 parser = etree.XMLParser(remove_blank_text=True)
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	814 tree = etree.parse(template_file, parser)
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	815 root = tree.getroot()
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	816 if args.version == 'dev':
140 e23e414987d7 flatpak (build_manifest): --no-binary is currently disabled as version parsing doesn't handle yet wheel packages. Goffi <goffi@goffi.org> parents: 139 diff changeset	817 print(f"addind release data for dev version ({dev_version_rev})")
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	818 releases_elt = root.find('releases')
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	819 if releases_elt is None:
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	820 raise ValueError(
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	821 "<releases/> element is missing in appdata template, please add it")
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	822 release_elt = etree.SubElement(
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	823 releases_elt,
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	824 "release",
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	825 {'type': 'development',
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	826 'version': dev_version_rev,
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	827 'date': dev_version_date},
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	828 )
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	829 description_elt = etree.SubElement(release_elt, 'description')
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	830 text_lines = APPDATA_RELEASE_DEV_TEXT.strip().split('\n')
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	831 for text in text_lines:
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	832 etree.SubElement(description_elt, 'p').text = text
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	833
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	834 with open(appdata_file, "wb") as f:
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	835 f.write(etree.tostring(root, encoding="utf-8", xml_declaration=True,
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	836 pretty_print=True))
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	837
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	838 return appdata_file.as_posix()
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	839
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	840
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	841 def get_app_metadata():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	842 desktop_file = build_settings.get('desktop_file')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	843 appdata_file = build_settings.get('appdata_file')
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	844 if desktop_file is None and appdata_file is None:
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	845 return
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	846
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	847 print_step("retrieving application metadata")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	848 # we don't use cache here to be sure to have always up-to-date files
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	849
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	850 data = []
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	851
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	852 if desktop_file is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	853 print("generating module for desktop metadata")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	854 data.extend(file_upload(
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	855 filename = desktop_file,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	856 dest=f"/app/share/applications/{app_id}.desktop",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	857 ))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	858
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	859 if appdata_file is not None:
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	860 if appdata_file.startswith('_tpl_'):
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	861 print("found appdata template, we now use it to generate the file")
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	862 appdata_file = generate_appdata_from_template(appdata_file)
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	863 print("generating module for appdata metadata")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	864 data.extend(file_upload(
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	865 filename = appdata_file,
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	866 dest=f"/app/share/metainfo/{app_id}.appdata.xml",
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	867 ))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	868
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	869 return data
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	870
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	871
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	872 ## main_script ##
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	873
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	874
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	875 if __name__ == '__main__':
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	876
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	877 args = parse_args()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	878 title = args.name.title()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	879 app_id = f"org.salutatoi.{title}"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	880 package_file = Path(f"{app_id}.json")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	881
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	882 print(f"generating manifest for {app_id} ({args.version})")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	883
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	884 if package_file.exists() and not args.force:
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	885 confirm = input(OVERWRITE_WARNING.format(package_file))
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	886 if confirm != 'y':
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	887 print("manifest building cancelled")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	888 sys.exit(0)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	889
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	890 tpl_file = Path(f"_tpl_{package_file}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	891
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	892 if not tpl_file.exists():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	893 raise RuntimeError(f"Can't find template {tpl_file}, it is mandatory to build"
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	894 "the manifest!")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	895
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	896 with tpl_file.open() as f:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	897 template = json.load(f)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	898
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	899 build_settings = template.pop(SETTINGS_KEY, {})
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	900 if "setup_requirements" in build_settings:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	901 PYTHON_SETUP_REQUIREMENTS.extend(build_settings["setup_requirements"])
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	902 main_package = canonical(build_settings.get('package', args.name))
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	903 if args.version == 'dev':
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	904 if 'dev_repos' not in build_settings:
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	905 raise NotImplementedError(
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	906 "dev version can currently only be handled with a dev repostiory "
d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	907 "(dev_repos)")
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	908 dev_repos = build_settings['dev_repos']
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	909 main_package_source = cache_from_repos()
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	910 dev_version_rev, dev_version_date = get_hg_id_date(main_package_source)
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	911 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	912 main_package_source = main_package
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	913
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	914 manifest = {}
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	915 manifest['app-id'] = app_id
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	916 manifest['default-branch'] = args.version
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	917 # we update DEFAULT_MANIFEST only now to have "app-id" and "default-branch" on the top
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	918 # of the manifest
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	919 manifest.update(DEFAULT_MANIFEST)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	920 manifest.update(template)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	921
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	922 modules.extend(get_libxslt())
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	923
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	924 # setup requirements
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	925 modules.extend(get_python_package(
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	926 PYTHON_SETUP_REQUIREMENTS,
146 64c02902b86a flatpak (build_manifest): fixed setup_requirements step cache by making dependent of app_id Goffi <goffi@goffi.org> parents: 145 diff changeset	927 step_name=f"setup_requirements__{app_id}",
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	928 step_message="generating modules for setup requirements")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	929 )
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	930
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	931 # python dependencies
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	932 modules.extend(get_python_deps())
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	933
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	934 # at this point we add things specified in the template
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	935 modules.extend(manifest.get('modules', []))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	936 manifest['modules'] = modules
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	937
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	938 # sat common things
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	939 existing_modules = {d['name'] for d in modules}
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	940 if "sat_templates" not in existing_modules:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	941 modules.extend(get_python_package("sat_templates", with_pre=True))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	942 modules.extend(get_sat_media())
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	943 modules.extend(file_upload('sat.conf', '/app'))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	944
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	945 # wrapper to launch the backend if needed
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	946 wrapped_command = build_settings.get('wrapped_command')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	947 if wrapped_command:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	948 modules.extend(file_upload('sat_wrapper', src='sat_wrapper.py',
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	949 replace={'##COMMAND##': wrapped_command},
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	950 use_install=True))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	951
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	952 # icon
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	953 modules.extend(get_icon())
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	954
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	955 # desktop file and appdata file
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	956 modules.extend(get_app_metadata())
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	957
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	958 # now the app itself
139 d36a68e396d5 flatpak: set appdata release version for dev version: Goffi <goffi@goffi.org> parents: 138 diff changeset	959 if args.version == 'dev':
138 274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	960 # mercurial is needed for dev version to install but also to
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	961 # retrieve revision used
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	962 modules.extend(get_python_package("mercurial"))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	963 modules.extend(get_repos_module())
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	964 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	965 modules.extend(get_python_package(main_package, version=args.version))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	966
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	967 print_step("writing manifest")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	968 with package_file.open('w') as f:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	969 json.dump(manifest, f, indent=4)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	970
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	971 if args.ignore_cache:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	972 print("/!\\ those --ignore-cache arguments don't correspond to any step: {}".
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	973 format(', '.join(args.ignore_cache)))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	974
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	975 print(f"manifest generated successfully at {package_file}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	976
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	977 if args.export is not None:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	978 print_step("exporting building files")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	979 print(f"exporting to {args.export}\n")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	980 to_export = [
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	981 ("package file", package_file),
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	982 ]
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	983
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	984 for m in modules:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	985 for s in m.get('sources', []):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	986 s_type = s.get('type')
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	987 if s_type in ('path', 'file'):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	988 try:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	989 path = s['path']
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	990 except KeyError:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	991 if 'url' in s:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	992 continue
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	993 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	994 raise ValueError(f"path missing for module source:\n{s}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	995 to_export.append((s_type, Path(path)))
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	996
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	997 for label, path in to_export:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	998 print(f"exporting {label}: {path}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	999 dest = args.export / path
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1000 if dest.exists():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1001 if args.force_export:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1002 print(f" replacing existing {dest}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1003 if path.is_dir():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1004 shutil.rmtree(dest)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1005 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1006 dest.unlink()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1007 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1008 print(" it already exists, skipping")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1009 continue
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1010 if args.symlink == 'all':
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1011 os.symlink(path, dest)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1012 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1013 if path.is_dir():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1014 shutil.copytree(path, dest)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1015 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1016 shutil.copyfile(path, dest)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1017
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1018 if args.version == 'dev':
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1019 print("exporting cache")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1020 dest = args.export / get_cache_dir()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1021 if args.force_export and os.path.lexists(dest):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1022 print(f" replacing existing {dest}")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1023 if dest.is_symlink():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1024 dest.unlink()
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1025 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1026 shutil.rmtree(dest)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1027
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1028 if dest.exists():
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1029 print(" it already exists, skipping")
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1030 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1031 if args.symlink in ('all', 'cache'):
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1032 os.symlink(get_cache_dir().resolve(), dest)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1033 else:
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1034 shutil.copytree(get_cache_dir(), dest)
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1035
274af514a5cf flatpak: reworked packages + made a building script: Goffi <goffi@goffi.org> parents: diff changeset	1036 print("\nexport completed")

Mercurial > sat_docs

annotate flatpak/build_manifest.py @ 151:6f8c849daf2a