Mercurial > sat_docs

annotate flatpak/build_manifest.py @ 138:274af514a5cf

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
flatpak: reworked packages + made a building script: a new `build_manifest.py` script can now be used to generate flatpak manifests for every frontend of SàT. The manifest can be used both for development versions and stable ones. Templates files (in the form `_tmp_<app-id>.json`) are used to set building instructions. A common runtime specific to SàT has been abandoned following a discussion on the official mailing list. A small wrapper is now used to launch backend automatically if it's not found. Desktop and app metadata have been added for Cagou. Jp and Primitivus don't have appdata and desktop files yet.
author Goffi <goffi@goffi.org>
date Sat, 22 Jun 2019 15:59:07 +0200
parents
children d36a68e396d5
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
138
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
1 #!/usr/bin/env python3
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
2
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
3 import tempfile
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
4 import subprocess
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
5 from pathlib import Path
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 from typing import List
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
7 from dataclasses import dataclass
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
8 import hashlib
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 from ftplib import FTP
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
10 from urllib.parse import urlparse
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
11 import sys
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 import os
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 import json
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 import time
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
15 import argparse
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 import shutil
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 from packaging.version import parse as parse_version
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
18 import requests
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
19
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
20
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
21 CACHE_LIMIT = 3600 * 24
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
22 PYTHON_DEP_OVERRIDE = {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
23 "lxml": {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
24 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
25 "python2 ./setup.py install --prefix=${FLATPAK_DEST} --optimize=1"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
26 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
27 "build-options": {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
28 "env": {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
29 "XSLT_CONFIG": "pkg-config libxslt"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
30 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
31 },
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
32 },
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
33 "dbus-python": {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
34 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
35 "python2 setup.py build",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
36 "python2 setup.py install --prefix=${FLATPAK_DEST}",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
37 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
38 },
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
39 "urwid": {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
40 "sources_extend": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
41 {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
42 "type": "patch",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
43 "path": "main_loop.patch"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
44 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
45 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
46 },
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
47 "kivy": {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
48 "post-install": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
49 # this file is not copied normally when installing with
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
50 # `python2 setup.py install`.
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
51 # TODO: report upstream
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
52 "cp kivy/setupconfig.py /app/lib/python2.7/site-packages/Kivy-*.egg/kivy/"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
53 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
54 },
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
55 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
56 PYTHON_DEP_REQUIREMENTS_UPDATE = {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
57 # service-identity is not seen as a twisted requirement, so it's sometimes misplaced
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
58 'twisted': ['service-identity'],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
59 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
60 PYTHON_SETUP_REQUIREMENTS = [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
61 'setuptools', # to have an up-to-date version
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
62 'setuptools_scm',
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
63 'docutils', # needed my m2r
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
64 'mistune', # needed my m2r
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
65 'm2r', # needed by automat
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
66 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
67 DEFAULT_MANIFEST = {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
68 "runtime": "org.freedesktop.Platform",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
69 "runtime-version": "1.6",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
70 "sdk": "org.freedesktop.Sdk",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
71 "sdk-extensions": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
72 "org.freedesktop.Sdk.Debug",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
73 "org.freedesktop.Sdk.Locale",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
74 "org.freedesktop.Sdk.Docs"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
75 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
76 "platform-extensions": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
77 "org.freedesktop.Platform.Locale"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
78 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
79 "command": "sat_wrapper",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
80 "finish-args": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
81 "--socket=session-bus",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
82 "--share=network",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
83 "--filesystem=home"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
84 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
85 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
86 SHOW_REQUIRES_HEADER = 'Requires: '
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
87 SETTINGS_KEY = '_build_settings'
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
88
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
89
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
90 @dataclass
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
91 class Package:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
92 name: str
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
93 version: str
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
94 hash_: str
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
95 url: str
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
96 requirements: List[str]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
97
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
98 def __hash__(self):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
99 return hash(self.name)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
100
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
101
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
102 def print_step(step):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
103 print()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
104 print("┌" + "─" * (len(step) + 2) + "┐")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
105 print("│ {} │".format(step))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
106 print("└" + "─" * (len(step) + 2) + "┘")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
107 print()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
108
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
109
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
110 def parse_args():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
111 parser = argparse.ArgumentParser(
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
112 description='Flatpak manifest builder for Salut à Toi')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
113
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
114 build_group = parser.add_argument_group('building', 'options used to generate the manifest')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
115 export_group = parser.add_argument_group('export', 'otions used to building files')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
116
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
117 # build group
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
118 build_group.add_argument('-f', '--force', action="store_true",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
119 help="force overwritting of existing manifest")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
120 build_group.add_argument('--ignore-cache', action='append', default=[],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
121 help='ignore the cache of this step ("all" to ignore all caches)')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
122 build_group.add_argument(
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
123 '--deps-dir',
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
124 help="use this directory to build_group python dependencies (it won't be deleted at "
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
125 "the end, useful when you want to re-use it and avoir re-downloading)")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
126 build_group.add_argument('--no-binary', help="don't use binary packages")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
127
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
128 # export group
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
129 export_group.add_argument('-s', '--symlink', choices=['no', 'all', 'cache'],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
130 default='no',
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
131 help='"no" to copy all files, "all" to symlink all files, "cache" to copy files '
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
132 'and symlink only cache (default: "no")')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
133 export_group.add_argument('-F', '--force-export', action='store_true',
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
134 help='force overwritting of existing files/symlinks when exporting (DEFAULT: '
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
135 'existing files are skipped)')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
136 export_group.add_argument('-e', '--export', type=Path,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
137 help="export build files to this directory (DEFAULT: don't export files)")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
138
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
139 # common
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
140 parser.add_argument('name', type=str, help="name of the package to build_group")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
141 parser.add_argument('version', type=str, help="version of the package to build_group")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
142
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
143 args = parser.parse_args()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
144
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
145 if 'all' in args.ignore_cache and len(args.ignore_cache) != 1:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
146 parser.error('"all" cannot be used with step names in --ignore-cache, '
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
147 'use it alone')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
148
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
149 if args.export is not None and not args.export.is_dir():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
150 parser.error(f"{args.export} is not a directory!")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
151
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
152 return args
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
153
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
154
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
155 ## useful methods ##
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
156
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
157 def get_cache(name):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
158 """Retrieve cache for a step
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
159
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
160 cache will be ignored if too old, or explicitly ignored by user
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
161 @param name(str): name of the step
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
162 @return (object): cached data
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
163 """
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
164 # name of the cache without prefix, mainly used for python_deps
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
165 shortname = name.split('__', 1)[0]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
166
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
167 if shortname in args.ignore_cache or 'all' in args.ignore_cache:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
168 print(f"ignoring cache for {shortname}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
169 args.ignore_cache.remove(shortname)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
170 return None
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
171 try:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
172 cache_name = cache[name]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
173 cache_updated = cache_name['updated']
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
174 except KeyError:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
175 return None
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
176 if time.time() - cache_updated > CACHE_LIMIT:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
177 print(f"cache for {name} is too old, we won't use it")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
178 return None
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
179 print(f"cache found for {shortname}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
180 return cache_name['data']
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
181
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
182
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
183 def set_cache(name, data):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
184 """Update cache for a step
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
185
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
186 @param name(str): name of the step
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
187 @param data(object): data to cache
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
188 """
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
189 cache_name = cache.setdefault(name, {})
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
190 cache_name['updated'] = time.time()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
191 cache_name['data'] = data
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
192 with cache_path.open('w') as f:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
193 json.dump(cache, f)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
194
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
195
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
196 def get_python_package(package_name, step_name=None, step_message=None, with_pre=False,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
197 version=None):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
198 """Generate module for a Python package
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
199
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
200 @param package_name(str, list[str]): name of the Python package
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
201 use list of str to get several packages at once
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
202 @param step_name(None, str): name of the step, None to use package_name
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
203 @param step_message(None, str): message of the step (None to use default one)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
204 use empty string to disable it
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
205 @param with_pre(bool): if True retrieve also pre-releases
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
206 @param version(str, None): indicate the version of the package to download
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
207 if None, the most recent version compativle with `with_pre` will be used
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
208 """
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
209 single = isinstance(package_name, str)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
210 if step_name is None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
211 step_name = package_name if single else ' - '.join(package_name)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
212 if step_message != "":
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
213 print_step(step_message or f"retrieving latest version of {package_name}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
214 cache = get_cache(step_name)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
215 if cache is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
216 return cache
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
217 package_names = [package_name] if single else package_name
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
218 data = []
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
219 for name in package_names:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
220 r = requests.get(f"https://pypi.org/pypi/{name}/json")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
221 r.raise_for_status()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
222
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
223 releases_data = []
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
224 for version_raw, release_data in r.json()["releases"].items():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
225 if version is not None and version != version_raw:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
226 continue
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
227 release_version = parse_version(version_raw)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
228 if release_version.is_prerelease and not with_pre:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
229 continue
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
230 releases_data.append((release_version, release_data))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
231
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
232 # we sort releases by version to be sure to have latests one
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
233 releases_data.sort(key=lambda d: d[0])
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
234
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
235 try:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
236 release_version, release_data = releases_data[-1]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
237 package = next(r for r in release_data if r["packagetype"] == "sdist")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
238 except (IndexError, StopIteration):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
239 raise RuntimeError(f"Can't find a matching package for {name}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
240
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
241 print(f"{name} {release_version} will be used")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
242 dep_data = {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
243 "name": name,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
244 "buildsystem": "simple",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
245 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
246 "python2 setup.py install --prefix=${FLATPAK_DEST}"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
247 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
248 "sources": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
249 {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
250 "type": "archive",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
251 "url": package["url"],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
252 "sha256": package["digests"]["sha256"],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
253 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
254 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
255 "ensure-writable": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
256 "/lib/python2.7/site-packages/easy-install.pth",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
257 "/lib/python2.7/site-packages/setuptools.pth",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
258 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
259 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
260 data.append(dep_data)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
261
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
262 set_cache(step_name, data)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
263 return data
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
264
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
265
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
266 def file_upload(filename, dest="/app/bin", src=None, replace=None, use_install=False):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
267 """Generate manifest modules to upload a local file
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
268
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
269 @param filename(str): name of the local file to upload
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
270 Note that it will be overwritted if replace is used
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
271 @param dest(str, Path): path where the file must be installed in the container
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
272 @param src(str, None): when replace is used, name of the source file
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
273 @param replace(dict, None): mapping of text to replace
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
274 Use when file must be dynamicly adapted to the container, note that src must be
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
275 set to the name of the template file if this is used.
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
276 e.g. {'##COMMAND##', 'cagou'} will replace all '##COMMAND##' in filename
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
277 by 'cagou'
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
278 @param use_install(bool): if True, install file with `install src dest`
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
279 else, use `mv src dest`
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
280 @return (list(dict)): modules to add to the manifest
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
281 """
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
282 assert isinstance(filename, str) and '/' not in filename
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
283 print_step(f"generating {filename} module")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
284 # no cache for this one, to be sure to have always latest version
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
285 filename = Path(filename)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
286 if src is None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
287 file_to_test = filename
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
288 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
289 src = Path(src)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
290 file_to_test = src
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
291 if not file_to_test.exists():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
292 raise RuntimeError(
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
293 f"{file_to_test} is missing, it is needed to build the manifest!")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
294
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
295 if replace:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
296 if src is None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
297 raise ValueError(
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
298 '"src" must be set to name of template file if replace is used')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
299 print(f'doing replacement in template "{src}" to dest "{filename}"')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
300 # there are strings to replace, we read file, do the changes and dump the
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
301 # result in <filename>
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
302 with open(src) as f:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
303 buff = f.read()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
304
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
305 for old, new in replace.items():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
306 buff = buff.replace(old, new)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
307
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
308 with open(filename, 'w') as f:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
309 f.write(buff)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
310 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
311 if src is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
312 raise ValueError('"src" must not be used if replace is not used')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
313
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
314 with filename.open('rb') as f:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
315 hash_ = hashlib.sha256(f.read()).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
316
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
317 dest = Path(dest)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
318
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
319 dep_data = {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
320 "name": str(filename),
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
321 "buildsystem": "simple",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
322 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
323 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
324 "sources": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
325 {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
326 "type": "file",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
327 "path": str(filename),
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
328 "sha256": hash_,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
329 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
330 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
331 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
332
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
333 build_commands = dep_data['build-commands']
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
334 if use_install:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
335 build_commands.append(f"install -Dv {filename} {dest}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
336 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
337 if dest.as_posix() not in ('/app', '/app/bin'):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
338 # just in case the destination directory doesn't exist
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
339 build_commands.append(f"mkdir -p {dest.parent}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
340 build_commands.append(f"mv -v {filename} {dest}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
341
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
342 return [dep_data]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
343
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
344
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
345 def file_from_url(url, dest=None, step_name=None, step_message=None):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
346 """Generate manifest modules for a file either on a http(s) url or local
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
347
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
348 @param url(str): url of the file to use, or local path
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
349 if it starts with http, url will be used, else file_upload(url) will be used
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
350 @param dest(str, None): path were the file should be copied
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
351 """
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
352 parsed = urlparse(url)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
353 if not parsed.scheme:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
354 return file_upload(url)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
355 if not parsed.scheme.startswith('http'):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
356 raise NotImplementedError (
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
357 f'we can\'t use the URL "{url}", the scheme "{parsed.scheme}" is not managed')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
358
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
359 filepath = Path(parsed.path)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
360 stem = filepath.stem
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
361
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
362 if step_name is None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
363 # we use this name to easily ignore cache (with stem) while avoiding
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
364 # conflict if we have 2 URLs with the same stem
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
365 step_name = f"{stem}__{url}"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
366
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
367 if step_message is None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
368 step_messate = f"generating module for {stem}"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
369
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
370 print_step(step_message)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
371 cache = get_cache(step_name)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
372 if cache is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
373 return cache
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
374
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
375 r = requests.get(url)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
376 r.raise_for_status()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
377 file_hash = hashlib.sha256(r.content).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
378
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
379 dep_data = {"name": stem}
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
380
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
381 if dest is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
382 dest = Path(dest)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
383 dep_data.update({
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
384 'buildsystem': 'simple',
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
385 'build-commands':[
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
386 f'mkdir -p {dest.parent}',
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
387 f'mv "{filepath.name}" "{dest}"',
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
388 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
389 })
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
390
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
391 dep_data['sources'] = [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
392 {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
393 "type": "file",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
394 "url": url,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
395 "sha256": file_hash,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
396 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
397 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
398
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
399 data = [dep_data]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
400 set_cache(step_name, data)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
401 return data
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
402
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
403
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
404 def get_requirements(pip, package_name):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
405 """Retrieve requirements for an installed python package
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
406
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
407 @param pip(Path): path to pip executable to use
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
408 package must be installed in the environment of this pip
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
409 @param package_name(str): name of the package to retrieve
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
410 @return(list[str]): found requirements
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
411 """
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
412 show_cplted = subprocess.run([pip, "show", package_name],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
413 capture_output=True, text=True)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
414 show_cplted.check_returncode()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
415
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
416 lines = show_cplted.stdout.split('\n')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
417 requirement_raw = next(l for l in lines if l.startswith(SHOW_REQUIRES_HEADER))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
418 requirement_raw = requirement_raw[len(SHOW_REQUIRES_HEADER):]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
419 requirements = [canonical(p) for p in requirement_raw.split(',') if p.strip()]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
420 requirements_update = [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
421 canonical(r) for r in PYTHON_DEP_REQUIREMENTS_UPDATE.get(package_name, [])]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
422 new_requirements = set(requirements_update).difference(requirements)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
423 if new_requirements:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
424 print("adding extra requirements to {}: {}".format(
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
425 package_name, ', '.join(new_requirements)))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
426 requirements.extend(new_requirements)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
427 return requirements
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
428
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
429
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
430 def resolve_requirements(package, deps_map, deps, indent=0):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
431 """Recursively resolve requirements
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
432
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
433 @param package(Package): retrieve dependencies of this package
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
434 @param deps_map(dict): map from dependency name to Package instance
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
435 @param deps(list[package]): ordered dependencies
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
436 this list is updated in place
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
437 @param indent(int): use internally to print dependencies tree
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
438 """
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
439 if package in deps:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
440 return
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
441 print(" " * indent + package.name)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
442 reqs_data = [deps_map[r] for r in package.requirements]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
443
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
444 for data in reqs_data:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
445 resolve_requirements(data, deps_map, deps, indent+1)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
446
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
447 deps.append(package)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
448
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
449
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
450 def get_cache_dir():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
451 """Return path to directory to use for cache"""
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
452 return Path(f"cache_{app_id}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
453
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
454
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
455 def canonical(name):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
456 """Get canonical name of a package"""
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
457 return name.lower().replace('_', '-').strip()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
458
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
459
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
460 modules = []
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
461 cache_path = Path.home() / Path('.cache/sat_flatpak_cache.json')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
462 if not cache_path.exists():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
463 cache = {}
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
464 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
465 with cache_path.open() as f:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
466 cache = json.load(f)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
467
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
468
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
469 ## steps ##
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
470
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
471 def get_libxslt():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
472 """Generate manifest module to install libxslt (needed for lxml)"""
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
473 step_name = 'libxslt'
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
474 print_step("retrieving latest version of libxslt")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
475 cache = get_cache(step_name)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
476 if cache is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
477 return cache
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
478
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
479 ftp = FTP("xmlsoft.org")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
480 ftp.login()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
481 ftp.cwd("libxml2")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
482 libxslt_archives = [l for l in ftp.nlst() if 'xslt' in l
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
483 and l.endswith('tar.gz')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
484 and 'git' not in l
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
485 and 'rc' not in l]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
486 latest_libxslt = libxslt_archives[-1]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
487 print(f"latest libxslt found: {latest_libxslt}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
488
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
489 with tempfile.TemporaryDirectory() as tmp_dirname:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
490 tmp_dir = Path(tmp_dirname)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
491 file_path = tmp_dir / latest_libxslt
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
492 with file_path.open('wb+') as f:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
493 ftp.retrbinary('RETR ' + latest_libxslt, f.write)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
494 f.seek(0)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
495 libxslt_hash = hashlib.sha256(f.read()).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
496
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
497 ftp.quit()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
498
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
499 print(f"libxstl hash: {libxslt_hash}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
500
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
501 data = [{
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
502 "name": "libxslt",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
503 "sources": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
504 {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
505 "type": "archive",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
506 "url": f"ftp://xmlsoft.org/libxml2/{latest_libxslt}",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
507 "sha256": libxslt_hash,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
508 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
509 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
510 }]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
511 set_cache(step_name, data)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
512 return data
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
513
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
514
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
515 def get_python_deps():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
516 """Generate manifest modules for python dependencies of main package"""
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
517 step_name = f'python_deps__{app_id}'
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
518 print_step("retrieving python dependencies")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
519 cache = get_cache(step_name)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
520 if cache is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
521 return cache
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
522
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
523 with tempfile.TemporaryDirectory() as tmp_dirname:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
524 if args.deps_dir is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
525 # we ignore the created temporary directory is we have an other one specified
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
526 tmp_dirname = args.deps_dir
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
527 tmp_dir = Path(tmp_dirname)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
528 env_dir = tmp_dir / 'env'
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
529 pip = env_dir / 'bin' / 'pip'
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
530 download_dir = tmp_dir / 'archives'
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
531 if download_dir.exists() and env_dir.exists():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
532 print("dependencies are already downloaded and packages are already "
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
533 "installed")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
534 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
535 download_dir.mkdir()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
536 print(f"working in temporary directory {tmp_dirname}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
537 venv_cplted = subprocess.run(["/usr/bin/env", "virtualenv2", env_dir])
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
538 venv_cplted.check_returncode()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
539 print("\ndownloading packages")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
540 command_args = [pip, "download", "-d", download_dir,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
541 "--progress-bar", "emoji"]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
542 if args.no_binary:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
543 command_args.extend(["--no-binary", ":all:"])
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
544 command_args.append(main_package_source)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
545 down_cplted = subprocess.run(command_args)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
546 down_cplted.check_returncode()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
547 print("\ninstalling package\n")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
548 # we install package to have a good map of requirements, pypi metadata
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
549 # are incomplete. Packages should be retrieved from pip cache
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
550 inst_cplted = subprocess.run([pip, "install", main_package_source])
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
551 inst_cplted.check_returncode()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
552
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
553 print_step("analyzing python dependencies")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
554 deps_map = {}
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
555
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
556 for archive_path in download_dir.iterdir():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
557 name, right_part = archive_path.name.rsplit('-', 1)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
558 name_canonical = canonical(name)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
559 if right_part.endswith('.tar.gz'):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
560 version = right_part[:-7]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
561 elif right_part.endswith('.tar.bz2'):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
562 version = right_part[:-8]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
563 elif right_part.endswith('.zip'):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
564 version = right_part[:-4]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
565 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
566 raise ValueError(
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
567 f"Archive are supposed to be .tar.gz, .tar.bz2 or .zip archives, but "
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
568 f"file found is {archive_path.name}, did something change on Pypy?")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
569 with open(archive_path, "rb") as f:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
570 dep_hash = hashlib.sha256(f.read()).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
571 r = requests.get(f"https://pypi.org/pypi/{name}/{version}/json")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
572 r.raise_for_status()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
573 dep_json = r.json()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
574 release_json = dep_json["releases"][version]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
575 try:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
576 version_json = next(v for v in release_json
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
577 if v['digests']['sha256'] == dep_hash)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
578 except IndexError:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
579 raise ValueError(f"Can't find the version we downloaded for {name}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
580
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
581 requirements = get_requirements(pip, name_canonical)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
582
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
583 dep = Package(name=name_canonical,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
584 version=version,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
585 hash_=dep_hash,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
586 url=version_json['url'],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
587 requirements=requirements,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
588 # extra_requirements=extra_requirements,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
589 )
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
590
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
591 deps_map[name_canonical] = dep
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
592 print(f"found dependency: {dep.name} {dep.version}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
593
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
594 print_step("ordering dependencies")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
595
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
596 requirements = get_requirements(pip, main_package)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
597 main_pack = Package(name=main_package,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
598 version=args.version,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
599 hash_="",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
600 url="",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
601 requirements=requirements,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
602 )
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
603
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
604 deps = []
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
605 print("resolving requirements\n")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
606 resolve_requirements(main_pack, deps_map, deps)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
607 missing_deps = set(deps_map.values()).difference(deps)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
608 if missing_deps:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
609 print("\n/!\\ some dependencies are missing this should not happen! "
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
610 "Adding them\n")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
611 print("additional requirements")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
612 for pack in missing_deps:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
613 resolve_requirements(pack, deps_map, deps)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
614
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
615 # we remove deps already installed with PYTHON_SETUP_REQUIREMENTS
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
616 for name in PYTHON_SETUP_REQUIREMENTS:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
617 package_data = deps_map.get(name)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
618 if package_data is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
619 deps.remove(package_data)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
620 print(f"removed {name} which is already a setup requirement")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
621
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
622 # main package is installed at the end
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
623 deps.remove(main_pack)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
624
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
625 print("\npackages are now ordered: {}".format(", ".join(d.name for d in deps)))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
626
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
627 print("\nwe now generate modules for python dependencies")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
628 data = []
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
629 version_force = {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
630 canonical(n): v
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
631 for n,v in build_settings.get('python_deps_version_force', {}).items()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
632 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
633
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
634 for dep in deps:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
635 version_forced = version_force.get(dep.name)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
636 if version_forced == 'pre-release':
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
637 print(f"using pre-release version for {dep.name} as requested in build "
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
638 "settings")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
639 dep_data = get_python_package(dep.name, step_message='', with_pre=True)[0]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
640 elif version_forced is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
641 print(f"using version {version_forced} for {dep.name} as requested in "
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
642 "build settings")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
643 dep_data = get_python_package(dep.name, step_message='',
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
644 version=version_forced)[0]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
645 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
646 dep_data = {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
647 "name": dep.name,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
648 "buildsystem": "simple",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
649 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
650 "python2 setup.py install --prefix=${FLATPAK_DEST}"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
651 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
652 "sources": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
653 {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
654 "type": "archive",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
655 "url": dep.url,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
656 "sha256": dep.hash_,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
657 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
658 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
659 "ensure-writable": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
660 "/lib/python2.7/site-packages/easy-install.pth",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
661 "/lib/python2.7/site-packages/setuptools.pth",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
662 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
663 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
664
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
665 if dep.name in PYTHON_DEP_OVERRIDE:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
666 print(f"applying override for {dep.name}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
667 override = PYTHON_DEP_OVERRIDE[dep.name]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
668
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
669 # key suffixed with "_extend" won't replace the data
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
670 for key in list(override.keys()):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
671 if key.endswith('_extend'):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
672 real_key = key[:-7]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
673 extend_data = override.pop(key)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
674 if real_key == 'sources':
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
675 for item_data in extend_data:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
676 # we look for missing hashes and add them
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
677 if (item_data.get('type') in ('file', 'patch')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
678 and 'sha256' not in item_data):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
679 with open(item_data['path'], 'rb') as f:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
680 hash_ = hashlib.sha256(f.read()).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
681 item_data['sha256'] = hash_
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
682
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
683 dep_data.setdefault(real_key, []).extend(extend_data)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
684
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
685 dep_data.update(override)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
686
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
687 data.append(dep_data)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
688
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
689 set_cache(step_name, data)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
690 return data
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
691
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
692
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
693 def cache_from_repos():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
694 """Get and cache locally a repository and returns cache path"""
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
695 print_step("retrieving code from repository")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
696 dev_repos = build_settings['dev_repos']
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
697 repos_type = dev_repos.get('type', 'hg')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
698 if repos_type != 'hg':
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
699 raise NotImplementedError("only Mercurial is managed so far")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
700 url = dev_repos['url']
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
701 dest_path = get_cache_dir() / main_package
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
702 dest_path.mkdir(parents=True, exist_ok=True)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
703 if (dest_path / '.hg').is_dir():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
704 print("code is already in cache")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
705 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
706 try:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
707 print(f"retrieving code from repository {url} into {dest_path}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
708 hg_cplted = subprocess.run(["hg", "clone", url, dest_path])
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
709 hg_cplted.check_returncode()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
710 except Exception as e:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
711 shutil.rmtree(dest_path)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
712 raise e
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
713 return dest_path
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
714
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
715
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
716 def get_repos_module():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
717 """Generate manifest module for the repository"""
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
718 dep_data = {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
719 "name": main_package,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
720 "buildsystem": "simple",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
721 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
722 "python2 setup.py install --prefix=${FLATPAK_DEST}"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
723 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
724 "sources": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
725 {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
726 "type": "dir",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
727 "path": str(main_package_source)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
728 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
729 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
730 "ensure-writable": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
731 "/lib/python2.7/site-packages/easy-install.pth",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
732 "/lib/python2.7/site-packages/setuptools.pth",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
733 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
734 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
735 return [dep_data]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
736
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
737
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
738 def get_sat_media():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
739 """Generate module for last version of sat_media available on the FTP"""
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
740 step_name = 'sat_media'
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
741 print_step("retrieving latest version of sat_media")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
742 cache = get_cache(step_name)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
743 if cache is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
744 return cache
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
745 url = "https://ftp.goffi.org/sat_media/sat_media.tar.bz2"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
746 r = requests.get(url)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
747 r.raise_for_status()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
748 hash_ = hashlib.sha256(r.content).hexdigest()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
749 dep_data = {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
750 "name": "sat-media",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
751 "buildsystem": "simple",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
752 "build-commands": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
753 "cp -vr . ${FLATPAK_DEST}/share/sat-media"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
754 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
755 "sources": [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
756 {
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
757 "type": "archive",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
758 "url": url,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
759 "sha256": hash_,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
760 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
761 ],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
762 }
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
763
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
764 data = [dep_data]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
765 set_cache(step_name, data)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
766 return data
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
767
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
768
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
769 def get_icon():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
770 icon = build_settings.get('icon')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
771 if icon is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
772 if isinstance(icon, str):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
773 icon = {'url': icon}
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
774 icon_path = Path(urlparse(icon['url']).path)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
775 suffix = icon_path.suffix[1:]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
776 if suffix not in ('svg', 'png'):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
777 raise ValueError("invalid icon, you must use a SVG or PNG image!")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
778 if 'size' not in icon:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
779 if suffix == 'svg':
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
780 icon['size'] = 'scalable'
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
781 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
782 raise ValueError('icon size is not specified, please add a "size" key')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
783
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
784 dest_path = f"/app/share/icons/hicolor/{icon['size']}/apps/{app_id}.{suffix}"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
785
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
786 return file_from_url(
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
787 url = icon['url'],
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
788 dest = dest_path,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
789 # we have common cache if several manifest use the same icon URL
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
790 step_name = f"icon__{icon}",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
791 step_message = "retrieving application icon",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
792 )
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
793
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
794
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
795 def get_app_metadata():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
796 desktop_file = build_settings.get('desktop_file')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
797 appdata_file = build_settings.get('appdata_file')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
798 if desktop_file is None and app_data_file is None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
799 return
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
800
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
801 print_step("retrieving application metadata")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
802 # we don't use cache here to be sure to have always up-to-date files
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
803
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
804 data = []
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
805
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
806 if desktop_file is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
807 print("generating module for desktop metadata")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
808 data.extend(file_upload(
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
809 filename = desktop_file,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
810 dest=f"/app/share/applications/{app_id}.desktop",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
811 ))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
812
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
813 if appdata_file is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
814 print("generating module for appdata metadata")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
815 data.extend(file_upload(
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
816 filename = appdata_file,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
817 dest=f"/app/share/metainfo/{app_id}.appdata.xml",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
818 ))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
819
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
820 return data
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
821
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
822
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
823 ## main_script ##
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
824
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
825
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
826 if __name__ == '__main__':
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
827
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
828 args = parse_args()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
829 title = args.name.title()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
830 app_id = f"org.salutatoi.{title}"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
831 package_file = Path(f"{app_id}.json")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
832
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
833 print(f"generating manifest for {app_id} ({args.version})")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
834
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
835 if package_file.exists() and not args.force:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
836 confirm = input(
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
837 f"{package_file} already exists, do you want to overwritte it (y/N)? ")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
838 if confirm != 'y':
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
839 print("manifest building cancelled")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
840 sys.exit(0)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
841
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
842 tpl_file = Path(f"_tpl_{package_file}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
843
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
844 if not tpl_file.exists():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
845 raise RuntimeError(f"Can't find template {tpl_file}, it is mandatory to build"
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
846 "the manifest!")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
847
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
848 with tpl_file.open() as f:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
849 template = json.load(f)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
850
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
851 build_settings = template.pop(SETTINGS_KEY, {})
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
852 if "setup_requirements" in build_settings:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
853 PYTHON_SETUP_REQUIREMENTS.extend(build_settings["setup_requirements"])
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
854 main_package = canonical(build_settings.get('package', args.name))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
855 if args.version == 'dev' and 'dev_repos' in build_settings:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
856 dev_repos = build_settings['dev_repos']
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
857 main_package_source = cache_from_repos()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
858 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
859 main_package_source = main_package
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
860
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
861 manifest = {}
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
862 manifest['app-id'] = app_id
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
863 manifest['default-branch'] = args.version
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
864 # we update DEFAULT_MANIFEST only now to have "app-id" and "default-branch" on the top
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
865 # of the manifest
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
866 manifest.update(DEFAULT_MANIFEST)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
867 manifest.update(template)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
868
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
869 modules.extend(get_libxslt())
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
870
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
871 # setup requirements
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
872 modules.extend(get_python_package(
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
873 PYTHON_SETUP_REQUIREMENTS,
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
874 step_name="setup_requirements",
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
875 step_message="generating modules for setup requirements")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
876 )
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
877
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
878 # python dependencies
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
879 modules.extend(get_python_deps())
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
880
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
881 # at this point we add things specified in the template
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
882 modules.extend(manifest.get('modules', []))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
883 manifest['modules'] = modules
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
884
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
885 # sat common things
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
886 existing_modules = {d['name'] for d in modules}
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
887 if "sat_templates" not in existing_modules:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
888 modules.extend(get_python_package("sat_templates", with_pre=True))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
889 modules.extend(get_sat_media())
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
890 modules.extend(file_upload('sat.conf', '/app'))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
891
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
892 # wrapper to launch the backend if needed
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
893 wrapped_command = build_settings.get('wrapped_command')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
894 if wrapped_command:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
895 modules.extend(file_upload('sat_wrapper', src='sat_wrapper.py',
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
896 replace={'##COMMAND##': wrapped_command},
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
897 use_install=True))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
898
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
899 # icon
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
900 modules.extend(get_icon())
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
901
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
902 # desktop file and appdata file
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
903 modules.extend(get_app_metadata())
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
904
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
905 # now the app itself
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
906 if args.version == 'dev' and 'dev_repos' in build_settings:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
907 # mercurial is needed for dev version to install but also to
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
908 # retrieve revision used
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
909 modules.extend(get_python_package("mercurial"))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
910 modules.extend(get_repos_module())
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
911 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
912 modules.extend(get_python_package(main_package, version=args.version))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
913
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
914 print_step("writing manifest")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
915 with package_file.open('w') as f:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
916 json.dump(manifest, f, indent=4)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
917
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
918 if args.ignore_cache:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
919 print("/!\\ those --ignore-cache arguments don't correspond to any step: {}".
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
920 format(', '.join(args.ignore_cache)))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
921
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
922 print(f"manifest generated successfully at {package_file}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
923
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
924 if args.export is not None:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
925 print_step("exporting building files")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
926 print(f"exporting to {args.export}\n")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
927 to_export = [
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
928 ("package file", package_file),
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
929 ]
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
930
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
931 for m in modules:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
932 for s in m.get('sources', []):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
933 s_type = s.get('type')
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
934 if s_type in ('path', 'file'):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
935 try:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
936 path = s['path']
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
937 except KeyError:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
938 if 'url' in s:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
939 continue
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
940 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
941 raise ValueError(f"path missing for module source:\n{s}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
942 to_export.append((s_type, Path(path)))
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
943
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
944 for label, path in to_export:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
945 print(f"exporting {label}: {path}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
946 dest = args.export / path
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
947 if dest.exists():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
948 if args.force_export:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
949 print(f" replacing existing {dest}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
950 if path.is_dir():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
951 shutil.rmtree(dest)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
952 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
953 dest.unlink()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
954 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
955 print(" it already exists, skipping")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
956 continue
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
957 if args.symlink == 'all':
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
958 os.symlink(path, dest)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
959 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
960 if path.is_dir():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
961 shutil.copytree(path, dest)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
962 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
963 shutil.copyfile(path, dest)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
964
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
965 if args.version == 'dev':
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
966 print("exporting cache")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
967 dest = args.export / get_cache_dir()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
968 if args.force_export and os.path.lexists(dest):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
969 print(f" replacing existing {dest}")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
970 if dest.is_symlink():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
971 dest.unlink()
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
972 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
973 shutil.rmtree(dest)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
974
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
975 if dest.exists():
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
976 print(" it already exists, skipping")
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
977 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
978 if args.symlink in ('all', 'cache'):
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
979 os.symlink(get_cache_dir().resolve(), dest)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
980 else:
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
981 shutil.copytree(get_cache_dir(), dest)
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
982
274af514a5cf flatpak: reworked packages + made a building script:
Goffi <goffi@goffi.org>
parents:
diff changeset
983 print("\nexport completed")