annotate libervia/frontends/tools/webrtc.py @ 4306:94e0968987cd

plugin XEP-0033: code modernisation, improve delivery, data validation: - Code has been rewritten using Pydantic models and `async` coroutines for data validation and cleaner element parsing/generation. - Delivery has been completely rewritten. It now works even if server doesn't support multicast, and send to local multicast service first. Delivering to local multicast service first is due to bad support of XEP-0033 in server (notably Prosody which has an incomplete implementation), and the current impossibility to detect if a sub-domain service handles fully multicast or only for local domains. This is a workaround to have a good balance between backward compatilibity and use of bandwith, and to make it work with the incoming email gateway implementation (the gateway will only deliver to entities of its own domain). - disco feature checking now uses `async` corountines. `host` implementation still use Deferred return values for compatibility with legacy code. rel 450
author Goffi <goffi@goffi.org>
date Thu, 26 Sep 2024 16:12:01 +0200
parents 2992f9d1e039
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1 #!/usr/bin/env python3
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
2
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
3 # Libervia WebRTC implementation
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # Copyright (C) 2009-2023 Jérôme Poisson (goffi@goffi.org)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
5
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 # This program is free software: you can redistribute it and/or modify
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # it under the terms of the GNU Affero General Public License as published by
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # the Free Software Foundation, either version 3 of the License, or
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # (at your option) any later version.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
10
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
11 # This program is distributed in the hope that it will be useful,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # GNU Affero General Public License for more details.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
15
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 # You should have received a copy of the GNU Affero General Public License
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
18
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
19 from collections.abc import Awaitable
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
20 import gi
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
21
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
22 gi.require_versions({"Gst": "1.0", "GstWebRTC": "1.0"})
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
23 from gi.repository import Gst, GstWebRTC, GstSdp
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
24
4145
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
25 from libervia.backend.core import exceptions
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
26
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
27 try:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
28 from gi.overrides import Gst as _
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
29 except ImportError:
4145
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
30 raise exceptions.MissingModule(
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
31 "No GStreamer Python overrides available. Please install relevant packages on "
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
32 "your system (e.g., `python3-gst-1.0` on Debian and derivatives)."
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
33 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
34 import asyncio
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
35 from datetime import datetime
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
36 import logging
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
37 import re
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
38 from typing import Callable, Final
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
39 from urllib.parse import quote_plus
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
40
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
41 from libervia.backend.tools.common import data_format
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
42 from libervia.frontends.tools import aio, display_servers, jid
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
43 from .webrtc_models import (
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
44 CallData,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
45 SinksApp,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
46 SinksAuto,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
47 SinksData,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
48 SinksDataChannel,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
49 SinksNone,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
50 SourcesAuto,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
51 SourcesData,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
52 SourcesDataChannel,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
53 SourcesNone,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
54 SourcesPipeline,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
55 SourcesTest,
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
56 )
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
57
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
58 current_server = display_servers.detect()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
59 if current_server == display_servers.X11:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
60 # GSTreamer's ximagesrc documentation asks to run this function
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
61 import ctypes
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
62
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
63 ctypes.CDLL("libX11.so.6").XInitThreads()
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
64
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
65
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
66 log = logging.getLogger(__name__)
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
67 VIDEO_SOURCE_AUTO: Final = "v4l2src"
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
68 AUDIO_SOURCE_AUTO: Final = "pulsesrc"
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
69 NONE_NOT_IMPLEMENTED_MSG: Final = "None value is not handled yet."
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
70
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
71 Gst.init(None)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
72
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
73
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
74 class WebRTC:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
75 """GSTreamer based WebRTC implementation for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
76
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
77 This class encapsulates the WebRTC functionalities required for initiating and
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
78 handling audio and video calls, and data channels.
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
79 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
80
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
81 def __init__(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
82 self,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
83 bridge,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
84 profile: str,
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
85 sources_data: SourcesData | None = None,
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
86 sinks_data: SinksData | None = None,
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
87 reset_cb: Callable | None = None,
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
88 merge_pip: bool | None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
89 target_size: tuple[int, int] | None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
90 call_start_cb: Callable[[str, dict, str], Awaitable[str]] | None = None,
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
91 dc_data_list: list[SourcesDataChannel | SinksDataChannel] | None = None,
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
92 ) -> None:
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
93 """Initializes a new WebRTC instance.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
94
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
95 @param bridge: An instance of backend bridge.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
96 @param profile: Libervia profile.
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
97 @param sources_data: Data of the sources.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
98 The model used will determine which sources to use.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
99 SourcesDataChannel can be used here as a convenience. It will then be moved
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
100 to ``data_channels`` and ``SourcesNone`` will be used instead for
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
101 ``sources_data``.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
102 If None, SourcesAuto will be used.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
103 @param sinks_data: Data of the sinks.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
104 The model used will determine which sinks to use.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
105 SinksDataChannel can be used here as a convenience. It will then be moved
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
106 to ``data_channels`` and ``SinksNone`` will be used instead for
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
107 ``sinks_data``.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
108 If None, SinksAuto will be used.
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
109 @param reset_cb: An optional Callable that is triggered on reset events. Can be
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
110 used to reset UI data on new calls.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
111 @param merge_pip: A boolean flag indicating whether Picture-in-Picture mode is
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
112 enabled. When PiP is used, local feedback is merged to remote video stream.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
113 Only one video stream is then produced (the local one).
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
114 If None, PiP mode is selected automatically according to selected sink (it's
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
115 used for SinksAuto only for now).
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
116 @param target_size: Expected size of the final sink stream. Mainly use by composer
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
117 when ``merge_pip`` is set.
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
118 None to autodetect (no real autodetection implemeted yet, default to
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
119 (1280,720)).
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
120 @param call_start_cb: Called when call is started.
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
121 @param dc_data_list: Data Channels to create.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
122 If a SourcesDataChannel is used as ``sources_data``, or a SinksDataChannel is
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
123 used as ``sinks_data``, they will be automatically added to this list.
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
124 """
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
125 self.main_loop = asyncio.get_event_loop()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
126 self.bridge = bridge
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
127 self.profile = profile
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
128 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
129 self._audio_muted = False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
130 self._video_muted = False
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
131 self._desktop_sharing = False
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
132 self.desktop_sharing_data = None
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
133 if dc_data_list is None:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
134 dc_data_list = []
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
135 self.dc_data_list = dc_data_list
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
136 if sources_data is None:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
137 sources_data = SourcesAuto()
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
138 elif isinstance(sources_data, SourcesDataChannel):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
139 dc_data_list.append(sources_data)
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
140 sources_data = SourcesNone()
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
141 self.sources_data = sources_data
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
142 if sinks_data is None:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
143 sinks_data = SinksAuto()
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
144 elif isinstance(sinks_data, SinksDataChannel):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
145 dc_data_list.append(sinks_data)
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
146 sinks_data = SinksNone()
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
147 self.sinks_data = sinks_data
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
148 if target_size is None:
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
149 target_size = (1280, 720)
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
150 self.target_width, self.target_height = target_size
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
151 if merge_pip is None:
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
152 merge_pip = isinstance(sinks_data, SinksAuto)
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
153 self.merge_pip = merge_pip
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
154 if call_start_cb is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
155 call_start_cb = self._call_start
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
156 self.call_start_cb = call_start_cb
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
157 if isinstance(sinks_data, SinksApp):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
158 if merge_pip and sinks_data.remote_video_cb is not None:
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
159 raise ValueError("Remote_video_cb can't be used when merge_pip is used!")
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
160 self.reset_cb = reset_cb
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
161 if current_server == display_servers.WAYLAND:
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
162 from .portal_desktop import DesktopPortal
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
163
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
164 self.desktop_portal = DesktopPortal(
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
165 on_session_closed_cb=self.on_portal_session_closed
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
166 )
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
167 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
168 self.desktop_portal = None
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
169 self.reset_instance()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
170
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
171 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
172 def audio_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
173 return self._audio_muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
174
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
175 @audio_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
176 def audio_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
177 if muted != self._audio_muted:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
178 self._audio_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
179 self.on_audio_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
180
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
181 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
182 def video_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
183 return self._video_muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
184
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
185 @video_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
186 def video_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
187 if muted != self._video_muted:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
188 self._video_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
189 self.on_video_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
190
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
191 @property
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
192 def desktop_sharing(self):
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
193 return self._desktop_sharing
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
194
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
195 @desktop_sharing.setter
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
196 def desktop_sharing(self, active: bool) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
197 if active != self._desktop_sharing:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
198 self._desktop_sharing = active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
199 self.on_desktop_switch(active)
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
200 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
201 cb = self.bindings["desktop_sharing"]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
202 except KeyError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
203 pass
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
204 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
205 cb(active)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
206
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
207 @property
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
208 def sdp_set(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
209 return self._sdp_set
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
210
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
211 @sdp_set.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
212 def sdp_set(self, is_set: bool):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
213 self._sdp_set = is_set
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
214 if is_set:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
215 self.on_ice_candidates_new(self.remote_candidates_buffer)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
216 for data in self.remote_candidates_buffer.values():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
217 data["candidates"].clear()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
218
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
219 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
220 def media_types(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
221 if self._media_types is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
222 raise Exception("self._media_types should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
223 return self._media_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
224
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
225 @media_types.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
226 def media_types(self, new_media_types: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
227 self._media_types = new_media_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
228 self._media_types_inv = {v: k for k, v in new_media_types.items()}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
229
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
230 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
231 def media_types_inv(self) -> dict:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
232 if self._media_types_inv is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
233 raise Exception("self._media_types_inv should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
234 return self._media_types_inv
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
235
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
236 def bind(self, **kwargs: Callable) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
237 self.bindings.clear()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
238 for key, cb in kwargs.items():
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
239 if key not in ("desktop_sharing",):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
240 raise ValueError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
241 'Only "desktop_sharing" is currently allowed for binding'
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
242 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
243 self.bindings[key] = cb
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
244
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
245 def generate_dot_file(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
246 self,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
247 filename: str = "pipeline",
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
248 details: Gst.DebugGraphDetails = Gst.DebugGraphDetails.ALL,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
249 with_timestamp: bool = True,
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
250 bin_: Gst.Bin | None = None,
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
251 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
252 """Generate Dot File for debugging
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
253
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
254 ``GST_DEBUG_DUMP_DOT_DIR`` environment variable must be set to destination dir.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
255 ``dot -Tpng -o <filename>.png <filename>.dot`` can be use to convert to a PNG file.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
256 See
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
257 https://gstreamer.freedesktop.org/documentation/gstreamer/debugutils.html?gi-language=python#GstDebugGraphDetails
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
258 for details.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
259
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
260 @param filename: name of the generated file
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
261 @param details: which details to print
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
262 @param with_timestamp: if True, add a timestamp to filename
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
263 @param bin_: which bin to output. By default, the whole pipeline
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
264 (``self.pipeline``) will be used.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
265 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
266 if bin_ is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
267 bin_ = self.pipeline
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
268 if with_timestamp:
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
269 timestamp = datetime.now().isoformat(timespec="milliseconds")
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
270 filename = f"{timestamp}_filename"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
271
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
272 Gst.debug_bin_to_dot_file(bin_, details, filename)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
273
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
274 def get_sdp_mline_index(self, media_type: str) -> int:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
275 """Gets the sdpMLineIndex for a given media type.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
276
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
277 @param media_type: The type of the media.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
278 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
279 for index, m_type in self.media_types.items():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
280 if m_type == media_type:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
281 return index
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
282 raise ValueError(f"Media type '{media_type}' not found")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
283
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
284 def _set_media_types(self, offer_sdp: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
285 """Sets media types from offer SDP
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
286
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
287 @param offer: RTC session description containing the offer
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
288 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
289 sdp_lines = offer_sdp.splitlines()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
290 media_types = {}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
291 mline_index = 0
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
292
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
293 for line in sdp_lines:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
294 if line.startswith("m="):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
295 media_types[mline_index] = line[2 : line.find(" ")]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
296 mline_index += 1
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
297
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
298 self.media_types = media_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
299
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
300 def _a_call(self, method, *args, **kwargs):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
301 """Call an async method in main thread"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
302 aio.run_from_thread(method, *args, **kwargs, loop=self.main_loop)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
303
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
304 def get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
305 self, sdpmsg, video_encoding: str, audio_encoding: str
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
306 ) -> dict[str, int | None]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
307 """Find the payload types for the specified video and audio encoding.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
308
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
309 Very simplistically finds the first payload type matching the encoding
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
310 name. More complex applications will want to match caps on
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
311 profile-level-id, packetization-mode, etc.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
312 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
313 # method coming from gstreamer example (Matthew Waters, Nirbheek Chauhan) at
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
314 # subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
315 video_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
316 audio_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
317 for i in range(0, sdpmsg.medias_len()):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
318 media = sdpmsg.get_media(i)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
319 for j in range(0, media.formats_len()):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
320 fmt = media.get_format(j)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
321 if fmt == "webrtc-datachannel":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
322 continue
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
323 pt = int(fmt)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
324 caps = media.get_caps_from_media(pt)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
325 s = caps.get_structure(0)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
326 encoding_name = s["encoding-name"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
327 if video_pt is None and encoding_name == video_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
328 video_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
329 elif audio_pt is None and encoding_name == audio_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
330 audio_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
331 return {video_encoding: video_pt, audio_encoding: audio_pt}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
332
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
333 def parse_ice_candidate(self, candidate_string):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
334 """Parses the ice candidate string.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
335
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
336 @param candidate_string: The ice candidate string to be parsed.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
337 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
338 pattern = re.compile(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
339 r"candidate:(?P<foundation>\S+) (?P<component_id>\d+) (?P<transport>\S+) "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
340 r"(?P<priority>\d+) (?P<address>\S+) (?P<port>\d+) typ "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
341 r"(?P<type>\S+)(?: raddr (?P<rel_addr>\S+) rport "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
342 r"(?P<rel_port>\d+))?(?: generation (?P<generation>\d+))?"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
343 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
344 match = pattern.match(candidate_string)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
345 if match:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
346 candidate_dict = match.groupdict()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
347
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
348 # Apply the correct types to the dictionary values
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
349 candidate_dict["component_id"] = int(candidate_dict["component_id"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
350 candidate_dict["priority"] = int(candidate_dict["priority"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
351 candidate_dict["port"] = int(candidate_dict["port"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
352
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
353 if candidate_dict["rel_port"]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
354 candidate_dict["rel_port"] = int(candidate_dict["rel_port"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
355
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
356 if candidate_dict["generation"]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
357 candidate_dict["generation"] = candidate_dict["generation"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
358
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
359 # Remove None values
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
360 return {k: v for k, v in candidate_dict.items() if v is not None}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
361 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
362 log.warning(f"can't parse candidate: {candidate_string!r}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
363 return None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
364
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
365 def build_ice_candidate(self, parsed_candidate):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
366 """Builds ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
367
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
368 @param parsed_candidate: Dictionary containing parsed ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
369 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
370 base_format = (
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
371 "candidate:{foundation} {component_id} {transport} {priority} "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
372 "{address} {port} typ {type}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
373 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
374
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
375 if parsed_candidate.get("rel_addr") and parsed_candidate.get("rel_port"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
376 base_format += " raddr {rel_addr} rport {rel_port}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
377
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
378 if parsed_candidate.get("generation"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
379 base_format += " generation {generation}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
380
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
381 return base_format.format(**parsed_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
382
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
383 def extract_ufrag_pwd(self, sdp: str) -> None:
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
384 """Retrieves ICE password and user fragment for SDP offer.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
385
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
386 @param sdp: The Session Description Protocol offer string.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
387 """
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
388 lines = sdp.splitlines()
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
389 media = ""
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
390 mid_media_map = {}
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
391 bundle_media = set()
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
392 bundle_ufrag = ""
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
393 bundle_pwd = ""
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
394 in_bundle = False
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
395
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
396 for line in lines:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
397 if line.startswith("m="):
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
398 media = line.split("=")[1].split()[0]
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
399 elif line.startswith("a=mid:"):
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
400 mid = line.split(":")[1].strip()
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
401 mid_media_map[mid] = media
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
402 elif line.startswith("a=group:BUNDLE"):
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
403 in_bundle = True
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
404 bundle_media = set(line.split(":")[1].strip().split())
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
405 elif line.startswith("a=ice-ufrag:"):
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
406 if in_bundle:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
407 bundle_ufrag = line.split(":")[1].strip()
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
408 else:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
409 self.ufrag[media] = line.split(":")[1].strip()
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
410 elif line.startswith("a=ice-pwd:"):
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
411 if in_bundle:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
412 bundle_pwd = line.split(":")[1].strip()
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
413 else:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
414 self.pwd[media] = line.split(":")[1].strip()
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
415 else:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
416 in_bundle = False
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
417
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
418 if bundle_ufrag and bundle_pwd:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
419 for mid in bundle_media:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
420 media = mid_media_map[mid]
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
421 self.ufrag[media] = bundle_ufrag
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
422 self.pwd[media] = bundle_pwd
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
423
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
424 def reset_instance(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
425 """Inits or resets the instance variables to their default state."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
426 self.role: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
427 if self.pipeline is not None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
428 self.pipeline.set_state(Gst.State.NULL)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
429 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
430 self._remote_video_pad = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
431 self.sid: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
432 self.offer: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
433 self.local_candidates_buffer = {}
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
434 self.ufrag: dict[str, str] = {}
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
435 self.pwd: dict[str, str] = {}
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
436 self.callee: jid.JID | None = None
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
437 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
438 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
439 self._sdp_set: bool = False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
440 self.remote_candidates_buffer: dict[str, dict[str, list]] = {
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
441 "audio": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
442 "video": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
443 }
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
444 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
445 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
446 self.audio_valve = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
447 self.video_valve = None
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
448 self.video_selector = None
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
449 if self.desktop_portal is not None:
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
450 self.desktop_portal.end_session()
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
451 self.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
452 self.desktop_sink_pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
453 self.bindings = {}
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
454 if self.reset_cb is not None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
455 self.reset_cb()
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
456 self.data_channels: dict[str, GstWebRTC.WebRTCDataChannel] = {}
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
457
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
458 @property
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
459 def data_channel(self) -> GstWebRTC.WebRTCDataChannel:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
460 """Convenience method to get WebRTCDataChannel instance when there is only one."""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
461 if len(self.data_channels) != 1:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
462 raise exceptions.InternalError(
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
463 "self.data_channel can only be used in a single Data Channel scenario. "
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
464 "Use self.data_channels dict instead."
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
465 )
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
466 return next(iter(self.data_channels.values()))
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
467
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
468 async def setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
469 self,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
470 role: str,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
471 audio_pt: int | None = 96,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
472 video_pt: int | None = 97,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
473 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
474 """Sets up the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
475
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
476 This method establishes the Gstreamer pipeline for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
477 The method also manages STUN and TURN server configurations, signal watchers, and
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
478 various connection handlers for the webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
479
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
480 @param role: The role for the call, either 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
481 @param audio_pt: The payload type for the audio stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
482 @param video_pt: The payload type for the video stream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
483
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
484 @raises NotImplementedError: If audio_pt or video_pt is set to None.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
485 @raises AssertionError: If the role is not 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
486 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
487 assert role in ("initiator", "responder")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
488 self.role = role
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
489
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
490 if isinstance(self.sources_data, SourcesPipeline):
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
491 if self.sources_data.video_pipeline != "" and video_pt is None:
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
492 raise NotImplementedError(NONE_NOT_IMPLEMENTED_MSG)
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
493 if self.sources_data.audio_pipeline != "" and audio_pt is None:
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
494 raise NotImplementedError(NONE_NOT_IMPLEMENTED_MSG)
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
495 elif isinstance(self.sources_data, SourcesNone):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
496 pass
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
497 else:
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
498 if audio_pt is None or video_pt is None:
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
499 raise NotImplementedError(NONE_NOT_IMPLEMENTED_MSG)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
500
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
501 match self.sources_data:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
502 case SourcesAuto():
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
503 video_source_elt = VIDEO_SOURCE_AUTO
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
504 audio_source_elt = AUDIO_SOURCE_AUTO
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
505 case SourcesNone():
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
506 video_source_elt = ""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
507 audio_source_elt = ""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
508 case SourcesPipeline() as source:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
509 if source.video_pipeline is None:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
510 video_source_elt = VIDEO_SOURCE_AUTO
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
511 else:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
512 video_source_elt = source.video_pipeline
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
513 if source.audio_pipeline is None:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
514 audio_source_elt = AUDIO_SOURCE_AUTO
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
515 else:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
516 audio_source_elt = source.audio_pipeline
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
517 case SourcesTest():
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
518 video_source_elt = "videotestsrc is-live=true pattern=ball"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
519 audio_source_elt = "audiotestsrc"
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
520 case _:
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
521 raise exceptions.InternalError(
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
522 f'Unexpected "sources_data" value: {self.sources_data!r}'
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
523 )
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
524
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
525 match self.sinks_data:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
526 case SinksApp():
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
527 local_video_sink_elt = (
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
528 "appsink name=local_video_sink emit-signals=true drop=true "
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
529 "max-buffers=1 sync=True"
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
530 )
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
531 case SinksAuto():
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
532 if isinstance(self.sources_data, SourcesNone):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
533 local_video_sink_elt = ""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
534 else:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
535 local_video_sink_elt = "autovideosink"
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
536 case SinksNone():
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
537 local_video_sink_elt = ""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
538 case _:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
539 raise exceptions.InternalError(
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
540 f'Unexpected "sinks_data" value {self.sinks_data!r}'
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
541 )
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
542
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
543 gst_pipe_elements = [
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
544 "webrtcbin latency=30 name=sendrecv bundle-policy=max-bundle"
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
545 ]
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
546
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
547 if self.merge_pip and local_video_sink_elt:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
548 # Compositor is used to merge local video feedback in video sink, useful when
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
549 # we have only a single video sink.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
550 gst_pipe_elements.append(
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
551 "compositor name=compositor background=black "
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
552 f"! video/x-raw,width={self.target_width},"
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
553 f"height={self.target_height},framerate=30/1 "
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
554 f"! {local_video_sink_elt}"
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
555 )
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
556 local_video_sink_elt = "compositor.sink_1"
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
557
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
558 if video_source_elt:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
559 # Video source with an input-selector to switch between normal and video mute
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
560 # (or desktop sharing).
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
561 gst_pipe_elements.append(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
562 f"""
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
563 input-selector name=video_selector
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
564 ! videorate drop-only=1 max-rate=30
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
565 ! video/x-raw,framerate=30/1
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
566 ! tee name=t
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
567
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
568 {video_source_elt} name=video_src
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
569 ! queue max-size-buffers=1 max-size-time=0 max-size-bytes=0 leaky=downstream
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
570 ! video_selector.
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
571
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
572 videotestsrc name=muted_src is-live=true pattern=black
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
573 ! queue leaky=downstream
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
574 ! video_selector.
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
575
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
576 t.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
577 ! queue max-size-buffers=1 max-size-time=0 max-size-bytes=0 leaky=downstream
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
578 ! videoscale
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
579 ! videoconvert
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
580 ! vp8enc deadline=1 keyframe-max-dist=30
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
581 ! rtpvp8pay picture-id-mode=15-bit
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
582 ! application/x-rtp,media=video,encoding-name=VP8,payload={video_pt}
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
583 ! sendrecv.
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
584 """
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
585 )
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
586
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
587 if local_video_sink_elt:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
588 # Local video feedback.
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
589 gst_pipe_elements.append(
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
590 f"""
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
591 t.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
592 ! queue max-size-buffers=1 max-size-time=0 max-size-bytes=0 leaky=downstream
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
593 ! videoconvert
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
594 ! {local_video_sink_elt}
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
595 """
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
596 )
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
597
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
598 if audio_source_elt:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
599 # Audio with a valve for muting.
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
600 gst_pipe_elements.append(
4274
2992f9d1e039 frontends (tools/webrtc): fix typo resulting in bad pipeline for audio.
Goffi <goffi@goffi.org>
parents: 4270
diff changeset
601 f"""
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
602 {audio_source_elt} name=audio_src
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
603 ! valve
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
604 ! queue max-size-buffers=10 max-size-time=0 max-size-bytes=0 leaky=downstream
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
605 ! audioconvert
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
606 ! audioresample
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
607 ! opusenc audio-type=voice
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
608 ! rtpopuspay
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
609 ! application/x-rtp,media=audio,encoding-name=OPUS,payload={audio_pt}
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
610 ! sendrecv.
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
611 """
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
612 )
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
613
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
614 self.gst_pipe_desc = "\n\n".join(gst_pipe_elements)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
615
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
616 log.debug(f"Gstreamer pipeline: {self.gst_pipe_desc}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
617
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
618 # Create the pipeline
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
619 try:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
620 self.pipeline = Gst.parse_launch(self.gst_pipe_desc)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
621 except Exception:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
622 log.exception("Can't parse pipeline")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
623 self.pipeline = None
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
624 if not self.pipeline:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
625 raise exceptions.InternalError("Failed to create Gstreamer pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
626
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
627 if not isinstance(self.pipeline, Gst.Pipeline):
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
628 # in the case of Data Channel there is a single element, and Gst.parse_launch
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
629 # doesn't create a Pipeline in this case, so we do it manually.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
630 pipeline = Gst.Pipeline()
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
631 pipeline.add(self.pipeline)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
632 self.pipeline = pipeline
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
633
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
634 self.webrtcbin = self.pipeline.get_by_name("sendrecv")
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
635 if self.webrtcbin is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
636 raise exceptions.InternalError("Can't get the pipeline.")
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
637
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
638 # If video or audio sources are not created, ``get_by_name`` will return None.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
639 self.video_src = self.pipeline.get_by_name("video_src")
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
640 self.muted_src = self.pipeline.get_by_name("muted_src")
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
641 self.video_selector = self.pipeline.get_by_name("video_selector")
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
642 if self.video_src and isinstance(self.sources_data, SourcesPipeline):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
643 for name, value in self.sources_data.video_properties.items():
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
644 self.video_src.set_property(name, value)
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
645
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
646 self.audio_src = self.pipeline.get_by_name("audio_src")
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
647 if self.audio_src and isinstance(self.sources_data, SourcesPipeline):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
648 for name, value in self.sources_data.audio_properties.items():
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
649 self.audio_src.set_property(name, value)
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
650
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
651 self.audio_valve = self.pipeline.get_by_name("audio_valve")
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
652
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
653 if self.video_muted:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
654 self.on_video_mute(True)
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
655 if self.audio_muted:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
656 self.on_audio_mute(True)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
657
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
658 # set STUN and TURN servers
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
659 external_disco = data_format.deserialise(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
660 await self.bridge.external_disco_get("", self.profile), type_check=list
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
661 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
662
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
663 for server in external_disco:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
664 if server["type"] == "stun":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
665 if server["transport"] == "tcp":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
666 log.info(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
667 "ignoring TCP STUN server, GStreamer only support one STUN server"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
668 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
669 url = f"stun://{server['host']}:{server['port']}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
670 log.debug(f"adding stun server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
671 self.webrtcbin.set_property("stun-server", url)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
672 elif server["type"] == "turn":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
673 url = "{scheme}://{username}:{password}@{host}:{port}".format(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
674 scheme="turns" if server["transport"] == "tcp" else "turn",
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
675 username=quote_plus(server["username"]),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
676 password=quote_plus(server["password"]),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
677 host=server["host"],
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
678 port=server["port"],
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
679 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
680 log.debug(f"adding turn server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
681
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
682 if not self.webrtcbin.emit("add-turn-server", url):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
683 log.warning(f"Erreur while adding TURN server {url}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
684
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
685 # local video feedback
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
686 if isinstance(self.sinks_data, SinksApp):
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
687 local_video_sink = self.pipeline.get_by_name("local_video_sink")
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
688 if local_video_sink is not None:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
689 local_video_sink.set_property("emit-signals", True)
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
690 local_video_sink.connect("new-sample", self.sinks_data.local_video_cb)
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
691 local_video_sink_caps = Gst.Caps.from_string(f"video/x-raw,format=RGB")
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
692 local_video_sink.set_property("caps", local_video_sink_caps)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
693
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
694 # Create bus and associate signal watchers
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
695 self.bus = self.pipeline.get_bus()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
696 if not self.bus:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
697 log.error("Failed to get bus from pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
698 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
699
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
700 self.bus.add_signal_watch()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
701 self.webrtcbin.connect("pad-added", self.on_pad_added)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
702 self.bus.connect("message::error", self.on_bus_error)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
703 self.bus.connect("message::eos", self.on_bus_eos)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
704 self.webrtcbin.connect("on-negotiation-needed", self.on_negotiation_needed)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
705 self.webrtcbin.connect("on-ice-candidate", self.on_ice_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
706 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
707 "notify::ice-gathering-state", self.on_ice_gathering_state_change
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
708 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
709 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
710 "notify::ice-connection-state", self.on_ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
711 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
712
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
713 for dc_data in self.dc_data_list:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
714 self.create_data_channel(dc_data)
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
715
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
716 def create_data_channel(self, dc_data: SourcesDataChannel | SinksDataChannel) -> None:
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
717 """Create a Data Channel and connect relevant callbacks."""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
718 assert self.pipeline is not None
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
719 if isinstance(dc_data, SourcesDataChannel):
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
720 # Data channel configuration for compatibility with browser defaults
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
721 data_channel_options = Gst.Structure.new_empty("data-channel-options")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
722 data_channel_options.set_value("ordered", True)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
723 data_channel_options.set_value("protocol", "")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
724
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
725 # Create the data channel
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
726 self.pipeline.set_state(Gst.State.READY)
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
727 self.data_channels[dc_data.name] = data_channel = self.webrtcbin.emit(
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
728 "create-data-channel", dc_data.name, data_channel_options
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
729 )
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
730 if data_channel is None:
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
731 log.error("Failed to create data channel")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
732 return
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
733 data_channel.connect("on-open", dc_data.dc_open_cb)
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
734 elif isinstance(dc_data, SinksDataChannel):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
735 self.webrtcbin.connect("on-data-channel", dc_data.dc_on_data_channel)
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
736 else:
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
737 raise ValueError("Only SourcesDataChannel or SinksDataChannel are allowed.")
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
738
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
739 def start_pipeline(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
740 """Starts the GStreamer pipeline."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
741 log.debug("starting the pipeline")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
742 self.pipeline.set_state(Gst.State.PLAYING)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
743
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
744 def on_negotiation_needed(self, webrtc):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
745 """Initiate SDP offer when negotiation is needed."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
746 log.debug("Negotiation needed.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
747 if self.role == "initiator":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
748 log.debug("Creating offer…")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
749 promise = Gst.Promise.new_with_change_func(self.on_offer_created)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
750 self.webrtcbin.emit("create-offer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
751
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
752 def on_offer_created(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
753 """Callback for when SDP offer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
754 log.info("on_offer_created called")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
755 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
756 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
757 if reply is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
758 log.error("Promise reply is None. Offer creation might have failed.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
759 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
760 offer = reply["offer"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
761 self.offer = offer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
762 log.info(f"SDP offer created: \n{self.offer}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
763 self._set_media_types(self.offer)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
764 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
765 self.webrtcbin.emit("set-local-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
766 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
767 self._a_call(self._start_call)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
768
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
769 def on_answer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
770 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
771
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
772 def on_answer_created(self, promise, _, __):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
773 """Callback for when SDP answer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
774 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
775 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
776 answer = reply["answer"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
777 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
778 self.webrtcbin.emit("set-local-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
779 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
780 answer_sdp = answer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
781 log.info(f"SDP answer set: \n{answer_sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
782 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
783 self._a_call(self.bridge.call_answer_sdp, self.sid, answer_sdp, self.profile)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
784
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
785 def on_offer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
786 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
787 promise = Gst.Promise.new_with_change_func(self.on_answer_created, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
788 self.webrtcbin.emit("create-answer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
789
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
790 def link_element_or_pad(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
791 self, source: Gst.Element, dest: Gst.Element | Gst.Pad
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
792 ) -> bool:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
793 """Check if dest is a pad or an element, and link appropriately"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
794 src_pad = source.get_static_pad("src")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
795
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
796 if isinstance(dest, Gst.Pad):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
797 # If the dest is a pad, link directly
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
798 if not src_pad.link(dest) == Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
799 log.error(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
800 "Failed to link 'conv' to the compositor's newly requested pad!"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
801 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
802 return False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
803 elif isinstance(dest, Gst.Element):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
804 return source.link(dest)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
805 else:
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
806 log.error(f"Unexpected type for dest: {type(dest)}")
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
807 return False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
808
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
809 return True
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
810
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
811 def scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
812 self, original_width: int, original_height: int, max_width: int, max_height: int
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
813 ) -> tuple[int, int]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
814 """Calculates the scaled dimensions preserving aspect ratio.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
815
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
816 @param original_width: Original width of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
817 @param original_height: Original height of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
818 @param max_width: Maximum desired width for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
819 @param max_height: Maximum desired height for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
820 @return: The width and height of the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
821 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
822 aspect_ratio = original_width / original_height
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
823 new_width = int(max_height * aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
824
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
825 if new_width <= max_width:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
826 return new_width, max_height
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
827
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
828 new_height = int(max_width / aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
829 return max_width, new_height
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
830
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
831 def on_remote_decodebin_stream(self, _, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
832 """Handle the stream from the remote decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
833
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
834 This method processes the incoming stream from the remote decodebin, determining
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
835 whether it's video or audio. It then sets up the appropriate GStreamer elements
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
836 for video/audio processing and adds them to the pipeline.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
837
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
838 @param pad: The Gst.Pad from the remote decodebin producing the stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
839 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
840 assert self.pipeline is not None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
841 if not pad.has_current_caps():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
842 log.error(f"{pad} has no caps, ignoring")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
843 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
844
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
845 caps = pad.get_current_caps()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
846 assert len(caps)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
847 s = caps[0]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
848 name = s.get_name()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
849 log.debug(f"====> NAME START: {name}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
850
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
851 q = Gst.ElementFactory.make("queue")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
852
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
853 if name.startswith("video"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
854 log.debug("===> VIDEO OK")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
855
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
856 self._remote_video_pad = pad
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
857
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
858 # Check and log the original size of the video
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
859 width = self.target_width
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
860 height = self.target_height
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
861 log.info(f"Original video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
862
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
863 # This is a fix for an issue found with Movim on desktop: a non standard
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
864 # resolution is used (990x557) resulting in bad alignement and no color in
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
865 # rendered image
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
866 adjust_resolution = width % 4 != 0 or height % 4 != 0
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
867 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
868 log.info("non standard resolution, we need to adjust size")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
869 width = (width + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
870 height = (height + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
871 log.info(f"Adjusted video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
872
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
873 conv = Gst.ElementFactory.make("videoconvert")
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
874 if self.merge_pip:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
875 # with ``merge_pip`` set, we plug the remote stream to the composer
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
876 compositor = self.pipeline.get_by_name("compositor")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
877
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
878 sink1_pad = compositor.get_static_pad("sink_1")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
879
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
880 local_width, local_height = self.scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
881 sink1_pad.get_property("width"),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
882 sink1_pad.get_property("height"),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
883 width // 3,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
884 height // 3,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
885 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
886
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
887 sink1_pad.set_property("xpos", width - local_width)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
888 sink1_pad.set_property("ypos", height - local_height)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
889 sink1_pad.set_property("width", local_width)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
890 sink1_pad.set_property("height", local_height)
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
891 sink1_pad.set_property("sizing-policy", 1)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
892 sink1_pad.set_property("zorder", 1)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
893
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
894 # Request a new pad for the remote stream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
895 sink_pad_template = compositor.get_pad_template("sink_%u")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
896 remote_video_sink = compositor.request_pad(sink_pad_template, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
897 remote_video_sink.set_property("zorder", 0)
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
898 remote_video_sink.set_property("width", width)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
899 remote_video_sink.set_property("height", height)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
900 remote_video_sink.set_property("sizing-policy", 1)
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
901 elif isinstance(self.sinks_data, SinksApp):
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
902 # ``app`` sink without ``self.merge_pip`` set, be create the sink and
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
903 # connect it to the ``remote_video_cb``.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
904 remote_video_sink = Gst.ElementFactory.make("appsink")
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
905
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
906 remote_video_caps = Gst.Caps.from_string("video/x-raw,format=RGB")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
907 remote_video_sink.set_property("caps", remote_video_caps)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
908
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
909 remote_video_sink.set_property("emit-signals", True)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
910 remote_video_sink.set_property("drop", True)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
911 remote_video_sink.set_property("max-buffers", 1)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
912 remote_video_sink.set_property("sync", True)
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
913 remote_video_sink.connect("new-sample", self.sinks_data.remote_video_cb)
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
914 self.pipeline.add(remote_video_sink)
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
915 elif isinstance(self.sinks_data, SinksAuto):
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
916 # if ``self.merge_pip`` is not set, we create a dedicated
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
917 # ``autovideosink`` for remote stream.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
918 remote_video_sink = Gst.ElementFactory.make("autovideosink")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
919 self.pipeline.add(remote_video_sink)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
920 else:
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
921 raise exceptions.InternalError(
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
922 f'Unhandled "sinks_data" value: {self.sinks_data!r}'
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
923 )
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
924
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
925 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
926 videoscale = Gst.ElementFactory.make("videoscale")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
927 adjusted_caps = Gst.Caps.from_string(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
928 f"video/x-raw,width={width},height={height}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
929 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
930 capsfilter = Gst.ElementFactory.make("capsfilter")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
931 capsfilter.set_property("caps", adjusted_caps)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
932
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
933 self.pipeline.add(q, conv, videoscale, capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
934
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
935 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
936 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
937 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
938 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
939 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
940 conv.link(videoscale)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
941 videoscale.link(capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
942 self.link_element_or_pad(capsfilter.link, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
943
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
944 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
945 self.pipeline.add(q, conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
946
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
947 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
948 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
949 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
950 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
951 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
952 self.link_element_or_pad(conv, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
953
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
954 elif name.startswith("audio"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
955 log.debug("===> Audio OK")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
956 conv = Gst.ElementFactory.make("audioconvert")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
957 resample = Gst.ElementFactory.make("audioresample")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
958 remote_audio_sink = Gst.ElementFactory.make("autoaudiosink")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
959 self.pipeline.add(q, conv, resample, remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
960 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
961 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
962 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
963 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
964 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
965 conv.link(resample)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
966 resample.link(remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
967
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
968 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
969 log.warning(f"unmanaged name: {name!r}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
970
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
971 def on_pad_added(self, __, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
972 """Handle the addition of a new pad to the element.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
973
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
974 When a new source pad is added to the element, this method creates a decodebin,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
975 connects it to handle the stream, and links the pad to the decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
976
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
977 @param __: Placeholder for the signal source. Not used in this method.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
978 @param pad: The newly added pad.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
979 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
980 log.debug("on_pad_added")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
981 if pad.direction != Gst.PadDirection.SRC:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
982 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
983
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
984 decodebin = Gst.ElementFactory.make("decodebin")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
985 decodebin.connect("pad-added", self.on_remote_decodebin_stream)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
986 self.pipeline.add(decodebin)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
987 decodebin.sync_state_with_parent()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
988 pad.link(decodebin.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
989
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
990 async def _call_start(self, callee: jid.JID, call_data: dict, profile: str) -> str:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
991 return await self.bridge.call_start(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
992 str(self.callee), data_format.serialise({"sdp": self.offer}), self.profile
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
993 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
994
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
995 async def _start_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
996 """Initiate the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
997
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
998 Initiates a call with the callee using the stored offer. If there are any buffered
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
999 local ICE candidates, they are sent as part of the initiation.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1000 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1001 assert self.callee
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1002 assert self.call_start_cb is not None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1003 self.sid = await self.call_start_cb(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1004 self.callee, {"sdp": self.offer}, self.profile
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1005 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1006 if self.local_candidates_buffer:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1007 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1008 f"sending buffered local ICE candidates: {self.local_candidates_buffer}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1009 )
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1010 if not self.pwd:
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1011 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1012 self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1013 ice_data = {}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1014 for media_type, candidates in self.local_candidates_buffer.items():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1015 ice_data[media_type] = {
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1016 "ufrag": self.ufrag[media_type],
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1017 "pwd": self.pwd[media_type],
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1018 "candidates": candidates,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1019 }
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1020 await self.bridge.ice_candidates_add(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1021 self.sid, data_format.serialise(ice_data), self.profile
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1022 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1023 self.local_candidates_buffer.clear()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1024
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1025 def _remote_sdp_set(self, promise) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1026 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1027 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1028
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1029 def on_accepted_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1030 """Outgoing call has been accepted.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1031
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1032 @param sdp: The SDP answer string received from the other party.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1033 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1034 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1035 log.debug(f"SDP answer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1036
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1037 __, sdpmsg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1038 answer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1039 GstWebRTC.WebRTCSDPType.ANSWER, sdpmsg
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1040 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1041 promise = Gst.Promise.new_with_change_func(self._remote_sdp_set)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1042 self.webrtcbin.emit("set-remote-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1043
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1044 async def answer_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1045 """Answer an incoming call
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1046
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1047 @param sdp: The SDP offer string received from the initiator.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1048 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1049
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1050 @raise AssertionError: Raised when either "VP8" or "OPUS" is not present in
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1051 payload types.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1052 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1053 log.debug(f"SDP offer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1054 self._set_media_types(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1055 __, offer_sdp_msg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1056 payload_types = self.get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1057 offer_sdp_msg, video_encoding="VP8", audio_encoding="OPUS"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1058 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1059 assert "VP8" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1060 assert "OPUS" in payload_types
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1061 try:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1062 await self.setup_call(
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1063 "responder", audio_pt=payload_types["OPUS"], video_pt=payload_types["VP8"]
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1064 )
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1065 except Exception:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1066 log.exception("Can't setup call")
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1067 raise
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1068 self.start_pipeline()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1069 offer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1070 GstWebRTC.WebRTCSDPType.OFFER, offer_sdp_msg
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1071 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1072 promise = Gst.Promise.new_with_change_func(self.on_offer_set)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1073 self.webrtcbin.emit("set-remote-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1074
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1075 def on_ice_candidate(self, webrtc, mline_index, candidate_sdp):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1076 """Handles the on-ice-candidate signal of webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1077
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1078 @param webrtc: The webrtcbin element.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1079 @param mlineindex: The mline index.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1080 @param candidate: The ICE candidate.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1081 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1082 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1083 f"Local ICE candidate. MLine Index: {mline_index}, Candidate: {candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1084 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1085 parsed_candidate = self.parse_ice_candidate(candidate_sdp)
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1086 if parsed_candidate is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1087 log.warning(f"Can't parse candidate: {candidate_sdp}")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1088 return
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1089 try:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1090 media_type = self.media_types[mline_index]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1091 except KeyError:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1092 raise exceptions.InternalError("can't find media type")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1093
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1094 if self.sid is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1095 log.debug("buffering local ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1096 self.local_candidates_buffer.setdefault(media_type, []).append(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1097 parsed_candidate
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1098 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1099 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1100 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1101 assert sdp is not None
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1102 self.extract_ufrag_pwd(sdp)
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1103 ice_data = {
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1104 "ufrag": self.ufrag[media_type],
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1105 "pwd": self.pwd[media_type],
4270
0d7bb4df2343 Reformatted code base using black.
Goffi <goffi@goffi.org>
parents: 4240
diff changeset
1106 "candidates": [parsed_candidate],
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1107 }
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1108 self._a_call(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1109 self.bridge.ice_candidates_add,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1110 self.sid,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1111 data_format.serialise({media_type: ice_data}),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1112 self.profile,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1113 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1114
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1115 def on_ice_candidates_new(self, candidates: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1116 """Handle new ICE candidates.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1117
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1118 @param candidates: A dictionary containing media types ("audio" or "video") as
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1119 keys and corresponding ICE data as values.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1120
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1121 @raise exceptions.InternalError: Raised when sdp mline index is not found.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1122 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1123 if not self.sdp_set:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1124 log.debug("buffering remote ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1125 for media_type in ("audio", "video"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1126 media_candidates = candidates.get(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1127 if media_candidates:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1128 buffer = self.remote_candidates_buffer[media_type]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1129 buffer["candidates"].extend(media_candidates["candidates"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1130 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1131 for media_type, ice_data in candidates.items():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1132 for candidate in ice_data["candidates"]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1133 candidate_sdp = self.build_ice_candidate(candidate)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1134 try:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1135 mline_index = self.get_sdp_mline_index(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1136 except Exception as e:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1137 raise exceptions.InternalError(f"Can't find sdp mline index: {e}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1138 self.webrtcbin.emit("add-ice-candidate", mline_index, candidate_sdp)
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1139 log.warning(
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1140 f"Remote ICE candidate added. MLine Index: {mline_index}, "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1141 f"{candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1142 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1143
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1144 def on_ice_gathering_state_change(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1145 state = self.webrtcbin.get_property("ice-gathering-state")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1146 log.debug(f"ICE gathering state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1147
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1148 def on_ice_connection_state(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1149 state = self.webrtcbin.props.ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1150 if state == GstWebRTC.WebRTCICEConnectionState.FAILED:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1151 log.error("ICE connection failed")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1152 log.info(f"ICE connection state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1153
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1154 def on_bus_error(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1155 """Handles the GStreamer bus error messages.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1156
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1157 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1158 @param message: The error message.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1159 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1160 err, debug = message.parse_error()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1161 log.error(f"Error from {message.src.get_name()}: {err.message}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1162 log.error(f"Debugging info: {debug}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1163
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1164 def on_bus_eos(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1165 """Handles the GStreamer bus eos messages.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1166
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1167 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1168 @param message: The eos message.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1169 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1170 log.info("End of stream")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1171
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1172 def on_audio_mute(self, muted: bool) -> None:
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1173 """Handles (un)muting of audio.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1174
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1175 @param muted: True if audio is muted.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1176 """
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1177 if self.audio_valve is not None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1178 self.audio_valve.set_property("drop", muted)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1179 state = "muted" if muted else "unmuted"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1180 log.info(f"audio is now {state}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1181
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1182 def on_video_mute(self, muted: bool) -> None:
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1183 """Handles (un)muting of video.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1184
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1185 @param muted: True if video is muted.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1186 """
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1187 if self.video_selector is not None:
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1188 current_source = (
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1189 None if muted else "desktop" if self.desktop_sharing else "video"
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1190 )
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1191 self.switch_video_source(current_source)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1192 state = "muted" if muted else "unmuted"
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1193 log.info(f"Video is now {state}")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1194
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1195 def on_desktop_switch(self, desktop_active: bool) -> None:
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1196 """Switches the video source between desktop and video.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1197
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1198 @param desktop_active: True if desktop must be active. False for video.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1199 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1200 if desktop_active and self.desktop_portal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1201 aio.run_async(self.on_desktop_switch_portal(desktop_active))
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1202 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1203 self.do_desktop_switch(desktop_active)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1204
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1205 async def on_desktop_switch_portal(self, desktop_active: bool) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1206 """Call freedesktop screenshare portal and the activate the shared stream"""
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1207 assert self.desktop_portal is not None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1208 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1209 screenshare_data = await self.desktop_portal.request_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1210 except exceptions.CancelError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1211 self.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1212 return
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1213 self.desktop_sharing_data = {"path": str(screenshare_data["node_id"])}
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1214 self.do_desktop_switch(desktop_active)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1215
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1216 def on_portal_session_closed(self) -> None:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1217 self.desktop_sharing = False
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1218
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1219 def do_desktop_switch(self, desktop_active: bool) -> None:
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1220 if self.video_muted:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1221 # Update the active source state but do not switch
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1222 self.desktop_sharing = desktop_active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1223 return
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1224
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1225 source = "desktop" if desktop_active else "video"
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1226 self.switch_video_source(source)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1227 self.desktop_sharing = desktop_active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1228
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1229 def switch_video_source(self, source: str | None) -> None:
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1230 """Activates the specified source while deactivating the others.
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1231
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1232 @param source: 'desktop', 'video', 'muted' or None for muted source.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1233 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1234 if source is None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1235 source = "muted"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1236 if source not in ["video", "muted", "desktop"]:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1237 raise ValueError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1238 f"Invalid source: {source!r}, use one of {'video', 'muted', 'desktop'}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1239 )
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1240
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1241 self.pipeline.set_state(Gst.State.PAUSED)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1242
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1243 # Create a new desktop source if necessary
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1244 if source == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1245 self._setup_desktop_source(self.desktop_sharing_data)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1246
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1247 # Activate the chosen source and deactivate the others
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1248 for src_name in ["video", "muted", "desktop"]:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1249 src_element = self.pipeline.get_by_name(f"{src_name}_src")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1250 if src_name == source:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1251 if src_element:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1252 src_element.set_state(Gst.State.PLAYING)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1253 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1254 if src_element:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1255 if src_name == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1256 self._remove_desktop_source(src_element)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1257 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1258 src_element.set_state(Gst.State.NULL)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1259
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1260 # Set the video_selector active pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1261 if source == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1262 if self.desktop_sink_pad:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1263 pad = self.desktop_sink_pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1264 else:
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1265 log.error(f"No desktop pad available")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1266 pad = None
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1267 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1268 pad_name = f"sink_{['video', 'muted'].index(source)}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1269 pad = self.video_selector.get_static_pad(pad_name)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1270
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1271 if pad is not None:
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1272 self.video_selector.props.active_pad = pad
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1273
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1274 self.pipeline.set_state(Gst.State.PLAYING)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1275
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1276 def _setup_desktop_source(self, properties: dict[str, object] | None) -> None:
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1277 """Set up a new desktop source.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1278
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1279 @param properties: The properties to set on the desktop source.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1280 """
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1281 source_elt = "ximagesrc" if self.desktop_portal is None else "pipewiresrc"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1282 desktop_src = Gst.ElementFactory.make(source_elt, "desktop_src")
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1283 if properties is None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1284 properties = {}
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1285 for key, value in properties.items():
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1286 log.debug(f"setting {source_elt} property: {key!r}={value!r}")
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1287 desktop_src.set_property(key, value)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1288 video_convert = Gst.ElementFactory.make("videoconvert", "desktop_videoconvert")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1289 queue = Gst.ElementFactory.make("queue", "desktop_queue")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1290 queue.set_property("leaky", "downstream")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1291
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1292 self.pipeline.add(desktop_src)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1293 self.pipeline.add(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1294 self.pipeline.add(queue)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1295
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1296 desktop_src.link(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1297 video_convert.link(queue)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1298
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1299 sink_pad_template = self.video_selector.get_pad_template("sink_%u")
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1300 self.desktop_sink_pad = self.video_selector.request_pad(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1301 sink_pad_template, None, None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1302 )
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1303 queue_src_pad = queue.get_static_pad("src")
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1304 queue_src_pad.link(self.desktop_sink_pad)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1305
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1306 desktop_src.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1307 video_convert.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1308 queue.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1309
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1310 def _remove_desktop_source(self, desktop_src: Gst.Element) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1311 """Remove the desktop source from the pipeline.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1312
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1313 @param desktop_src: The desktop source to remove.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1314 """
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1315 # Remove elements for the desktop source
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1316 video_convert = self.pipeline.get_by_name("desktop_videoconvert")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1317 queue = self.pipeline.get_by_name("desktop_queue")
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1318
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1319 if video_convert:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1320 video_convert.set_state(Gst.State.NULL)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1321 desktop_src.unlink(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1322 self.pipeline.remove(video_convert)
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1323
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1324 if queue:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1325 queue.set_state(Gst.State.NULL)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1326 self.pipeline.remove(queue)
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1327
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1328 desktop_src.set_state(Gst.State.NULL)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1329 self.pipeline.remove(desktop_src)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1330
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1331 # Release the pad associated with the desktop source
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1332 if self.desktop_sink_pad:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1333 self.video_selector.release_request_pad(self.desktop_sink_pad)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1334 self.desktop_sink_pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1335
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1336 if self.desktop_portal is not None:
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1337 self.desktop_portal.end_session()
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1338
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1339 async def end_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1340 """Stop streaming and clean instance"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1341 self.reset_instance()
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1342
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1343
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1344 class WebRTCCall:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1345 """Helper class to create and handle WebRTC.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1346
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1347 This class handles signals and communication of connection data with backend.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1348
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1349 """
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1350
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1351 def __init__(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1352 self,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1353 bridge,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1354 profile: str,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1355 callee: jid.JID,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1356 on_call_setup_cb: Callable | None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1357 on_call_ended_cb: Callable | None = None,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1358 **kwargs,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1359 ):
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1360 """Create and setup a webRTC instance
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1361
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1362 @param bridge: async Bridge.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1363 @param profile: profile making or receiving the call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1364 @param callee: peer jid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1365 @param kwargs: extra kw args to use when instantiating WebRTC
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1366 """
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1367 self.profile = profile
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1368 self.webrtc = WebRTC(bridge, profile, **kwargs)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1369 self.webrtc.callee = callee
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1370 self.on_call_setup_cb = on_call_setup_cb
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1371 self.on_call_ended_cb = on_call_ended_cb
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1372 bridge.register_signal("ice_candidates_new", self.on_ice_candidates_new, "plugin")
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1373 bridge.register_signal("call_setup", self.on_call_setup, "plugin")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1374 bridge.register_signal("call_ended", self.on_call_ended, "plugin")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1375
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1376 @classmethod
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1377 async def make_webrtc_call(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1378 cls, bridge, profile: str, call_data: CallData, **kwargs
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1379 ) -> "WebRTCCall":
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1380 """Create the webrtc_call instance
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1381
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1382 @param call_data: Call data of the command
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1383 @param kwargs: extra args used to instanciate WebRTCCall
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1384
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1385 """
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1386 webrtc_call = cls(bridge, profile, call_data.callee, **call_data.kwargs, **kwargs)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1387 if call_data.sid is None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1388 # we are making the call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1389 await webrtc_call.start()
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1390 else:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1391 # we are receiving the call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1392 webrtc_call.sid = call_data.sid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1393 if call_data.action_id is not None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1394 await bridge.action_launch(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1395 call_data.action_id,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1396 data_format.serialise({"cancelled": False}),
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1397 profile,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1398 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1399 return webrtc_call
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1400
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1401 @property
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1402 def sid(self) -> str | None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1403 return self.webrtc.sid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1404
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1405 @sid.setter
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1406 def sid(self, new_sid: str | None) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1407 self.webrtc.sid = new_sid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1408
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1409 async def on_ice_candidates_new(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1410 self, sid: str, candidates_s: str, profile: str
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1411 ) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1412 if sid != self.webrtc.sid or profile != self.profile:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1413 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1414 self.webrtc.on_ice_candidates_new(
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1415 data_format.deserialise(candidates_s),
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1416 )
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1417
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1418 async def on_call_setup(self, sid: str, setup_data_s: str, profile: str) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1419 if sid != self.webrtc.sid or profile != self.profile:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1420 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1421 setup_data = data_format.deserialise(setup_data_s)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1422 try:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1423 role = setup_data["role"]
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1424 sdp = setup_data["sdp"]
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1425 except KeyError:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1426 log.error(f"Invalid setup data received: {setup_data}")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1427 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1428 if role == "initiator":
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1429 self.webrtc.on_accepted_call(sdp, profile)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1430 elif role == "responder":
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1431 await self.webrtc.answer_call(sdp, profile)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1432 else:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1433 log.error(f"Invalid role received during setup: {setup_data}")
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1434 if self.on_call_setup_cb is not None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1435 await aio.maybe_async(self.on_call_setup_cb(sid, profile))
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1436
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1437 async def on_call_ended(self, sid: str, data_s: str, profile: str) -> None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1438 if sid != self.webrtc.sid or profile != self.profile:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1439 return
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1440 await self.webrtc.end_call()
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1441 if self.on_call_ended_cb is not None:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1442 await aio.maybe_async(self.on_call_ended_cb(sid, profile))
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1443
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1444 async def start(self):
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1445 """Start a call.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1446
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1447 To be used only if we are initiator
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1448 """
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1449 try:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1450 await self.webrtc.setup_call("initiator")
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1451 except Exception:
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1452 log.exception("Can't setup call")
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
1453 raise
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents: 4209
diff changeset
1454 self.webrtc.start_pipeline()