Mercurial > libervia-backend

annotate libervia/frontends/tools/webrtc.py @ 4219:1b5cf2ee1d86

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
plugin XEP-0384, XEP-0391: download missing devices list: when a peer jid was not in our roster, devices list was not retrieved, resulting in failed en/decryption. This patch does check it and download missing devices list in necessary. There is no subscription managed yet, so the list won't be updated in case of new devices, this should be addressed at some point.
author Goffi <goffi@goffi.org>
date Tue, 05 Mar 2024 17:31:36 +0100
parents fe29fbdabce6
children d01b8d002619
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1 #!/usr/bin/env python3
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
2
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
3 # Libervia WebRTC implementation
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # Copyright (C) 2009-2023 Jérôme Poisson (goffi@goffi.org)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
5
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 # This program is free software: you can redistribute it and/or modify
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # it under the terms of the GNU Affero General Public License as published by
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # the Free Software Foundation, either version 3 of the License, or
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # (at your option) any later version.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
10
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
11 # This program is distributed in the hope that it will be useful,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # GNU Affero General Public License for more details.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
15
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 # You should have received a copy of the GNU Affero General Public License
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
18
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
19 import gi
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
20 gi.require_versions({
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
21 "Gst": "1.0",
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
22 "GstWebRTC": "1.0"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
23 })
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
24 from gi.repository import Gst, GstWebRTC, GstSdp
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
25
4145
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
26 from libervia.backend.core import exceptions
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
27
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
28 try:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
29 from gi.overrides import Gst as _
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
30 except ImportError:
4145
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
31 raise exceptions.MissingModule(
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
32 "No GStreamer Python overrides available. Please install relevant packages on "
970b6209526a frontends (tools/webrtc): Better message on Python override missing:
Goffi <goffi@goffi.org>
parents: 4139
diff changeset
33 "your system (e.g., `python3-gst-1.0` on Debian and derivatives)."
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
34 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
35 import asyncio
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
36 from dataclasses import dataclass
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
37 from datetime import datetime
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
38 import logging
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
39 from random import randint
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
40 import re
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
41 from typing import Callable
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
42 from urllib.parse import quote_plus
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
43
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
44 from libervia.backend.tools.common import data_format
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
45 from libervia.frontends.tools import aio, display_servers
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
46
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
47 current_server = display_servers.detect()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
48 if current_server == display_servers.X11:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
49 # GSTreamer's ximagesrc documentation asks to run this function
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
50 import ctypes
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
51 ctypes.CDLL('libX11.so.6').XInitThreads()
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
52
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
53
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
54 log = logging.getLogger(__name__)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
55
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
56 Gst.init(None)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
57
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
58 SOURCES_AUTO = "auto"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
59 SOURCES_TEST = "test"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
60 SINKS_APP = "app"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
61 SINKS_AUTO = "auto"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
62 SINKS_TEST = "test"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
63
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
64
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
65 class ScreenshareError(Exception):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
66 pass
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
67
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
68
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
69 @dataclass
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
70 class AppSinkData:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
71 local_video_cb: Callable
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
72 remote_video_cb: Callable|None
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
73
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
74
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
75 class DesktopPortal:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
76
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
77 def __init__(self, webrtc: "WebRTC"):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
78 import dbus
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
79 from dbus.mainloop.glib import DBusGMainLoop
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
80 # we want monitors + windows, see https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html#org-freedesktop-portal-screencast-availablesourcetypes
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
81 self.dbus = dbus
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
82 self.webrtc = webrtc
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
83 self.sources_type = dbus.UInt32(7)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
84 DBusGMainLoop(set_as_default=True)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
85 self.session_bus = dbus.SessionBus()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
86 portal_object = self.session_bus.get_object(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
87 'org.freedesktop.portal.Desktop',
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
88 '/org/freedesktop/portal/desktop'
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
89 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
90 self.screencast_interface = dbus.Interface(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
91 portal_object,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
92 'org.freedesktop.portal.ScreenCast'
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
93 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
94 self.session_interface = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
95 self.session_signal = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
96 self.handle_counter = 0
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
97 self.session_handle = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
98 self.stream_data: dict|None = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
99
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
100 @property
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
101 def handle_token(self):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
102 self.handle_counter += 1
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
103 return f"libervia{self.handle_counter}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
104
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
105 def on_session_closed(self, details: dict) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
106 if self.session_interface is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
107 self.session_interface = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
108 self.webrtc.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
109 if self.session_signal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
110 self.session_signal.remove()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
111 self.session_signal = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
112
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
113
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
114 async def dbus_call(self, method_name: str, *args) -> dict:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
115 """Call a screenshare portal method
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
116
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
117 This method handle the signal response.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
118 @param method_name: method to call
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
119 @param args: extra args
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
120 `handle_token` will be automatically added to the last arg (option dict)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
121 @return: method result
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
122 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
123 if self.session_handle is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
124 self.end_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
125 method = getattr(self.screencast_interface, method_name)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
126 options = args[-1]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
127 reply_fut = asyncio.Future()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
128 signal_fut = asyncio.Future()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
129 # cf. https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.Request.html
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
130 handle_token = self.handle_token
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
131 sender = self.session_bus.get_unique_name().replace(".", "_")[1:]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
132 path = f"/org/freedesktop/portal/desktop/request/{sender}/{handle_token}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
133 signal_match = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
134
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
135 def on_signal(response, results):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
136 assert signal_match is not None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
137 signal_match.remove()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
138 if response == 0:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
139 signal_fut.set_result(results)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
140 elif response == 1:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
141 signal_fut.set_exception(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
142 exceptions.CancelError("Cancelled by user.")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
143 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
144 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
145 signal_fut.set_exception(ScreenshareError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
146 f"Can't get signal result"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
147 ))
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
148
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
149 signal_match = self.session_bus.add_signal_receiver(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
150 on_signal,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
151 signal_name="Response",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
152 dbus_interface="org.freedesktop.portal.Request",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
153 path=path
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
154 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
155
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
156 options["handle_token"] = handle_token
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
157
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
158 method(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
159 *args,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
160 reply_handler=reply_fut.set_result,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
161 error_handler=reply_fut.set_exception
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
162 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
163 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
164 await reply_fut
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
165 except Exception as e:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
166 raise ScreenshareError(f"Can't ask screenshare permission: {e}")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
167 return await signal_fut
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
168
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
169 async def request_screenshare(self) -> dict:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
170 session_data = await self.dbus_call(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
171 "CreateSession",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
172 {
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
173 "session_handle_token": str(randint(1, 2**32)),
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
174 }
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
175 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
176 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
177 session_handle = session_data["session_handle"]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
178 except KeyError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
179 raise ScreenshareError("Can't get session handle")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
180 self.session_handle = session_handle
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
181
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
182
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
183 await self.dbus_call(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
184 "SelectSources",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
185 session_handle,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
186 {
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
187 "multiple": True,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
188 "types": self.sources_type,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
189 "modal": True
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
190 }
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
191 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
192 screenshare_data = await self.dbus_call(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
193 "Start",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
194 session_handle,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
195 "",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
196 {}
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
197 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
198
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
199 session_object = self.session_bus.get_object(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
200 'org.freedesktop.portal.Desktop',
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
201 session_handle
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
202 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
203 self.session_interface = self.dbus.Interface(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
204 session_object,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
205 'org.freedesktop.portal.Session'
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
206 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
207
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
208 self.session_signal = self.session_bus.add_signal_receiver(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
209 self.on_session_closed,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
210 signal_name="Closed",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
211 dbus_interface="org.freedesktop.portal.Session",
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
212 path=session_handle
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
213 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
214
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
215 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
216 node_id, stream_data = screenshare_data["streams"][0]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
217 source_type = int(stream_data["source_type"])
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
218 except (IndexError, KeyError):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
219 raise ScreenshareError("Can't parse stream data")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
220 self.stream_data = stream_data = {
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
221 "session_handle": session_handle,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
222 "node_id": node_id,
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
223 "source_type": source_type
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
224 }
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
225 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
226 height = int(stream_data["size"][0])
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
227 weight = int(stream_data["size"][1])
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
228 except (IndexError, KeyError):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
229 pass
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
230 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
231 stream_data["size"] = (height, weight)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
232
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
233 return self.stream_data
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
234
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
235 def end_screenshare(self) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
236 """Close a running screenshare session, if any."""
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
237 if self.session_interface is None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
238 return
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
239 self.session_interface.Close()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
240 self.on_session_closed({})
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
241
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
242
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
243 class WebRTC:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
244 """GSTreamer based WebRTC implementation for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
245
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
246 This class encapsulates the WebRTC functionalities required for initiating and
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
247 handling audio and video calls.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
248 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
249
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
250 def __init__(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
251 self,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
252 bridge,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
253 profile: str,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
254 sources: str = SOURCES_AUTO,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
255 sinks: str = SINKS_AUTO,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
256 appsink_data: AppSinkData | None = None,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
257 reset_cb: Callable | None = None,
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
258 merge_pip: bool|None = None,
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
259 target_size: tuple[int, int]|None = None,
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
260 ) -> None:
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
261 """Initializes a new WebRTC instance.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
262
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
263 @param bridge: An instance of backend bridge.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
264 @param profile: Libervia profile.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
265 @param sources: Which kind of source to use.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
266 @param sinks: Which kind of sinks to use.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
267 @param appsink_data: configuration data for appsink (when SINKS_APP is used). Must
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
268 not be used for other sinks.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
269 @param reset_cb: An optional Callable that is triggered on reset events. Can be
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
270 used to reset UI data on new calls.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
271 @param merge_pip: A boolean flag indicating whether Picture-in-Picture mode is
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
272 enabled. When PiP is used, local feedback is merged to remote video stream.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
273 Only one video stream is then produced (the local one).
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
274 If None, PiP mode is selected automatically according to selected sink (it's
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
275 used for SINKS_AUTO only for now).
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
276 @param target_size: Expected size of the final sink stream. Mainly use by composer
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
277 when ``merge_pip`` is set.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
278 None to autodetect (not real autodetection implemeted yet, default to
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
279 (1280,720)).
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
280 """
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
281 self.main_loop = asyncio.get_event_loop()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
282 self.bridge = bridge
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
283 self.profile = profile
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
284 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
285 self._audio_muted = False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
286 self._video_muted = False
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
287 self._desktop_sharing = False
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
288 self.desktop_sharing_data = None
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
289 self.sources = sources
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
290 self.sinks = sinks
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
291 if target_size is None:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
292 target_size=(1280, 720)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
293 self.target_width, self.target_height = target_size
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
294 if merge_pip is None:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
295 merge_pip = sinks == SINKS_AUTO
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
296 self.merge_pip = merge_pip
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
297 if sinks == SINKS_APP:
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
298 if (
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
299 merge_pip
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
300 and appsink_data is not None
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
301 and appsink_data.remote_video_cb is not None
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
302 ):
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
303 raise ValueError("Remote_video_cb can't be used when merge_pip is used!")
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
304 self.appsink_data = appsink_data
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
305 elif appsink_data is not None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
306 raise exceptions.InternalError(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
307 "appsink_data can only be used for SINKS_APP sinks"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
308 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
309 self.reset_cb = reset_cb
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
310 if current_server == display_servers.WAYLAND:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
311 self.desktop_portal = DesktopPortal(self)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
312 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
313 self.desktop_portal = None
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
314 self.reset_instance()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
315
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
316 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
317 def audio_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
318 return self._audio_muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
319
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
320 @audio_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
321 def audio_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
322 if muted != self._audio_muted:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
323 self._audio_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
324 self.on_audio_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
325
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
326 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
327 def video_muted(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
328 return self._video_muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
329
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
330 @video_muted.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
331 def video_muted(self, muted: bool) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
332 if muted != self._video_muted:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
333 self._video_muted = muted
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
334 self.on_video_mute(muted)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
335
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
336 @property
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
337 def desktop_sharing(self):
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
338 return self._desktop_sharing
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
339
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
340 @desktop_sharing.setter
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
341 def desktop_sharing(self, active: bool) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
342 if active != self._desktop_sharing:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
343 self._desktop_sharing = active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
344 self.on_desktop_switch(active)
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
345 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
346 cb = self.bindings["desktop_sharing"]
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
347 except KeyError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
348 pass
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
349 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
350 cb(active)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
351
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
352 @property
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
353 def sdp_set(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
354 return self._sdp_set
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
355
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
356 @sdp_set.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
357 def sdp_set(self, is_set: bool):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
358 self._sdp_set = is_set
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
359 if is_set:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
360 self.on_ice_candidates_new(self.remote_candidates_buffer)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
361 for data in self.remote_candidates_buffer.values():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
362 data["candidates"].clear()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
363
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
364 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
365 def media_types(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
366 if self._media_types is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
367 raise Exception("self._media_types should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
368 return self._media_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
369
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
370 @media_types.setter
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
371 def media_types(self, new_media_types: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
372 self._media_types = new_media_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
373 self._media_types_inv = {v: k for k, v in new_media_types.items()}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
374
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
375 @property
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
376 def media_types_inv(self) -> dict:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
377 if self._media_types_inv is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
378 raise Exception("self._media_types_inv should not be None!")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
379 return self._media_types_inv
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
380
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
381 def bind(self, **kwargs: Callable) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
382 self.bindings.clear()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
383 for key, cb in kwargs.items():
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
384 if key not in ("desktop_sharing",):
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
385 raise ValueError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
386 'Only "desktop_sharing" is currently allowed for binding'
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
387 )
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
388 self.bindings[key] = cb
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
389
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
390
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
391 def generate_dot_file(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
392 self,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
393 filename: str = "pipeline",
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
394 details: Gst.DebugGraphDetails = Gst.DebugGraphDetails.ALL,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
395 with_timestamp: bool = True,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
396 bin_: Gst.Bin|None = None,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
397 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
398 """Generate Dot File for debugging
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
399
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
400 ``GST_DEBUG_DUMP_DOT_DIR`` environment variable must be set to destination dir.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
401 ``dot -Tpng -o <filename>.png <filename>.dot`` can be use to convert to a PNG file.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
402 See
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
403 https://gstreamer.freedesktop.org/documentation/gstreamer/debugutils.html?gi-language=python#GstDebugGraphDetails
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
404 for details.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
405
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
406 @param filename: name of the generated file
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
407 @param details: which details to print
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
408 @param with_timestamp: if True, add a timestamp to filename
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
409 @param bin_: which bin to output. By default, the whole pipeline
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
410 (``self.pipeline``) will be used.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
411 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
412 if bin_ is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
413 bin_ = self.pipeline
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
414 if with_timestamp:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
415 timestamp = datetime.now().isoformat(timespec='milliseconds')
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
416 filename = f"{timestamp}_filename"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
417
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
418 Gst.debug_bin_to_dot_file(bin_, details, filename)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
419
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
420 def get_sdp_mline_index(self, media_type: str) -> int:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
421 """Gets the sdpMLineIndex for a given media type.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
422
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
423 @param media_type: The type of the media.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
424 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
425 for index, m_type in self.media_types.items():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
426 if m_type == media_type:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
427 return index
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
428 raise ValueError(f"Media type '{media_type}' not found")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
429
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
430 def _set_media_types(self, offer_sdp: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
431 """Sets media types from offer SDP
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
432
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
433 @param offer: RTC session description containing the offer
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
434 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
435 sdp_lines = offer_sdp.splitlines()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
436 media_types = {}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
437 mline_index = 0
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
438
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
439 for line in sdp_lines:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
440 if line.startswith("m="):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
441 media_types[mline_index] = line[2 : line.find(" ")]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
442 mline_index += 1
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
443
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
444 self.media_types = media_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
445
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
446 def _a_call(self, method, *args, **kwargs):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
447 """Call an async method in main thread"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
448 aio.run_from_thread(method, *args, **kwargs, loop=self.main_loop)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
449
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
450 def get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
451 self, sdpmsg, video_encoding: str, audio_encoding: str
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
452 ) -> dict[str, int | None]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
453 """Find the payload types for the specified video and audio encoding.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
454
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
455 Very simplistically finds the first payload type matching the encoding
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
456 name. More complex applications will want to match caps on
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
457 profile-level-id, packetization-mode, etc.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
458 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
459 # method coming from gstreamer example (Matthew Waters, Nirbheek Chauhan) at
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
460 # subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
461 video_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
462 audio_pt = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
463 for i in range(0, sdpmsg.medias_len()):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
464 media = sdpmsg.get_media(i)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
465 for j in range(0, media.formats_len()):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
466 fmt = media.get_format(j)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
467 if fmt == "webrtc-datachannel":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
468 continue
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
469 pt = int(fmt)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
470 caps = media.get_caps_from_media(pt)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
471 s = caps.get_structure(0)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
472 encoding_name = s["encoding-name"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
473 if video_pt is None and encoding_name == video_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
474 video_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
475 elif audio_pt is None and encoding_name == audio_encoding:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
476 audio_pt = pt
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
477 return {video_encoding: video_pt, audio_encoding: audio_pt}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
478
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
479 def parse_ice_candidate(self, candidate_string):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
480 """Parses the ice candidate string.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
481
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
482 @param candidate_string: The ice candidate string to be parsed.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
483 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
484 pattern = re.compile(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
485 r"candidate:(?P<foundation>\S+) (?P<component_id>\d+) (?P<transport>\S+) "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
486 r"(?P<priority>\d+) (?P<address>\S+) (?P<port>\d+) typ "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
487 r"(?P<type>\S+)(?: raddr (?P<rel_addr>\S+) rport "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
488 r"(?P<rel_port>\d+))?(?: generation (?P<generation>\d+))?"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
489 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
490 match = pattern.match(candidate_string)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
491 if match:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
492 candidate_dict = match.groupdict()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
493
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
494 # Apply the correct types to the dictionary values
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
495 candidate_dict["component_id"] = int(candidate_dict["component_id"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
496 candidate_dict["priority"] = int(candidate_dict["priority"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
497 candidate_dict["port"] = int(candidate_dict["port"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
498
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
499 if candidate_dict["rel_port"]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
500 candidate_dict["rel_port"] = int(candidate_dict["rel_port"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
501
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
502 if candidate_dict["generation"]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
503 candidate_dict["generation"] = candidate_dict["generation"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
504
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
505 # Remove None values
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
506 return {k: v for k, v in candidate_dict.items() if v is not None}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
507 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
508 log.warning(f"can't parse candidate: {candidate_string!r}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
509 return None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
510
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
511 def build_ice_candidate(self, parsed_candidate):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
512 """Builds ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
513
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
514 @param parsed_candidate: Dictionary containing parsed ICE candidate
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
515 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
516 base_format = (
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
517 "candidate:{foundation} {component_id} {transport} {priority} "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
518 "{address} {port} typ {type}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
519 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
520
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
521 if parsed_candidate.get("rel_addr") and parsed_candidate.get("rel_port"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
522 base_format += " raddr {rel_addr} rport {rel_port}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
523
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
524 if parsed_candidate.get("generation"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
525 base_format += " generation {generation}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
526
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
527 return base_format.format(**parsed_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
528
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
529 def extract_ufrag_pwd(self, sdp: str) -> tuple[str, str]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
530 """Retrieves ICE password and user fragment for SDP offer.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
531
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
532 @param sdp: The Session Description Protocol offer string.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
533 @return: ufrag and pwd
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
534 @raise ValueError: Can't extract ufrag and password
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
535 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
536 ufrag_line = re.search(r"ice-ufrag:(\S+)", sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
537 pwd_line = re.search(r"ice-pwd:(\S+)", sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
538
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
539 if ufrag_line and pwd_line:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
540 ufrag = self.ufrag = ufrag_line.group(1)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
541 pwd = self.pwd = pwd_line.group(1)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
542 return ufrag, pwd
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
543 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
544 log.error(f"SDP with missing ice-ufrag or ice-pwd:\n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
545 raise ValueError("Can't extract ice-ufrag and ice-pwd from SDP")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
546
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
547 def reset_instance(self):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
548 """Inits or resets the instance variables to their default state."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
549 self.role: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
550 if self.pipeline is not None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
551 self.pipeline.set_state(Gst.State.NULL)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
552 self.pipeline = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
553 self._remote_video_pad = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
554 self.sid: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
555 self.offer: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
556 self.local_candidates_buffer = {}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
557 self.ufrag: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
558 self.pwd: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
559 self.callee: str | None = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
560 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
561 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
562 self._sdp_set: bool = False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
563 self.remote_candidates_buffer: dict[str, dict[str, list]] = {
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
564 "audio": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
565 "video": {"candidates": []},
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
566 }
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
567 self._media_types = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
568 self._media_types_inv = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
569 self.audio_valve = None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
570 self.video_valve = None
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
571 if self.desktop_portal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
572 self.desktop_portal.end_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
573 self.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
574 self.desktop_sink_pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
575 self.bindings = {}
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
576 if self.reset_cb is not None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
577 self.reset_cb()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
578
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
579
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
580 async def setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
581 self,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
582 role: str,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
583 audio_pt: int | None = 96,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
584 video_pt: int | None = 97,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
585 ) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
586 """Sets up the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
587
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
588 This method establishes the Gstreamer pipeline for audio and video communication.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
589 The method also manages STUN and TURN server configurations, signal watchers, and
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
590 various connection handlers for the webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
591
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
592 @param role: The role for the call, either 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
593 @param audio_pt: The payload type for the audio stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
594 @param video_pt: The payload type for the video stream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
595
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
596 @raises NotImplementedError: If audio_pt or video_pt is set to None.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
597 @raises AssertionError: If the role is not 'initiator' or 'responder'.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
598 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
599 assert role in ("initiator", "responder")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
600 self.role = role
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
601 if audio_pt is None or video_pt is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
602 raise NotImplementedError("None value is not handled yet")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
603
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
604 if self.sources == SOURCES_AUTO:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
605 video_source_elt = "v4l2src"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
606 audio_source_elt = "pulsesrc"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
607 elif self.sources == SOURCES_TEST:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
608 video_source_elt = "videotestsrc is-live=true pattern=ball"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
609 audio_source_elt = "audiotestsrc"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
610 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
611 raise exceptions.InternalError(f'Unknown "sources" value: {self.sources!r}')
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
612
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
613
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
614 if self.sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
615 local_video_sink_elt = (
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
616 "appsink name=local_video_sink emit-signals=true drop=true max-buffers=1 "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
617 "sync=True"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
618 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
619 elif self.sinks == SINKS_AUTO:
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
620 local_video_sink_elt = "autovideosink"
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
621 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
622 raise exceptions.InternalError(f"Unknown sinks value {self.sinks!r}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
623
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
624 if self.merge_pip:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
625 extra_elt = (
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
626 "compositor name=compositor background=black "
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
627 f"! video/x-raw,width={self.target_width},height={self.target_height},"
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
628 "framerate=30/1 "
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
629 f"! {local_video_sink_elt}"
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
630 )
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
631 local_video_sink_elt = "compositor.sink_1"
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
632 else:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
633 extra_elt = ""
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
634
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
635 self.gst_pipe_desc = f"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
636 webrtcbin latency=100 name=sendrecv bundle-policy=max-bundle
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
637
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
638 input-selector name=video_selector
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
639 ! videorate
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
640 ! video/x-raw,framerate=30/1
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
641 ! tee name=t
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
642
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
643 {extra_elt}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
644
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
645 {video_source_elt} name=video_src ! queue leaky=downstream ! video_selector.
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
646 videotestsrc name=muted_src is-live=true pattern=black ! queue leaky=downstream ! video_selector.
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
647
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
648 t.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
649 ! queue max-size-buffers=5 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
650 ! videoconvert
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
651 ! vp8enc deadline=1 keyframe-max-dist=60
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
652 ! rtpvp8pay picture-id-mode=15-bit
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
653 ! application/x-rtp,media=video,encoding-name=VP8,payload={video_pt}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
654 ! sendrecv.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
655
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
656 t.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
657 ! queue max-size-buffers=5 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
658 ! videoconvert
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
659 ! {local_video_sink_elt}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
660
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
661 {audio_source_elt} name=audio_src
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
662 ! valve
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
663 ! queue max-size-buffers=10 max-size-time=0 max-size-bytes=0 leaky=downstream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
664 ! audioconvert
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
665 ! audioresample
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
666 ! opusenc audio-type=voice
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
667 ! rtpopuspay
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
668 ! application/x-rtp,media=audio,encoding-name=OPUS,payload={audio_pt}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
669 ! sendrecv.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
670 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
671
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
672 log.debug(f"Gstreamer pipeline: {self.gst_pipe_desc}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
673
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
674 # Create the pipeline
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
675 try:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
676 self.pipeline = Gst.parse_launch(self.gst_pipe_desc)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
677 except Exception:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
678 log.exception("Can't parse pipeline")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
679 self.pipeline = None
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
680 if not self.pipeline:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
681 raise exceptions.InternalError("Failed to create Gstreamer pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
682
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
683 self.webrtcbin = self.pipeline.get_by_name("sendrecv")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
684 self.video_src = self.pipeline.get_by_name("video_src")
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
685 self.muted_src = self.pipeline.get_by_name("muted_src")
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
686 self.video_selector = self.pipeline.get_by_name("video_selector")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
687 self.audio_valve = self.pipeline.get_by_name("audio_valve")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
688
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
689 if self.video_muted:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
690 self.on_video_mute(True)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
691 if self.audio_muted:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
692 self.on_audio_mute(True)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
693
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
694 # set STUN and TURN servers
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
695 external_disco = data_format.deserialise(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
696 await self.bridge.external_disco_get("", self.profile), type_check=list
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
697 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
698
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
699 for server in external_disco:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
700 if server["type"] == "stun":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
701 if server["transport"] == "tcp":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
702 log.info(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
703 "ignoring TCP STUN server, GStreamer only support one STUN server"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
704 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
705 url = f"stun://{server['host']}:{server['port']}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
706 log.debug(f"adding stun server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
707 self.webrtcbin.set_property("stun-server", url)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
708 elif server["type"] == "turn":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
709 url = "{scheme}://{username}:{password}@{host}:{port}".format(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
710 scheme="turns" if server["transport"] == "tcp" else "turn",
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
711 username=quote_plus(server["username"]),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
712 password=quote_plus(server["password"]),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
713 host=server["host"],
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
714 port=server["port"],
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
715 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
716 log.debug(f"adding turn server: {url}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
717
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
718 if not self.webrtcbin.emit("add-turn-server", url):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
719 log.warning(f"Erreur while adding TURN server {url}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
720
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
721 # local video feedback
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
722 if self.sinks == SINKS_APP:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
723 assert self.appsink_data is not None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
724 local_video_sink = self.pipeline.get_by_name("local_video_sink")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
725 local_video_sink.set_property("emit-signals", True)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
726 local_video_sink.connect("new-sample", self.appsink_data.local_video_cb)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
727 local_video_sink_caps = Gst.Caps.from_string(f"video/x-raw,format=RGB")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
728 local_video_sink.set_property("caps", local_video_sink_caps)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
729
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
730 # Create bus and associate signal watchers
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
731 self.bus = self.pipeline.get_bus()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
732 if not self.bus:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
733 log.error("Failed to get bus from pipeline.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
734 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
735
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
736 self.bus.add_signal_watch()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
737 self.webrtcbin.connect("pad-added", self.on_pad_added)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
738 self.bus.connect("message::error", self.on_bus_error)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
739 self.bus.connect("message::eos", self.on_bus_eos)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
740 self.webrtcbin.connect("on-negotiation-needed", self.on_negotiation_needed)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
741 self.webrtcbin.connect("on-ice-candidate", self.on_ice_candidate)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
742 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
743 "notify::ice-gathering-state", self.on_ice_gathering_state_change
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
744 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
745 self.webrtcbin.connect(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
746 "notify::ice-connection-state", self.on_ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
747 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
748
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
749 def start_pipeline(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
750 """Starts the GStreamer pipeline."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
751 log.debug("starting the pipeline")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
752 self.pipeline.set_state(Gst.State.PLAYING)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
753
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
754 def on_negotiation_needed(self, webrtc):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
755 """Initiate SDP offer when negotiation is needed."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
756 log.debug("Negotiation needed.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
757 if self.role == "initiator":
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
758 log.debug("Creating offer…")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
759 promise = Gst.Promise.new_with_change_func(self.on_offer_created)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
760 self.webrtcbin.emit("create-offer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
761
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
762 def on_offer_created(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
763 """Callback for when SDP offer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
764 log.info("on_offer_created called")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
765 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
766 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
767 if reply is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
768 log.error("Promise reply is None. Offer creation might have failed.")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
769 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
770 offer = reply["offer"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
771 self.offer = offer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
772 log.info(f"SDP offer created: \n{self.offer}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
773 self._set_media_types(self.offer)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
774 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
775 self.webrtcbin.emit("set-local-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
776 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
777 self._a_call(self._start_call)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
778
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
779 def on_answer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
780 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
781
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
782 def on_answer_created(self, promise, _, __):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
783 """Callback for when SDP answer is created."""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
784 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
785 reply = promise.get_reply()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
786 answer = reply["answer"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
787 promise = Gst.Promise.new()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
788 self.webrtcbin.emit("set-local-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
789 promise.interrupt()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
790 answer_sdp = answer.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
791 log.info(f"SDP answer set: \n{answer_sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
792 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
793 self._a_call(self.bridge.call_answer_sdp, self.sid, answer_sdp, self.profile)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
794
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
795 def on_offer_set(self, promise):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
796 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
797 promise = Gst.Promise.new_with_change_func(self.on_answer_created, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
798 self.webrtcbin.emit("create-answer", None, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
799
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
800 def link_element_or_pad(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
801 self, source: Gst.Element, dest: Gst.Element | Gst.Pad
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
802 ) -> bool:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
803 """Check if dest is a pad or an element, and link appropriately"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
804 src_pad = source.get_static_pad("src")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
805
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
806 if isinstance(dest, Gst.Pad):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
807 # If the dest is a pad, link directly
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
808 if not src_pad.link(dest) == Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
809 log.error(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
810 "Failed to link 'conv' to the compositor's newly requested pad!"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
811 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
812 return False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
813 elif isinstance(dest, Gst.Element):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
814 return source.link(dest)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
815 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
816 log.error(f"Unexpected type for dest: {type(sink)}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
817 return False
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
818
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
819 return True
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
820
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
821 def scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
822 self, original_width: int, original_height: int, max_width: int, max_height: int
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
823 ) -> tuple[int, int]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
824 """Calculates the scaled dimensions preserving aspect ratio.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
825
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
826 @param original_width: Original width of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
827 @param original_height: Original height of the video stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
828 @param max_width: Maximum desired width for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
829 @param max_height: Maximum desired height for the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
830 @return: The width and height of the scaled video.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
831 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
832 aspect_ratio = original_width / original_height
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
833 new_width = int(max_height * aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
834
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
835 if new_width <= max_width:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
836 return new_width, max_height
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
837
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
838 new_height = int(max_width / aspect_ratio)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
839 return max_width, new_height
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
840
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
841 def on_remote_decodebin_stream(self, _, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
842 """Handle the stream from the remote decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
843
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
844 This method processes the incoming stream from the remote decodebin, determining
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
845 whether it's video or audio. It then sets up the appropriate GStreamer elements
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
846 for video/audio processing and adds them to the pipeline.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
847
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
848 @param pad: The Gst.Pad from the remote decodebin producing the stream.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
849 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
850 assert self.pipeline is not None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
851 if not pad.has_current_caps():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
852 log.error(f"{pad} has no caps, ignoring")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
853 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
854
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
855 caps = pad.get_current_caps()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
856 assert len(caps)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
857 s = caps[0]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
858 name = s.get_name()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
859 log.debug(f"====> NAME START: {name}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
860
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
861 q = Gst.ElementFactory.make("queue")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
862
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
863 if name.startswith("video"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
864 log.debug("===> VIDEO OK")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
865
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
866 self._remote_video_pad = pad
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
867
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
868 # Check and log the original size of the video
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
869 width = self.target_width
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
870 height = self.target_height
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
871 log.info(f"Original video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
872
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
873 # This is a fix for an issue found with Movim on desktop: a non standard
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
874 # resolution is used (990x557) resulting in bad alignement and no color in
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
875 # rendered image
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
876 adjust_resolution = width % 4 != 0 or height % 4 != 0
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
877 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
878 log.info("non standard resolution, we need to adjust size")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
879 width = (width + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
880 height = (height + 3) // 4 * 4
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
881 log.info(f"Adjusted video size: {width}x{height}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
882
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
883 conv = Gst.ElementFactory.make("videoconvert")
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
884 if self.merge_pip:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
885 # with ``merge_pip`` set, we plug the remote stream to the composer
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
886 compositor = self.pipeline.get_by_name("compositor")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
887
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
888 sink1_pad = compositor.get_static_pad("sink_1")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
889
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
890 local_width, local_height = self.scaled_dimensions(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
891 sink1_pad.get_property("width"),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
892 sink1_pad.get_property("height"),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
893 width // 3,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
894 height // 3,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
895 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
896
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
897 sink1_pad.set_property("xpos", width - local_width)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
898 sink1_pad.set_property("ypos", height - local_height)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
899 sink1_pad.set_property("width", local_width)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
900 sink1_pad.set_property("height", local_height)
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
901 sink1_pad.set_property("sizing-policy", 1)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
902 sink1_pad.set_property("zorder", 1)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
903
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
904 # Request a new pad for the remote stream
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
905 sink_pad_template = compositor.get_pad_template("sink_%u")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
906 remote_video_sink = compositor.request_pad(sink_pad_template, None, None)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
907 remote_video_sink.set_property("zorder", 0)
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
908 remote_video_sink.set_property("width", width)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
909 remote_video_sink.set_property("height", height)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
910 remote_video_sink.set_property("sizing-policy", 1)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
911 elif self.sinks == SINKS_APP:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
912 # ``app`` sink without ``self.merge_pip`` set, be create the sink and
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
913 # connect it to the ``remote_video_cb``.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
914 assert self.appsink_data is not None
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
915 remote_video_sink = Gst.ElementFactory.make("appsink")
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
916
4209
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
917 remote_video_caps = Gst.Caps.from_string("video/x-raw,format=RGB")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
918 remote_video_sink.set_property("caps", remote_video_caps)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
919
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
920 remote_video_sink.set_property("emit-signals", True)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
921 remote_video_sink.set_property("drop", True)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
922 remote_video_sink.set_property("max-buffers", 1)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
923 remote_video_sink.set_property("sync", True)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
924 remote_video_sink.connect("new-sample", self.appsink_data.remote_video_cb)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
925 self.pipeline.add(remote_video_sink)
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
926 elif self.sinks == SINKS_AUTO:
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
927 # if ``self.merge_pip`` is not set, we create a dedicated
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
928 # ``autovideosink`` for remote stream.
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
929 remote_video_sink = Gst.ElementFactory.make("autovideosink")
fe29fbdabce6 frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
Goffi <goffi@goffi.org>
parents: 4205
diff changeset
930 self.pipeline.add(remote_video_sink)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
931 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
932 raise exceptions.InternalError(f'Unhandled "sinks" value: {self.sinks!r}')
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
933
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
934 if adjust_resolution:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
935 videoscale = Gst.ElementFactory.make("videoscale")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
936 adjusted_caps = Gst.Caps.from_string(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
937 f"video/x-raw,width={width},height={height}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
938 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
939 capsfilter = Gst.ElementFactory.make("capsfilter")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
940 capsfilter.set_property("caps", adjusted_caps)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
941
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
942 self.pipeline.add(q, conv, videoscale, capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
943
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
944
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
945 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
946 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
947 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
948 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
949 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
950 conv.link(videoscale)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
951 videoscale.link(capsfilter)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
952 self.link_element_or_pad(capsfilter.link, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
953
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
954 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
955 self.pipeline.add(q, conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
956
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
957 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
958 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
959 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
960 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
961 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
962 self.link_element_or_pad(conv, remote_video_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
963
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
964 elif name.startswith("audio"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
965 log.debug("===> Audio OK")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
966 conv = Gst.ElementFactory.make("audioconvert")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
967 resample = Gst.ElementFactory.make("audioresample")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
968 remote_audio_sink = Gst.ElementFactory.make("autoaudiosink")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
969 self.pipeline.add(q, conv, resample, remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
970 self.pipeline.sync_children_states()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
971 ret = pad.link(q.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
972 if ret != Gst.PadLinkReturn.OK:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
973 log.error(f"Error linking pad: {ret}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
974 q.link(conv)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
975 conv.link(resample)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
976 resample.link(remote_audio_sink)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
977
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
978 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
979 log.warning(f"unmanaged name: {name!r}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
980
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
981 def on_pad_added(self, __, pad: Gst.Pad) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
982 """Handle the addition of a new pad to the element.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
983
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
984 When a new source pad is added to the element, this method creates a decodebin,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
985 connects it to handle the stream, and links the pad to the decodebin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
986
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
987 @param __: Placeholder for the signal source. Not used in this method.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
988 @param pad: The newly added pad.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
989 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
990 log.debug("on_pad_added")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
991 if pad.direction != Gst.PadDirection.SRC:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
992 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
993
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
994 decodebin = Gst.ElementFactory.make("decodebin")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
995 decodebin.connect("pad-added", self.on_remote_decodebin_stream)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
996 self.pipeline.add(decodebin)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
997 decodebin.sync_state_with_parent()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
998 pad.link(decodebin.get_static_pad("sink"))
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
999
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1000 async def _start_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1001 """Initiate the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1002
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1003 Initiates a call with the callee using the stored offer. If there are any buffered
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1004 local ICE candidates, they are sent as part of the initiation.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1005 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1006 assert self.callee
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1007 self.sid = await self.bridge.call_start(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1008 str(self.callee), data_format.serialise({"sdp": self.offer}), self.profile
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1009 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1010 if self.local_candidates_buffer:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1011 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1012 f"sending buffered local ICE candidates: {self.local_candidates_buffer}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1013 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1014 if self.pwd is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1015 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1016 self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1017 ice_data = {}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1018 for media_type, candidates in self.local_candidates_buffer.items():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1019 ice_data[media_type] = {
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1020 "ufrag": self.ufrag,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1021 "pwd": self.pwd,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1022 "candidates": candidates,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1023 }
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1024 await self.bridge.ice_candidates_add(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1025 self.sid, data_format.serialise(ice_data), self.profile
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1026 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1027 self.local_candidates_buffer.clear()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1028
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1029 def _remote_sdp_set(self, promise) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1030 assert promise.wait() == Gst.PromiseResult.REPLIED
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1031 self.sdp_set = True
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1032
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1033 def on_accepted_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1034 """Outgoing call has been accepted.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1035
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1036 @param sdp: The SDP answer string received from the other party.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1037 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1038 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1039 log.debug(f"SDP answer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1040
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1041 __, sdpmsg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1042 answer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1043 GstWebRTC.WebRTCSDPType.ANSWER, sdpmsg
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1044 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1045 promise = Gst.Promise.new_with_change_func(self._remote_sdp_set)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1046 self.webrtcbin.emit("set-remote-description", answer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1047
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1048 async def answer_call(self, sdp: str, profile: str) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1049 """Answer an incoming call
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1050
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1051 @param sdp: The SDP offer string received from the initiator.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1052 @param profile: Profile used for the call.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1053
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1054 @raise AssertionError: Raised when either "VP8" or "OPUS" is not present in
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1055 payload types.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1056 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1057 log.debug(f"SDP offer received: \n{sdp}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1058 self._set_media_types(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1059 __, offer_sdp_msg = GstSdp.SDPMessage.new_from_text(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1060 payload_types = self.get_payload_types(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1061 offer_sdp_msg, video_encoding="VP8", audio_encoding="OPUS"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1062 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1063 assert "VP8" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1064 assert "OPUS" in payload_types
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1065 await self.setup_call(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1066 "responder", audio_pt=payload_types["OPUS"], video_pt=payload_types["VP8"]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1067 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1068 self.start_pipeline()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1069 offer = GstWebRTC.WebRTCSessionDescription.new(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1070 GstWebRTC.WebRTCSDPType.OFFER, offer_sdp_msg
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1071 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1072 promise = Gst.Promise.new_with_change_func(self.on_offer_set)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1073 self.webrtcbin.emit("set-remote-description", offer, promise)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1074
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1075 def on_ice_candidate(self, webrtc, mline_index, candidate_sdp):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1076 """Handles the on-ice-candidate signal of webrtcbin.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1077
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1078 @param webrtc: The webrtcbin element.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1079 @param mlineindex: The mline index.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1080 @param candidate: The ICE candidate.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1081 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1082 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1083 f"Local ICE candidate. MLine Index: {mline_index}, Candidate: {candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1084 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1085 parsed_candidate = self.parse_ice_candidate(candidate_sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1086 try:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1087 media_type = self.media_types[mline_index]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1088 except KeyError:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1089 raise exceptions.InternalError("can't find media type")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1090
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1091 if self.sid is None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1092 log.debug("buffering local ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1093 self.local_candidates_buffer.setdefault(media_type, []).append(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1094 parsed_candidate
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1095 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1096 else:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1097 sdp = self.webrtcbin.props.local_description.sdp.as_text()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1098 assert sdp is not None
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1099 ufrag, pwd = self.extract_ufrag_pwd(sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1100 ice_data = {"ufrag": ufrag, "pwd": pwd, "candidates": [parsed_candidate]}
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1101 self._a_call(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1102 self.bridge.ice_candidates_add,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1103 self.sid,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1104 data_format.serialise({media_type: ice_data}),
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1105 self.profile,
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1106 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1107
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1108 def on_ice_candidates_new(self, candidates: dict) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1109 """Handle new ICE candidates.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1110
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1111 @param candidates: A dictionary containing media types ("audio" or "video") as
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1112 keys and corresponding ICE data as values.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1113
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1114 @raise exceptions.InternalError: Raised when sdp mline index is not found.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1115 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1116 if not self.sdp_set:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1117 log.debug("buffering remote ICE candidate")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1118 for media_type in ("audio", "video"):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1119 media_candidates = candidates.get(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1120 if media_candidates:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1121 buffer = self.remote_candidates_buffer[media_type]
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1122 buffer["candidates"].extend(media_candidates["candidates"])
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1123 return
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1124 for media_type, ice_data in candidates.items():
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1125 for candidate in ice_data["candidates"]:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1126 candidate_sdp = self.build_ice_candidate(candidate)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1127 try:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1128 mline_index = self.get_sdp_mline_index(media_type)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1129 except Exception as e:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1130 raise exceptions.InternalError(f"Can't find sdp mline index: {e}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1131 self.webrtcbin.emit("add-ice-candidate", mline_index, candidate_sdp)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1132 log.debug(
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1133 f"Remote ICE candidate added. MLine Index: {mline_index}, "
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1134 f"{candidate_sdp}"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1135 )
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1136
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1137 def on_ice_gathering_state_change(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1138 state = self.webrtcbin.get_property("ice-gathering-state")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1139 log.debug(f"ICE gathering state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1140
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1141 def on_ice_connection_state(self, pspec, __):
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1142 state = self.webrtcbin.props.ice_connection_state
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1143 if state == GstWebRTC.WebRTCICEConnectionState.FAILED:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1144 log.error("ICE connection failed")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1145 log.info(f"ICE connection state changed to {state}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1146
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1147 def on_bus_error(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1148 """Handles the GStreamer bus error messages.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1149
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1150 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1151 @param message: The error message.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1152 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1153 err, debug = message.parse_error()
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1154 log.error(f"Error from {message.src.get_name()}: {err.message}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1155 log.error(f"Debugging info: {debug}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1156
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1157 def on_bus_eos(self, bus: Gst.Bus, message: Gst.Message) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1158 """Handles the GStreamer bus eos messages.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1159
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1160 @param bus: The GStreamer bus.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1161 @param message: The eos message.
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1162 """
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1163 log.info("End of stream")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1164
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1165 def on_audio_mute(self, muted: bool) -> None:
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1166 """Handles (un)muting of audio.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1167
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1168 @param muted: True if audio is muted.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1169 """
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1170 if self.audio_valve is not None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1171 self.audio_valve.set_property("drop", muted)
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1172 state = "muted" if muted else "unmuted"
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1173 log.info(f"audio is now {state}")
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1174
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1175 def on_video_mute(self, muted: bool) -> None:
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1176 """Handles (un)muting of video.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1177
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1178 @param muted: True if video is muted.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1179 """
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1180 if self.video_selector is not None:
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1181 current_source = None if muted else "desktop" if self.desktop_sharing else "video"
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1182 self.switch_video_source(current_source)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1183 state = "muted" if muted else "unmuted"
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1184 log.info(f"Video is now {state}")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1185
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1186 def on_desktop_switch(self, desktop_active: bool) -> None:
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1187 """Switches the video source between desktop and video.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1188
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1189 @param desktop_active: True if desktop must be active. False for video.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1190 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1191 if desktop_active and self.desktop_portal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1192 aio.run_async(self.on_desktop_switch_portal(desktop_active))
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1193 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1194 self.do_desktop_switch(desktop_active)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1195
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1196 async def on_desktop_switch_portal(self, desktop_active: bool) -> None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1197 """Call freedesktop screenshare portal and the activate the shared stream"""
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1198 assert self.desktop_portal is not None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1199 try:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1200 screenshare_data = await self.desktop_portal.request_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1201 except exceptions.CancelError:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1202 self.desktop_sharing = False
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1203 return
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1204 self.desktop_sharing_data = {
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1205 "path": str(screenshare_data["node_id"])
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1206 }
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1207 self.do_desktop_switch(desktop_active)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1208
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1209 def do_desktop_switch(self, desktop_active: bool) -> None:
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1210 if self.video_muted:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1211 # Update the active source state but do not switch
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1212 self.desktop_sharing = desktop_active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1213 return
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1214
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1215 source = "desktop" if desktop_active else "video"
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1216 self.switch_video_source(source)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1217 self.desktop_sharing = desktop_active
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1218
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1219 def switch_video_source(self, source: str|None) -> None:
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1220 """Activates the specified source while deactivating the others.
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1221
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1222 @param source: 'desktop', 'video', 'muted' or None for muted source.
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1223 """
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1224 if source is None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1225 source = "muted"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1226 if source not in ["video", "muted", "desktop"]:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1227 raise ValueError(
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1228 f"Invalid source: {source!r}, use one of {'video', 'muted', 'desktop'}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1229 )
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1230
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1231 self.pipeline.set_state(Gst.State.PAUSED)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1232
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1233 # Create a new desktop source if necessary
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1234 if source == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1235 self._setup_desktop_source(self.desktop_sharing_data)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1236
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1237 # Activate the chosen source and deactivate the others
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1238 for src_name in ["video", "muted", "desktop"]:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1239 src_element = self.pipeline.get_by_name(f"{src_name}_src")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1240 if src_name == source:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1241 if src_element:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1242 src_element.set_state(Gst.State.PLAYING)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1243 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1244 if src_element:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1245 if src_name == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1246 self._remove_desktop_source(src_element)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1247 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1248 src_element.set_state(Gst.State.NULL)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1249
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1250 # Set the video_selector active pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1251 if source == "desktop":
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1252 if self.desktop_sink_pad:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1253 pad = self.desktop_sink_pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1254 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1255 log.error(f"No desktop pad available")
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1256 pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1257 else:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1258 pad_name = f"sink_{['video', 'muted'].index(source)}"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1259 pad = self.video_selector.get_static_pad(pad_name)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1260
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1261 if pad is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1262 self.video_selector.props.active_pad = pad
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1263
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1264 self.pipeline.set_state(Gst.State.PLAYING)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1265
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1266 def _setup_desktop_source(self, properties: dict[str, object]|None) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1267 """Set up a new desktop source.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1268
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1269 @param properties: The properties to set on the desktop source.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1270 """
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1271 source_elt = "ximagesrc" if self.desktop_portal is None else "pipewiresrc"
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1272 desktop_src = Gst.ElementFactory.make(source_elt, "desktop_src")
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1273 if properties is None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1274 properties = {}
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1275 for key, value in properties.items():
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1276 log.debug(f"setting {source_elt} property: {key!r}={value!r}")
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1277 desktop_src.set_property(key, value)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1278 video_convert = Gst.ElementFactory.make("videoconvert", "desktop_videoconvert")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1279 queue = Gst.ElementFactory.make("queue", "desktop_queue")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1280 queue.set_property("leaky", "downstream")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1281
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1282 self.pipeline.add(desktop_src)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1283 self.pipeline.add(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1284 self.pipeline.add(queue)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1285
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1286 desktop_src.link(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1287 video_convert.link(queue)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1288
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1289 sink_pad_template = self.video_selector.get_pad_template("sink_%u")
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1290 self.desktop_sink_pad = self.video_selector.request_pad(sink_pad_template, None, None)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1291 queue_src_pad = queue.get_static_pad("src")
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1292 queue_src_pad.link(self.desktop_sink_pad)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1293
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1294 desktop_src.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1295 video_convert.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1296 queue.sync_state_with_parent()
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1297
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1298 def _remove_desktop_source(self, desktop_src: Gst.Element) -> None:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1299 """Remove the desktop source from the pipeline.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1300
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1301 @param desktop_src: The desktop source to remove.
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1302 """
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1303 # Remove elements for the desktop source
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1304 video_convert = self.pipeline.get_by_name("desktop_videoconvert")
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1305 queue = self.pipeline.get_by_name("desktop_queue")
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1306
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1307 if video_convert:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1308 video_convert.set_state(Gst.State.NULL)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1309 desktop_src.unlink(video_convert)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1310 self.pipeline.remove(video_convert)
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1311
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1312 if queue:
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1313 queue.set_state(Gst.State.NULL)
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1314 self.pipeline.remove(queue)
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1315
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1316 desktop_src.set_state(Gst.State.NULL)
4204
879bad48cc2d frontends (tools/webrtc): implement X11 desktop sharing:
Goffi <goffi@goffi.org>
parents: 4146
diff changeset
1317 self.pipeline.remove(desktop_src)
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1318
4205
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1319 # Release the pad associated with the desktop source
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1320 if self.desktop_sink_pad:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1321 self.video_selector.release_request_pad(self.desktop_sink_pad)
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1322 self.desktop_sink_pad = None
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1323
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1324 if self.desktop_portal is not None:
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1325 self.desktop_portal.end_screenshare()
17a8168966f9 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
Goffi <goffi@goffi.org>
parents: 4204
diff changeset
1326
4139
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1327 async def end_call(self) -> None:
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1328 """Stop streaming and clean instance"""
6745c6bd4c7a frontends (tools): webrtc implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1329 self.reset_instance()