annotate libervia/frontends/tools/webrtc_models.py @ 4240:79c8a70e1813

backend, frontend: prepare remote control: This is a series of changes necessary to prepare the implementation of remote control feature: - XEP-0166: add a `priority` attribute to `ApplicationData`: this is needed when several applications are working in a same session, to know which one must be handled first. Will be used to make Remote Control have precedence over Call content. - XEP-0166: `_call_plugins` is now async and is not used with `DeferredList` anymore: the benefit to have methods called in parallels is very low, and it cause a lot of trouble as we can't predict order. Methods are now called sequentially so workflow can be predicted. - XEP-0167: fix `senders` XMPP attribute <=> SDP mapping - XEP-0234: preflight acceptance key is now `pre-accepted` instead of `file-accepted`, so the same key can be used with other jingle applications. - XEP-0167, XEP-0343: move some method to XEP-0167 - XEP-0353: use new `priority` feature to call preflight methods of applications according to it. - frontend (webrtc): refactor the sources/sink handling with a more flexible mechanism based on Pydantic models. It is now possible to have has many Data Channel as necessary, to have them in addition to A/V streams, to specify manually GStreamer sources and sinks, etc. - frontend (webrtc): rework of the pipeline to reduce latency. - frontend: new `portal_desktop` method. Screenshare portal handling has been moved there, and RemoteDesktop portal has been added. - frontend (webrtc): fix `extract_ufrag_pwd` method. rel 436
author Goffi <goffi@goffi.org>
date Sat, 11 May 2024 13:52:41 +0200
parents d01b8d002619
children 0d7bb4df2343
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
1 #!/usr/bin/env python3
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
2
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
3 # Libervia WebRTC implementation
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # Copyright (C) 2009-2023 Jérôme Poisson (goffi@goffi.org)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
5
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 # This program is free software: you can redistribute it and/or modify
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # it under the terms of the GNU Affero General Public License as published by
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # the Free Software Foundation, either version 3 of the License, or
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # (at your option) any later version.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
10
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
11 # This program is distributed in the hope that it will be useful,
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # GNU Affero General Public License for more details.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
15
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 # You should have received a copy of the GNU Affero General Public License
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
18
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
19 from collections.abc import Awaitable
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
20 from dataclasses import dataclass, field
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
21 from typing import Any, Callable
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
22 import uuid
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
23 import gi
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
24
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
25 gi.require_versions({"Gst": "1.0", "GstWebRTC": "1.0"})
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
26 from gi.repository import GstWebRTC
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
27 from pydantic import BaseModel, Field
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
28
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
29 from libervia.frontends.tools import jid
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
30
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
31
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
32 @dataclass
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
33 class CallData:
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
34 callee: jid.JID
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
35 sid: str | None = None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
36 action_id: str | None = None
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
37 kwargs: dict[str, Any] = field(default_factory=dict)
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
38
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
39
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
40 class SourcesData(BaseModel):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
41 """Data for Sources"""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
42
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
43
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
44 class SourcesNone(SourcesData):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
45 """No source is used.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
46
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
47 This is used when the WebRTC connection will be used for data channels only."""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
48
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
49
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
50 class SourcesAuto(SourcesData):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
51 """Automatic Sources (webcam/microphone)"""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
52
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
53
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
54 class SourcesTest(SourcesData):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
55 """Test Sources (pattern)"""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
56
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
57
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
58 class SourcesDataChannel(SourcesData):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
59 """Sources for transmitting data over Data Channel
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
60
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
61
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
62 @param dc_open_cb: Called when Data Channel is open.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
63 This callback will be run in a GStreamer thread.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
64 """
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
65 name: str = Field(default_factory=lambda: str(uuid.uuid4()))
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
66 dc_open_cb: Callable[[GstWebRTC.WebRTCDataChannel], None]
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
67
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
68
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
69 class SourcesPipeline(SourcesData):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
70 """Use custom pipeline description as a source.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
71
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
72 @param video_pipeline: pipeline description of video source.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
73 None to use automatic video source (same as SourcesAuto).
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
74 Empty string to disable video.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
75 @param audio_pipeline: pipeline description of audio source.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
76 None to use automatic audio source (same as SourcesAuto).
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
77 Empty string to disable audio.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
78 @param video_properties: Elements properties to set.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
79 @param audio_properties: Elements properties to set.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
80
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
81 """
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
82 video_pipeline: str|None = None
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
83 audio_pipeline: str|None = None
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
84 video_properties: dict = Field(default_factory=lambda: {})
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
85 audio_properties: dict = Field(default_factory=lambda: {})
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
86
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
87
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
88 class SinksData(BaseModel):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
89 """Data for Sinks"""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
90
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
91
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
92 class SinksNone(SinksData):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
93 """No sink is used.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
94
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
95 This is used when the WebRTC connection will be used for data channels only."""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
96
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
97
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
98 class SinksAuto(SinksData):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
99 """Automatic Sinks (create windows/default audio)"""
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
100
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
101
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
102 class SinksApp(SinksData):
4233
d01b8d002619 cli (call, file), frontends: implement webRTC data channel transfer:
Goffi <goffi@goffi.org>
parents:
diff changeset
103 local_video_cb: Callable
4240
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
104 remote_video_cb: Callable | None
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
105
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
106
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
107 class SinksDataChannel(SinksData):
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
108 """Sinks for transmitting data over Data Channel
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
109
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
110 @param dc_on_data_channel: Called when Data Channel is created.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
111 This callback will be run in a GStreamer thread.
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
112 """
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
113
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
114 dc_on_data_channel: (
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
115 Callable[[GstWebRTC.WebRTCDataChannel], Awaitable[None]] | None
79c8a70e1813 backend, frontend: prepare remote control:
Goffi <goffi@goffi.org>
parents: 4233
diff changeset
116 ) = None