annotate libervia/cli/call_webrtc.py @ 4231:e11b13418ba6

plugin XEP-0353, XEP-0234, jingle: WebRTC data channel signaling implementation: Implement XEP-0343: Signaling WebRTC Data Channels in Jingle. The current version of the XEP (0.3.1) has no implementation and contains some flaws. After discussing this on xsf@, Daniel (from Conversations) mentioned that they had a sprint with Larma (from Dino) to work on another version and provided me with this link: https://gist.github.com/iNPUTmice/6c56f3e948cca517c5fb129016d99e74 . I have used it for my implementation. This implementation reuses work done on Jingle A/V call (notably XEP-0176 and XEP-0167 plugins), with adaptations. When used, XEP-0234 will not handle the file itself as it normally does. This is because WebRTC has several implementations (browser for web interface, GStreamer for others), and file/data must be handled directly by the frontend. This is particularly important for web frontends, as the file is not sent from the backend but from the end-user's browser device. Among the changes, there are: - XEP-0343 implementation. - `file_send` bridge method now use serialised dict as output. - New `BaseTransportHandler.is_usable` method which get content data and returns a boolean (default to `True`) to tell if this transport can actually be used in this context (when we are initiator). Used in webRTC case to see if call data are available. - Support of `application` media type, and everything necessary to handle data channels. - Better confirmation message, with file name, size and description when available. - When file is accepted in preflight, it is specified in following `action_new` signal for actual file transfer. This way, frontend can avoid the display or 2 confirmation messages. - XEP-0166: when not specified, default `content` name is now its index number instead of a UUID. This follows the behaviour of browsers. - XEP-0353: better handling of events such as call taken by another device. - various other updates. rel 441
author Goffi <goffi@goffi.org>
date Sat, 06 Apr 2024 12:57:23 +0200
parents 9218d4331bb2
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4210
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
1 #!/usr/bin/env python3
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
2
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
3 # Libervia CLI
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
4 # Copyright (C) 2009-2024 Jérôme Poisson (goffi@goffi.org)
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
5
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
6 # This program is free software: you can redistribute it and/or modify
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
7 # it under the terms of the GNU Affero General Public License as published by
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
8 # the Free Software Foundation, either version 3 of the License, or
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
9 # (at your option) any later version.
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
10
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
11 # This program is distributed in the hope that it will be useful,
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
12 # but WITHOUT ANY WARRANTY; without even the implied warranty of
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
13 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
14 # GNU Affero General Public License for more details.
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
15
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
16 # You should have received a copy of the GNU Affero General Public License
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
17 # along with this program. If not, see <http://www.gnu.org/licenses/>.
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
18
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
19 from dataclasses import dataclass
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
20
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
21 from libervia.backend.tools.common import data_format
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
22 from libervia.frontends.tools import aio, jid
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
23
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
24
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
25 @dataclass
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
26 class CallData:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
27 callee: jid.JID
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
28 sid: str | None = None
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
29 action_id: str | None = None
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
30
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
31
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
32 class WebRTCCall:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
33 def __init__(self, host, profile: str, callee: jid.JID, **kwargs):
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
34 """Create and setup a webRTC instance
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
35
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
36 @param profile: profile making or receiving the call
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
37 @param callee: peer jid
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
38 @param kwargs: extra kw args to use when instantiating WebRTC
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
39 """
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
40 from libervia.frontends.tools import webrtc
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
41
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
42 aio.install_glib_asyncio_iteration()
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
43 self.host = host
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
44 self.profile = profile
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
45 self.webrtc = webrtc.WebRTC(host.bridge, profile, **kwargs)
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
46 self.webrtc.callee = callee
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
47 host.bridge.register_signal(
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
48 "ice_candidates_new", self.on_ice_candidates_new, "plugin"
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
49 )
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
50 host.bridge.register_signal("call_setup", self.on_call_setup, "plugin")
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
51 host.bridge.register_signal("call_ended", self.on_call_ended, "plugin")
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
52
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
53 @classmethod
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
54 async def make_webrtc_call(
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
55 cls,
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
56 host,
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
57 profile: str,
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
58 call_data: CallData,
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
59 **kwargs
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
60 ) -> "WebRTCCall":
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
61 """Create the webrtc_call instance
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
62
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
63 @param call_data: Call data of the command
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
64 @param kwargs: extra args used to instanciate WebRTCCall
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
65
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
66 """
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
67 webrtc_call = cls(host, profile, call_data.callee, **kwargs)
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
68 if call_data.sid is None:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
69 # we are making the call
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
70 await webrtc_call.start()
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
71 else:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
72 # we are receiving the call
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
73 webrtc_call.sid = call_data.sid
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
74 if call_data.action_id is not None:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
75 await host.bridge.action_launch(
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
76 call_data.action_id,
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
77 data_format.serialise({"cancelled": False}),
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
78 profile
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
79 )
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
80 return webrtc_call
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
81
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
82 @property
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
83 def sid(self) -> str | None:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
84 return self.webrtc.sid
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
85
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
86 @sid.setter
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
87 def sid(self, new_sid: str | None) -> None:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
88 self.webrtc.sid = new_sid
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
89
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
90 async def on_ice_candidates_new(
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
91 self, sid: str, candidates_s: str, profile: str
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
92 ) -> None:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
93 if sid != self.webrtc.sid or profile != self.profile:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
94 return
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
95 self.webrtc.on_ice_candidates_new(
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
96 data_format.deserialise(candidates_s),
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
97 )
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
98
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
99 async def on_call_setup(self, sid: str, setup_data_s: str, profile: str) -> None:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
100 if sid != self.webrtc.sid or profile != self.profile:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
101 return
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
102 setup_data = data_format.deserialise(setup_data_s)
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
103 try:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
104 role = setup_data["role"]
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
105 sdp = setup_data["sdp"]
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
106 except KeyError:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
107 self.host.disp(f"Invalid setup data received: {setup_data}", error=True)
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
108 return
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
109 if role == "initiator":
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
110 self.webrtc.on_accepted_call(sdp, profile)
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
111 elif role == "responder":
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
112 await self.webrtc.answer_call(sdp, profile)
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
113 else:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
114 self.host.disp(
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
115 f"Invalid role received during setup: {setup_data}", error=True
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
116 )
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
117 # we want to be sure that call is ended if user presses `Ctrl + c` or anything
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
118 # else stops the session.
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
119 self.host.add_on_quit_callback(
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
120 lambda: self.host.bridge.call_end(sid, "", profile)
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
121 )
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
122
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
123 async def on_call_ended(self, sid: str, data_s: str, profile: str) -> None:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
124 if sid != self.webrtc.sid or profile != self.profile:
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
125 return
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
126 await self.webrtc.end_call()
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
127 await self.host.a_quit()
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
128
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
129 async def start(self):
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
130 """Start a call.
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
131
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
132 To be used only if we are initiator
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
133 """
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
134 await self.webrtc.setup_call("initiator")
9218d4331bb2 cli (call): `tui` output implementation:
Goffi <goffi@goffi.org>
parents:
diff changeset
135 self.webrtc.start_pipeline()