Real-Time Communications on the Universal Windows Platform with WebRTC and ORTC
Readers of this blog interested in Real-Time Communications are probably familiar with Google’s WebRTC project. From the WebRTC site:
“WebRTC is a free, open project that provides browsers and mobile applications with Real-Time Communications (RTC) capabilities via simple APIs. The WebRTC components have been optimized to best serve this purpose.”
At Microsoft, we’ve seen tremendous support grow for WebRTC over the past five years. One of the most pivotal uses of WebRTC is building native video chat apps, which now reach more than one billion users.
Google’s native supported platforms for WebRTC include iOS, Android and traditional Win32 desktop apps. On Windows, Microsoft Edge already supports ORTC APIs and now supports WebRTC 1.0 APIs in Insider Preview builds on Desktop devices. For example, if you need to build a WebRTC app in HTML/JS targeted at desktop browsers or desktop web apps using the Web App Template, then Microsoft Edge and Windows web platform are a great choice.
But what if you want to write in C# or C++ and run WebRTC on Xbox, HoloLens, Surface Hub or Windows Phone, or write in HTML/JS and run on Raspberry Pi? What if you are using Google’s iOS and Android libraries and need bit-for-bit compatibility for your UWP application? What if you modify WebRTC source in your application and need to use those modifications in your Universal Windows Platform (UWP) application?
To fulfill these additional scenarios, we have ported and optimized WebRTC 1.0 for UWP. This is now available as an Open Source project on GitHub as well as in binary form as a NuGet package. The project is 100 percent compatible with Google’s source, enabling scenarios such as a WebRTC video call from Xbox running UWP to a Chrome browser on the Desktop.
WebRTC ChatterBox sample running as a native Windows 10 application.
Microsoft has also long been a supporter of the ORTC APIs and we work closely with the Open Peer Foundation to ensure optimal support of ORTC for UWP apps. ORTC is an evolution of the WebRTC API, which gives developers fine-grained control over the media and data transport channels, and uses a standard JSON format to describe peer capabilities rather than SDP, which is unique to WebRTC.
ORTC was designed with WebRTC interoperability in mind and all media is wire-compatible with WebRTC. ORTC also includes an adapter that converts SDP to JSON and exposes APIs that match WebRTC. Those two considerations make it possible for developers to migrate from WebRTC to ORTC at their own pace and enable video calls between WebRTC and ORTC clients. ORTC for UWP is available both as an Open Source project on GitHub as well as a NuGet package.
The net result of combined UWP and Edge support for WebRTC 1.0 and ORTC is that all Windows 10 platforms support RTC and developers can choose the solution they prefer.
Let’s take a look at an example from our samples repository on GitHub.
The DataChannel, part of both the WebRTC and ORTC specs, is a method for two peers to exchange arbitrary data. This can be very useful in IoT applications – for example, a Raspberry Pi may collect sensor data and relay it to a Mobile or HoloLens peer in real-time. Keep in mind that while the sample code below uses ORTC APIs, the same scenario is possible via WebRTC.
To exchange messages between peers in ORTC, a few things must happen first (see MainPage.OpenDataChannel() in the sample code):
- The peers must exchange ICE candidates, a successful pair of which will be used to establish a peer-to-peer connection.
- The peers must exchange ICE parameters and start an ICE transport session – the underlying data path used for the peers to exchange data.
- The peers must exchange DTLS parameters. which includes encryption certificate and fingerprint data used to establish a secure peer-to-peer connection, and start a DTLS transport session.
- The peers must exchange SCTP capabilities and start an SCTP transport session. At this stage, a secure connection between the peers has been established and a DataChannel can be opened.
It’s important to understand two things about the above sequence. First, the data exchanges are in simple JSON, and as long as two peers can exchange strings, they can exchange all necessary data. Second, the identification of the peers and the exchange of these parameters, called signaling, is outside of the specification of ORTC and WebRTC by design. There are plenty of mechanisms available for signaling and we won’t go into them, but NFC, Bluetooth RFCOMM or a simple TCP socket server like the one included in the sample code, would suffice.
With the SCTP transport session established, the peers can open a Data Channel. The peer initiating the call creates an instance of an RTCDataChannel() passing the SCTP transport instance, and the remote peer receives the event RTCSctpTransport.OnDataChannel. When the remote peer receives this event, the Data Channel has been established and the peers can send messages to each other.
The code below is an excerpt from MainPage.Signaler_MessageFromPeer() in the sample code. The string message contains data received from the peer via the signaling method (in this case, the TCP socket server):
var sctpCaps = RTCSctpCapabilities.FromJsonString(message);
if (!_isInitiator)
{
// The remote side will receive notification when the data channel is opened. Send SCTP capabilities back to the initiator and wait.
_sctp.OnDataChannel += Sctp_OnDataChannel;
_sctp.Start(sctpCaps);
var caps = RTCSctpTransport.GetCapabilities();
_signaler.SendToPeer(peer.Id, caps.ToJsonString());
}
else
{
// The initiator has received SCTP caps back from the remote peer, which means the remote peer has already
// called _sctp.Start(). It's now safe to open a data channel, which will fire the Sctp.OnDataChannel event on the remote peer.
_sctp.Start(sctpCaps);
_dataChannel = new RTCDataChannel(_sctp, _dataChannelParams);
_dataChannel.OnMessage += DataChannel_OnMessage;
_dataChannel.OnError += DataChannel_OnError;
}
When the DataChannel has been established, the remote peer receives the OnDataChannel event. The parameter data for that event includes a secure DataChannel which is open and ready to send messages:
private void Sctp_OnDataChannel(RTCDataChannelEvent evt)
{
_dataChannel = evt.DataChannel;
_dataChannel.OnMessage += DataChannel_OnMessage;
_dataChannel.OnError += DataChannel_OnError;
_dataChannel.SendMessage("Hello ORTC peer!");
}
You can now freely exchange encrypted messages between the peers over the DataChannel. The signaling server is no longer required and that connection can be closed.
Real-time peer connectivity in Universal Windows Applications enables many exciting scenarios. We’ve seen developers use this technology to enable a remote peer to see what HoloLens users sees in real-time and interact with their 3D environment. Xbox developers have used the DataChannel to enable low-latency FPS style gaming. And one of our close collaborators, Blackboard, relies on the technology to stream classroom video feeds and enable collaboration in their Windows app. Check out our Universal Windows samples and the library source on GitHub – we look forward to your PRs!
Source: Real-Time Communications on the Universal Windows Platform with WebRTC and ORTC
Leave a Reply