Online-Study_Chapter00

Summary

This project will develop a video chat-related service that has been delayed under the pretext of "I think it will be difficult" and "I don't have time". Real-time remids me of the problem of delay and connections even if I don't actually lear it. I don't know if the project will success, but...let's try it.

In order to implement video chat, I need to know a technology called Web Real-Time-Communication(Web RTC), so I will study this first.

Of course, WebRTC is not the only technology in live video streaming. Depending on the delay in live video streaming technology, there are many technologies such as the following.

Let's look at RTMP, WebRTC, And HLS, the three most commonly used video protocols.

• RTMP(Real-Time Messaging Protocol)
• The primary intent of this protocol was to achieve low latency and reliable communication by maintaining a persistent connection between the streaming server and the video player. In layman’s terms, it can deliver a reliable video stream from the source to the viewer in under 5 seconds of delay on average.
• As more modern clients like mobile and IoT emerged in the last decade, RTMP started losing its ground due to its inability to support native playback in these platforms. 
• HLS(HTTP Live Streaming)
• Following the rise of HTML5, HLS took over the reins of video streaming by offering robust, reliable media distribution that can scale to hundreds of millions of users with the support of CDNs.Today, HLS is widely used by all browsers, mobile, and LR platforms to facilitate video consumption and streaming. HLS also supports DRM encryption (Digital Rights Management), a high security mechanism to protect the content it is streaming. This makes it the ideal choice for last-mile content delivery by all streaming and OTT providers
• For live video streaming, the main drawback of HLS is latency.Taking into account the time for ingesting the source, generating multiple renditions, distribution over CDNs, and final playback buffer requirements, latency can easily peak at 20-30 seconds in HLS.
• WebRTC
• WebRTC works on a long stack of protocols to abstract the media engine, codec, and transport layer into a bunch of APIs. The web developer just has to use these APIs to capture the media from a webcam, set up a peer connection, and transmit the data directly between any two browsers across the world with less than 500 milliseconds of latency.
• The challenge, however, lies in scaling. Due to the intense bandwidth configuration required to support multiple peer connections, WebRTC natively does not scale well beyond a few thousand connections at most.

Considering these characteristics and the caracateristics of the study(used by fewer than 10 people at most), I chose WebRTC because it was suitable for the project.

What is WebRTC?

as its name suggests, WebRTC can add real-time communicatioin function to applications. The WebRTC website can see the phrase "It supports video, voice, and generic data to be sent between peers, allowing developers to build powerful voice- and video-communication solutions." for the technology. Then let's learn about the technologies and protocols that WebRTC included.

ICE(Interactive Connectivity Establishment)

ICE is framework used in computer networking to find ways for two computers to talk to each other as directly as possible in peer-to-peer networking, but presence of NATs and Firewalls make it difficult for nodes(computers) to communicate with each other. It is a Networking technique which makes use of STUN(Session Traversal Utilities for NAT) and TURN(Traversal Using Relays Around NAT) to establish a connections between to nodes which is as direct as posiible.

To understand how ICE works, we need to know the workings of STUN and its extension TURN.

STUN(Session Traversal Utilities for NAT)

STUN and TURN servers are "servers that tell our where to communicate with each other and where to communicate". If we think about it, when someone asks where we commincate with each other on the Internet, we can answer that we communicate through IP. After all, STUN and TURN servers are servers thath inform each other's IPs.

For an endpoint under a NAT which has a local address, it is not reachable for other endpoints outside the local network, Hence a connection cannot be established. When this occurs the endpoint can request it’s public IP address from a STUN server. This publicly reachable IP can be used by other endpoints to establish a connection. But this case fails when endpoints are under symmetric NAT, which happens in most of the practical cases.This is where a TURN server comes into picture.

TURN(Traversal Using Relays Around NATs)

TURN server as the name suggests is used as a relay server or an intermediate server to exchange data. When any endpoint under Symmetric NAT can contact a TURN server which is on the public internet to establish a connection the endpoint is then called a TURN client. The disadvantage of using a TURN server is that it is required throughout the whole time span of the session unlike STUN server which no longer needed after the connection is established. Therefore in ICE technique STUN is used as default.

SDP(Session Description Protocol)

Through ICE, we identified the address candidates capable of P2P commuications.Now, we have to make information exchange with each other, which is SDP.

Session Description Protocol (SDP) is a protocol used in WebRTC to describe the initial acquisition of multimedia content, such as the resolution, format, and codec of streaming media. Video resolution, audio transmission or reception may be transmitted and received. hmm...this part is hard to understand.

Type of WebRTC implementations

we learn about the types of WebRTC implementations.

There are three main implementation methods of WebRTC : Mesh, MCU, and SFU.

Mesh

It's P2P method that we can easily think of. As shown in the figure above, each client has one Uplink to transmit its media information and one Downlink to receive media information. the server is used only to relay information for the connections because the connection between peers proceeds.

If five clients form Peer Connection for each as shown in the figure above, each client will have a total of eight links, with four uplinks to send their media information to the remaining four clients and four downlinks to receive media information from four clients. the load on the server is small, but the more clients there are, the more the load on the client becomes.

MCU(Multi-point Control Unit)

Unlike P2P, the server relays media traffic.

The MCU mixes, processes, and transmits media information from each client to the receiving side. Therefore, it is not necessary to have N downlinks to receive media information, but only one downlink to receive mixed data. In other words, it is an optimized method for the network because only two links, one Uplink to transmit media information and one Downlink to receive media information, need to be maintained.

However, as mentioned above, the load on the server increases exponentially because media information from all connected clients must be received, mixed, and processed.

SFU(Selective Forwarding Unit)

like MCU, the server relay media traffic too. Hower, the difference from the MCU is that the server does not mix and process media infromation from the client side, but forwards it as it is. In a way, it can be seen as an intermediate method between Mesh and MCU

As shown in the figure above, we will have one Uplink to transmit its media information and N Downlinks to receive media information. In other words, both the client and the server share the load

Our video study has a zoom-like format. In an environment where real-time guaranteed, several people need to access and exchange video and voice data, but the number of participants may be small or large. In addition, we excluded P2P that look bad in N:M.

And we tried to use SFU as a server cost problem between SFU and MCU.