Location>code7788 >text

FFmpeg development notes (fifty) to talk about the past and present life of several streaming media transfer technology

Popularity:823 ℃/2024-09-01 11:53:43
Since the popularization of the Internet, streaming technologies for live video have developed. In these decades, the more influential ones are mainly MMS, RTSP, RTMP, HLS, SRT, RIST, which are introduced as follows.

1. MMS protocol

MMS is known as Microsoft Multimedia Server, meaning Microsoft Multimedia Server, which is a multimedia server solution released by Microsoft in the 1990s that can be used to transmit live streaming data in Microsoft audio and video formats.
The live address of the MMS protocol is shaped like mms://***, and the video format that can be transmitted through MMS is WMV, the audio format is WMA, and the file format after the audio and video data is encapsulated is ASF.
The MMS protocol is internally subdivided into MMSU and MMST, where MMSU indicates the MMS protocol combined with UDP data transfer. If the MMSU connection fails, the server tries to use MMST, which represents the MMS protocol combined with TCP data transfer.
Because the MMS protocol was proposed by Microsoft and is not compatible with other formats of audio and video data streams, the MMS protocol has gradually gone unnoticed with the decline of the WMV/WMA standard.

2. RTSP protocol

RTSP full name Real Time Streaming Protocol, meaning Real Time Streaming Protocol, which is a multimedia real-time transmission protocol jointly proposed by Netscape and RealNetworks in the 1990s.
RTSP protocol live address is shaped like rtsp://***, early RTSP can be transmitted through the video format RM, audio format RA, audio and video data encapsulation after the file format RM or RMVB. later RTSP protocol to increase the support for MPEG audio and video standards, that is, to support the transmission of video format H.264, audio format AAC.
The secure version of the RTSP protocol is RTSPS, which is the addition of TLS/SSL support to the RTSP protocol.RTSPS uses the TLS/SSL protocol to encrypt and protect data transmissions from eavesdropping and tampering during transmission.
Because RTSP was proposed earlier, the complexity requirements for the server side are relatively high, so much so that the streaming media server SRS simply gave up supporting the RTSP protocol, and the live recording software OBS Studio also did not support the protocol. In the streaming media server, EasyDarwin, MediaMTX, ZLMediaKit support RTSP protocol. Cell phone live streaming software, on the other hand, RTMP Streamer supports the RTSP protocol.

3、RTMP protocol

RTMP full name Real Time Messaging Protocol, meaning Real Time Messaging Protocol, which is a streaming media data transmission protocol proposed by Adobe in the 2000s.
RTMP protocol live address in the shape of rtmp://***, can be transmitted via RTMP video format for H.264, audio format for MP3 or AAC, audio and video data encapsulation after the file format for FLV or F4V.
The secure version of the RTMP protocol is RTMPS, which means that TLS/SSL support has been added to the RTMP protocol.RTMPS utilizes two encryption protocols, Secure Sockets Layer (SSL) and Transport Layer Security (TLS), to make data transmission more secure.
RTMP was proposed earlier, last updated in 2012, so it failed to support HEVC and AV1 and other late audio and video coding standards. And because the FLV format has been down for a long time, so that the HTML5 specification simply removed the Flash plug-in, resulting in today's browsers do not support rtmp links, even FFmpeg also as late as version 6.1 to the rtmp protocol to support the hevc format.
However, the good thing is that RTMP's stability is high, the server-side implementation is relatively easy, and before the rapid outbreak of the mobile Internet, the new streaming protocol failed to launch in time, so that the RTMP protocol has been used in a large number of webcasting field.
Among streaming servers, MediaMTX, ZLMediaKit, SRS all support RTMP protocol. Among the live streaming software, OBS Studio on the computer side supports RTMP protocol, and RTMP Streamer and SRT Streamer on the mobile side both support RTMP protocol.
See the previous articles "Using RTMP protocol to build a computer and cell phone live Demo" and "Using RTMP Streamer to open the APP live push stream" for instructions on implementing the live broadcast function via RTMP protocol.

4. HLS protocol

HLS full name HTTP Live Streaming, meaning HTTP-based streaming media transmission protocol, it is Apple in 2009 proposed a due to the transmission of audio and video protocol interaction.
HLS uses HTTP protocol to transmit audio and video data, the access address is like "http://***.m3u8". HLS protocol cuts the audio and video streams into TS slices and generates m3u8 playlist files, and notifies the client to download the playlist files through HTTP protocol, download the sliced files and play them in the order of the list files, thus realizing the effect of real-time online playback while downloading. The client is notified to download the playlist file via HTTP protocol, download the sliced file according to the order in the list file and play it, thus realizing playback while downloading, similar to the effect of real-time online playback.
Since HLS only uses HTTP protocol at the transport layer, it has the network advantages of HTTP protocol, such as easy access through firewalls or proxy servers, and simple load balancing of media streams. HLS is more often used in the video-on-demand field because the HLS protocol transmits video streams in pieces, which results in higher latency during live broadcasts.
For more information on the HLS protocol, see the previous articles Analyzing SRS Interpolation of TS Packets in the HLS Protocol and Analyzing SPS Frames and PPS Frames in H.264 Streams.

5. SRT protocol

SRT is called Secure Reliable Transport, which means Secure and Reliable Transport Protocol, and it is proposed by the SRT consortium co-founded by Haivision and Wowza.
SRT protocol protocol live address shaped like srt://***, it introduces the AES encryption algorithm, without the need to introduce special SSL certificates like RTSP and RTMP. As a newer streaming protocol, SRT supports more audio and video encapsulation formats. It's just that the protocol's support library libsrt was only open sourced in 2017, so it failed to spread a lot in the mobile Internet era, and is currently mainly used in large-scale live TV broadcasting.
FFmpeg supports integration of third-party libsrt libraries since 4.0. Among streaming servers, MediaMTX, ZLMediaKit, SRS all support SRT protocol. Among the live streaming software, OBS Studio for computer supports SRT protocol since 25.0, and SRT Streamer for cell phone supports SRT protocol, while RTMP Streamer does not support SRT protocol, only its upgraded version supports SRT protocol.
For instructions on implementing the live streaming function via SRT protocol see the previous articles Building a Live Streaming Demo for a Mobile App Using SRT Protocol and Enabling APP Live Streaming Push Streaming Using SRT Streamer.

6. RIST protocol

RTST is known as Reliable Internet Stream Transport, which means Reliable Internet Streaming Protocol, and it was proposed by the RIST Working Group, which was formed in 2017.
RIST is a streaming transport protocol that uses the UDP protocol at the transport layer and provides reliability and flow control features at the application layer. It is not a pure application layer protocol, but rather a protocol that operates between the transport and application layers.RIST and SRT have the same encryption level, and both support high-capacity streaming and forward error correction.
The RIST protocol was developed later than SRT, although the late development will consider more new features, such as RIST supports point-to-multipoint broadcasting, which SRT does not support; but the late development has dragged down the support for RIST in various open source software, for example, OBS Studio started to support SRT as early as 25.0, and started to support RIST as late as 27.0, another live streaming recording software RootEncoder has supported SRT but not RIST, and MediaMTX, a streaming media server, has supported SRT but not RIST.
FFmpeg supports integration of third-party librist libraries since 4.4. Among streaming servers, MediaMTX, ZLMediaKit, SRS do not support RIST protocol. Among the live streaming software, OBS Studio on the computer side has supported the SRT protocol since 27.0, and no open source software on the mobile side supports the RIST protocol yet.

In general, the current domestic market share of the main live protocol is still RTMP, but has a better performance of the SRT protocol is gradually catching up, such as Tencent Video Cloud, Jingdong Video Cloud and so on on the introduction of the SRT protocol. For instructions on building a live broadcast system, see the previous article "open source software architecture to build a live broadcast system from 0".

For more details on FFmpeg development seeFFmpeg Development in Action: From Zero Basics to Short Video OnlineA book.