I. Conceptual structure of a live broadcast system
If you look at it only from the user's point of view, the live broadcasting system is nothing more than a source of live broadcasting and a viewer of live broadcasting. The source sets up a tripod, turns on the fill light, adjusts the cell phone camera, and starts the live broadcast after a round of operation. Watching party open the live APP, rubbing into the live room, a moment to pinch the face, a moment to brush the rocket, play a lot of fun. For general developers, between the source of the live broadcast and the viewer of the live broadcast, there is also a layer of cloud platform used to transit audio and video data. Then at the hardware level, it constitutes the conceptual structure of the live broadcast system, as shown in the figure below.
But the conceptual structure is too general, because nothing is reflected in the software, God knows how to implement ah. For audio and video developers, to provide live system software architecture diagrams, to be considered a guide. Throughout the network of live system architecture articles, many of which belong to the sprawling long-winded, given the live system architecture diagram of a hundred twists and turns, looks professional enough to be high. However, the content of this live book is too rich, in the face of the towering mountains of the general live architecture, the ordinary developers often look forward to sighing, may not enter the door.
There are also companies that provide a dedicated audio and video live SDK, only developers in the code to call the API provided by the SDK, a few lines can be achieved live recording and live viewing functions. However, this SDK both the possibility of charges, it is also difficult to do in-depth customization of business needs, after all, unlike open source software that free, when it is the fish and the bear's paw can not be both ah.
II. Software architecture of the live broadcasting system
In view of this, the blogger after years of painstaking research, figuring out a set of all the use of open source software live system software architecture, not much to say first look at this live system software architecture diagram.
This live system software architecture diagram above is still divided into three levels, which are described below:
1, open source live recording software
There are two main types of open source live recording software, one is the computer side of the live recording software, represented by OBS Studio, Streamlabs; the other is the cell phone side of the live recording software, represented by RTMP Streamer, SRT Streamer.
OBS Studio is an open source live recording software , based on QT + FFmpeg coding , it is widely used in video recording , real-time live broadcasting and other fields.OBS is not only open source , but also cross-platform , compatible with Windows, Mac OS, Linux and other operating systems . About OBS Studio source code download, installation process and instructions for use see the previous article "Use OBS Studio to open RTMP live push streaming".
RTMP Streamer is an Android mobile open source RTMP live push streaming framework , can be used for RTMP live and RTSP live , its upgraded version also supports SRT live ( Tencent video cloud on the use of SRT protocol ).RTMP Streamer supports video encoding , including H264, H265, AV1 and so on, support for audio encoding , including AAC , G711, OPUS and so on. About RTMP Streamer source code download, installation process and instructions for use see the previous article "use RTMP Streamer to open the APP live push stream".
SRT Streamer is an open source SRT protocol live streaming framework for Android phones, which can be used for RTMP live broadcast and SRT live broadcast.SRT Streamer supports video encodings including H264, H265, etc., and audio encodings including AAC, OPUS, etc. For more information, please refer to the previous article "Using SRT Streamer to Open APP Live Streaming" to download the source and install the process and instructions for use. About SRT Streamer source code download, installation process and instructions for use see the previous article "use SRT Streamer to open the APP live push streaming".
2. Open source streaming media server
There are two main types of open source streaming media servers, one is foreign open source streaming media servers, represented by MediaMTX and nginx-rtmp; the other is domestic open source streaming media servers, represented by SRS and ZLMediaKit.
MediaMTX is an open source lightweight streaming media server, its installation process see "FFmpeg Development in Action: From Zero Basis to Short Video Online" book "10.2.2 FFmpeg to the network to push the stream", the use of the instructions in detail in the previous article "Detailed explanation of MediaMTX's push and pull the stream".
nginx-rtmp is an open source WEB server Nginx can be enhanced by a third-party rtmp module that can provide simple rtmp streaming media server functions. nginx default did not open the rtmp module, you need to add the rtmp module in the compilation of nginx, but also to modify the configuration file in order to open the rtmp, the nginx-rtmp about the source code download, installation process and instructions for use see the previous article "Linux compile nginx-rtmp realize RTMP push-pull stream".
SRS is a domestic open source real-time video server , supports RTMP, WebRTC, HLS, HTTP-FLV, SRT and other streaming protocols.SRS and FFmpeg, OBS, VLC, WebRTC and other clients with the use of audio and video streams to provide the ability to receive and distribute , but also supports a variety of audio and video protocol conversion . About SRS source code download, installation process and instructions for use see the previous article "Linux environment to install SRS to realize the video push stream".
ZLMediaKit is a domestic open source streaming media server , it is based on C++11 development , supports a variety of streaming media protocols (RTSP/RTMP/HLS/HTTP-FLV/SRT/WebRTC , etc.) , support for audio and video coding standards including H265, H264, AAC, G711, OPUS and so on. About ZLMediaKit source code download, installation process and instructions for use see the previous article "Linux environment to install ZLMediaKit to realize the video push stream".
Among the above streaming media servers, MediaMTX can be deployed in Windows, while nginx-rtmp, SRS, and ZLMediaKit need to be deployed in Linux, and the cloud platform vendors for deployment can be Huawei Cloud, Ali Cloud, Tencent Cloud, and so on.
3, open source audio and video player
There are two main types of open source audio and video players, one is the computer side of the playback software, represented by ffplay and VLC media player; the other is the cell phone side of the playback software, represented by ExoPlayer and ijkplayer.
ffplay is a player program that comes with FFmpeg to play audio and video files directly from the command line. When playing audio, ffplay will not only let the speakers play the sound, but also show the waveform of the audio on the screen. When playing video, ffplay will display a continuous video screen, just like watching a movie or TV. For more information about ffplay, please refer to "1.4.1 Executable Programs" in the book "FFmpeg Development in Action: From Zero Basics to Short Videos".
VLC media player is a cross-platform audio and video player, it is not only open source, but also compatible with multiple languages, and with the deep integration of FFmpeg, it supports almost all audio and video formats. In addition to common audio and video files, VLC media player can also play RTMP, RTSP and other network streams. For the source code download, installation process and instructions of VLC media player, please refer to "3.4.1 Universal Audio/Video Player" in the book "FFmpeg Development in Action: From Zero Basis to Short Video Online".
ExoPlayer is a new type of player that Android officially carries in the Jetpack library. Its audio and video kernel relies on the native MediaCodec interface, which not only can play any format of video supported by MediaPlayer, but also supports video broadcasting protocols, including DASH, HLS, RTMP, and so on. For detailed usage of ExoPlayer, please refer to "14.3.3 New Player ExoPlayer" in the book "Android Studio Development in Action: From Zero to App Launch (3rd Edition)".
ijkplayer is an open source player for cell phones based on ffplay developed by Beili Beili, which supports live video streaming with rtmp, rtsp, http and other streaming protocols, and also has the application pop-up feature specific to B station. ijkplayer's source code is hosted at /bilibili/ijkplayer, which can be imported into the app project as shown below. dependency package in the app project, as shown below.
However, ijkplayer is written based on FFmpeg 3.4 and hasn't been updated for a long time. Supported Android platforms are limited to API 9~23 and supported iOS platforms are limited to iOS 7.0~10.
Comprehensive above open source live recording software, open source streaming media server and open source audio and video player, you can build a complete live broadcast system, the system's effect diagrams see the previous article "RTMP protocol to build a computer and cell phone live Demo" and "SRT protocol to build a cell phone APP live Demo".
For more details on FFmpeg development see