With this in mind, the blogger has compiled a list of several popular open source frameworks for audio and video, each of which is summarized below:
I. Computer vision library OpenCV
OpenCV is known as Open Source Computer Vision Library, which is a cross-platform computer vision and machine learning software library that runs on Linux, Windows, Android and MacOS operating systems.
OpenCV's official website address is /, and its source hosting portal page on Github is /opencv/opencv. As of Github statistics on August 4, 2024, the OpenCV project has 77,300 stars and 55,700 copies.
There are three main uses of OpenCV in programming practice:
1, through the Python script to use OpenCV
First execute the following command to install the Python support library for OpenCV.
pip install opencv-python
The OpenCV library is then imported at the beginning of the Python code with the following statement:
import cv2
Then you can call various OpenCV processing functions in Python code.
2, through the Qt desktop program in the integration of OpenCV
First download the OpenCV C++ support library to a local directory, then open the Qt C++ project and add the following lines before qt_add_executable to specify the OpenCV library header file directory and library file directory.
# Add the directory where the header files are located
include_directories(opencv/include)
# Specify the link directory for the lib file
link_directories(opencv/lib)
Then put the following line in the configuration:
target_link_libraries(test_opencv PRIVATE Qt${QT_VERSION_MAJOR}::Widgets)
The purpose of changing the configuration to the following lines is to introduce the linking of the added OpenCV library files:
# Set up a collection of libraries called opencv-libs, specifying which so library files it contains
set(opencv-libs opencv_world480)
target_link_libraries(test_opencv PRIVATE Qt${QT_VERSION_MAJOR}::Widgets ${opencv-libs})
Then you can import OpenCV's header files into your C++ code and call OpenCV's various processing functions.
3, through the Android Studio in the mobile app integration OpenCV
First of all, download the OpenCV source package, unzip it and find the OpenCV-android-sdk directory, there are two subdirectories under the directory, respectively, and samples and sdk, of which sdk is the library project, the library project should be referenced before using OpenCV in the app, and samples is the sample project, there are a number of sample modules below.
Then integrate the above sdk directory into the Android Studio App project, detailed integration steps see "Android App Development Advanced and Project Combat" book chapter 12 of "12.2.2 Introduction to OpenCV and its integration", will not repeat here.
Second, computer graphics library learning tutorial LearnOpenGL
LearnOpenGL is an open source OpenGL tutorial for beginners to learn how to use OpenGL to create 3D graphics and animation.LearnOpenGL provides both the physical English book, but also provides a free English electronic version of the PDF, the official website of the electronic version of the PDF download link for /book/book_pdf.pdf.
LearnOpenGL's official website is at /, and its source code hosting entry page on Github is /JoeyDeVries/LearnOpenGL. As of Github statistics as of August 4, 2024, the LearnOpenGL project has 10.7k stars and 0.27k copies.
III. Multimedia processing framework FFmpeg
FFmpeg full name Fast Forward MPEG, meaning fast mastery of MPEG, it is an open source cross-platform multimedia processing framework, can run on Linux, Windows, Android and MacOS operating systems, etc. FFmpeg not only able to deal with audio and video files, but also can deal with images, subtitles and other files, can be said to be powerful. FFmpeg can not only handle audio and video files, but also image and subtitle files, which is very powerful.
FFmpeg's official website address is /, and its source code hosting entry page on Github is /FFmpeg/FFmpeg. As of Github's statistics on August 4, 2024, the FFmpeg project has 44.1k stars and 11.9k copies.
There are three main uses of FFmpeg in programming practice:
1, run FFmpeg in the command line
Follow the instructions in the previous article, "Compiling and Installing FFmpeg on the Euler System", to compile and install FFmpeg on the Linux system, you can generate three executable programs, namely ffmpeg, ffplay, and ffprobe.Then follow the instructions in the previous article, "Introducing FFmpeg's Executable Programs", you can execute the relevant audio and video processing operations on the command line. processing operations on the command line.
2, through Qt in the desktop program integrated FFmpeg
First, according to the "FFmpeg development in action: from zero to short video on the line" book chapter 8 "8.1 Windows environment compilation FFmpeg" introduction, in the Windows system to compile the FFmpeg dll dynamic library. Then follow the introduction in Chapter 11 "11.2.1 Integrating FFmpeg with Qt" in the book "FFmpeg Development in Action: From Zero Basis to Short Video" to integrate the compiled dll library and its header files into Qt's C++ project. Then you can call various APIs provided by FFmpeg in your C++ code.
3, through the Android Studio in the mobile app integration FFmpeg
First, according to the "FFmpeg development practice: from zero to short video on the line" book chapter 12 "12.1.2 cross-compile Android needs so library" introduction, in the Windows system cross-compile FFmpeg for Android platform so dynamic library. Then according to the "FFmpeg development practice: from zero to short video on the line" book chapter 12 "12.1.3 App project call FFmpeg so library" introduction, to the Android Studio App project to integrate the compiled so dynamic library and its header files. Then you can call various APIs provided by FFmpeg in the app code.
Fourth, the video recording live tool OBS Studio
OBS Studio full name is Open Broadcaster Software Studio, it is an open source cross-platform real-time streaming and screen recording tool that runs on operating systems such as Linux, Windows and MacOS.
OBS Studio's official website address is /, and its source code hosting entry page on Github is /obsproject/obs-studio.As of Github statistics as of August 4, 2024, the OBS Studio project has 58,000 stars and 0.78 million copies.
For instructions on installing and using OBS Studio, see the previous articles "Using OBS Studio to enable RTMP live push streaming" and "Combining OBS and MediaMTX to realize SRT live push streaming", so I won't repeat them here.
V. Audio and video player VLC media player
VLC media player is an open source cross-platform multimedia player and framework that can play almost all multimedia files, as well as DVD, audio CD, VCD and various streaming protocols live address.
The official website address for VLC media player is /, and its source code hosting entry page on Github is /videolan/vlc. As of Github statistics as of August 4, 2024, the VLC media player project has 13.5k stars and 0.5k copies.
In addition to playing media files, VLC media player is often used by developers to debug whether audio and video files are correctly formatted, and whether streaming data is correctly transmitted over the network. Any audio/video file that cannot be opened by VLC media player is almost certain to have a problem with the internal encoding or encapsulation of the file; and any streaming media address that cannot be played by VLC media player is almost certain to have missing streaming media data or poor network conditions.
For instructions on installing and using VLC media player, see "3.4.1 Universal Video Player VLC" in Chapter 3 of the book "FFmpeg Development in Action: From Zero Basis to Short Video Online", and we won't repeat it here.
For more details on FFmpeg development see