Experiments on shared library search path prioritization during compile-time linking

preamble

Path Search Prioritization for Linking and Loading Shared LibrariesIt is mentioned that the use ofg++The library path search priority for shared libraries at compile-time linking is when:-L specified path>Path to the LIBRARY_PATH record>default path。

This experiment verifies the above conclusion in three steps
① Test the feasibility of the path specified by each method individually
② Compare and contrast the priorities among the three methods of testing
③ Use DEBUG mode to view the details of the linker output and verify the above conclusions for a second time

It's worth noting that I've read all over the internet thatPath specified by LIBRARY_PATHPriority should be greater thandefault pathpriority, but as far as my tests go, the conclusion is the opposite (probably because I'm using g++ instead of directly using the underlyingld?）。

experimental environment

Operating System:Ubuntu 20.04
Compiler:g++-11.4.0
make：GNU Make 4.2.1

Catalog description

The directory structure of the project is as follows:

.
├── lib
├── obj
├── 
├── libhello_alt.cpp
├── 
└── makefile

Among them:

• libFor the folder holding the shared libraries
• objFolder for relocatable target files
• cap (a poem)libhello_alt.cppfor shared library source code (used to simulate different versions of shared libraries), they both have only one hello function in them, and the function signatures of the two hello functions are exactly the same. The two hello functions have identical signatures.will be compiled as.1.1.0(soname is.1），libhello_alt.cppwill be compiled as.2.1.0(soname is.2）
• The main function, which calls the hello function
• makefileBuilding scripts for automation

In the appendix, I will provide the code involved in this experiment.

preliminary

Go to the project path in the terminal and typemakeIt will be in the./libgenerated under.1.1.0together with.2.1.0in./objgenerated under。
The directory structure of the generated project is as follows:

.
├── lib
│   ├── .1.1.0	
│   └── .2.1.0
├── obj
│   └── 
├── 
├── libhello_alt.cpp
├── 
└── makefile

individual testing

No path configuration

Do not configure any paths and do not place thefile to see if therespond in singinghelloThe link to the shared library file was successful.

This can be accomplished by using the predefined commands directly from the makefile:

make main_none

Output:

You can see that since we didn't configure any additional search paths and didn't place the default search path under thefile, the linker will not find the corresponding shared library file and the link will fail.

At the end of a single experiment, usemake cleancommand to clear the files generated by this experiment, and then again use themakecommand regenerates the shared library file and the relocatable target file. (Repeat this step every time you finish a small experiment, which will not be repeated later)

default path

commander-in-chief (military).1.1.0Copy to default search path/usr/liband in/usr/libCreate a soft link under () to point to it, and then perform a linking operation to see if thecap (a poem)helloThe link to the shared library file was successful.

This can be accomplished by directly using the commands predefined in the makefile:

make main_default

Output:

No errors were reported.

Then usereadelf -dViewing the dynamic segment information of the executable, it can be seen that the link was successful and that the shared library'ssonamehas been written to the dynamic segment information of the executable file.

LIBRARY_PATH

Create a path/opt/hellolibwill.1.1.0Copy to/opt/helloliband in/opt/hellolibCreate a soft link under () point to it. Then point the/opt/hellolibAdd toLIBRARY_PATHcarry outcap (a poem)helloof a shared library file to see if it can be linked successfully.

This can be accomplished by using the predefined commands directly from the makefile:

make main_library_path

Output:

No errors were reported.

Then usereadelf -dViewing the dynamic segment information of the executable, it can be seen that the link was successful and that the shared library'ssonamehas been written to the dynamic segment information of the executable file.

-L

Create a path/opt/hellolibwill.1.1.0Copy to/opt/helloliband in/opt/hellolibCreate a soft link under () point to it and then add the link option-L/opt/helloliband perform a linking operation to see if thecap (a poem)helloThe link to the shared library file was successful.

This can be accomplished by using the predefined commands directly from the makefile:

make main_l

Output:

No errors were reported.

Then usereadelf -dViewing the dynamic segment information of the executable, it can be seen that the link was successful and that the shared library'ssonamehas been written to the dynamic segment information of the executable file.

Priority testing

Default path and LIBRARY_PATH

• ① will.2.1.0Copy to default search path/usr/liband in/usr/libCreate a soft link under () point to it.
• ②Creating a path/opt/hellolibwill.1.1.0Copy to/opt/helloliband in/opt/hellolibCreate a soft link under () point to it.
• (iii) will/opt/hellolibAdd toLIBRARY_PATHcarry outcap (a poem)helloThe linking operation of the shared library files of the

This can be accomplished by using the predefined commands directly from the makefile:

make cmp_default_libpath

Output:

Then usereadelf -dViewing the dynamic segment information of the executable, it can be seen that the link was successful and the shared library file is linked under the default path.2.1.0(its soname is.2). It can therefore be concluded:default pathSearch is prioritized higher thanPath specified by LIBRARY_PATHThe search priority of the

A more detailed verification of the above conclusions will be given later in the DEBUG model.

-L and default path

• ①Creating a path/opt/hellolibwill.2.1.0Copy to/opt/helloliband in/opt/hellolibCreate a soft link under () point to it.
• ② will.1.1.0Copy to default search path/usr/liband in/usr/libCreate a soft link under () point to it.
• ③Add link option-L/opt/hellolibcarry outcap (a poem)helloThe linking operation of the shared library files of the

This can be accomplished by directly using the commands predefined in the makefile:

make cmp_l_default

Output:

Then usereadelf -dViewing the dynamic segment information of the executable file, it can be seen that the link was successful and the shared library file under the path specified by -L was linked.2.1.0(its soname is.2). It can therefore be concluded:-L specifies the pathSearch is prioritized higher thanDefault search pathThe search priority of the

A more detailed verification of the above conclusions will be given later in the DEBUG model.

DEBUG mode

In the makefile I added a target for comparing the priority of the three pathscmp_allwhich

• -L specifies that the path is/opt/hellolib_L
• The default path is/usr/lib
• LIBRARY_PATH specifies the path to the/opt/hellolib，.soDocumentation (.1.1.0soft links) are placed only under this path.

In addition, I also preset a DEBUG mode, turn on the DEBUG mode can view the detailed information of the compilation process, the way to turn on is to add after the commandDEBUG_MODE=1, for example:

make cmp_all DEBUG_MODE=1

We'll run it in DEBUG mode below.cmp_allView its output (there is a lot of output information, so I'll intercept the key parts to explain):

Compiler Configuration Details

Let's look at the compiler configuration details that gcc outputs during compilation:

The text in the image reads as follows:

LIBRARY_PATH=
	/usr/lib/gcc/x86_64-linux-gnu/11/:
	/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/:
	/usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/:
	/lib/x86_64-linux-gnu/:
	/lib/../lib/:
	/usr/lib/x86_64-linux-gnu/:
	/usr/lib/../lib/:
	/opt/hellolib/:
	/usr/lib/gcc/x86_64-linux-gnu/11/../../../:
	/lib/:
	/usr/lib/

We can find the compiler listing the contents of the system environment variable LIBRARY_PATH contained:

• ①What we added to the environment variable/opt/hellolib/The location should be specified by the compiler.
• ② The system default library path (/usr/libcap (a poem)/lib), located at the end
• ③ Paths that are automatically added according to the compiler configuration, such as/usr/lib/gcc/x86_64-linux-gnu/11/et al. (and other authors)

And then further down the line.COLLECT_GCC_OPTIONSLists the values passed to theg++Some of the options:

The text in the image reads as follows (omitting some of the ones that don't need attention):

COLLECT_GCC_OPTIONS=...
	-L/opt/hellolib_L
	-L/usr/lib/gcc/x86_64-linux-gnu/11 
	-L/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu 
	-L/usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib 
	-L/lib/x86_64-linux-gnu 
	-L/lib/../lib 
	-L/usr/lib/x86_64-linux-gnu 
	-L/usr/lib/../lib 
	-L/opt/hellolib 
	-L/usr/lib/gcc/x86_64-linux-gnu/11/../../.. 
	...

It can be found:

• ①We pass-LExplicitly added paths/opt/hellolib_LIt's at the top of the list.
• ②LIBRARY_PATH(in addition to the paths in the/usr/lib/cap (a poem)/lib/(for reasons unknown at this time), are added-Land passed it on toCOLLECT_GCC_OPTIONSside-by-side in the/opt/hellolib_LAfter.

Linker Details

Then we look at the details of the linker output:

The text in the image reads as follows:

SEARCH_DIR("=/usr/local/lib/x86_64-linux-gnu"); 
SEARCH_DIR("=/lib/x86_64-linux-gnu"); 
SEARCH_DIR("=/usr/lib/x86_64-linux-gnu"); 
SEARCH_DIR("=/usr/lib/x86_64-linux-gnu64"); 
SEARCH_DIR("=/usr/local/lib64"); 
SEARCH_DIR("=/lib64"); 
SEARCH_DIR("=/usr/lib64"); 
SEARCH_DIR("=/usr/local/lib"); 
SEARCH_DIR("=/lib"); 
SEARCH_DIR("=/usr/lib"); 
SEARCH_DIR("=/usr/x86_64-linux-gnu/lib64"); 
SEARCH_DIR("=/usr/x86_64-linux-gnu/lib");

SEARCH_DIRdirective is used to specify the directories that the linker should consider when searching for dynamic and static library files; these paths usually include the system's standard library directory, such as the/usr/libcap (a poem)/libetc. But note that by-LThe specified path is temporarily added at runtime to theSEARCH_DIRThe front of the list, i.e.-LThe specified path search has a higher priority.

DEBUG Summary

At this point, we can briefly summarize the above information:

• The value of LIBRARY_PATH that we set is passed to the compiler
• The compiler generates a new LIBRARY_PATH based on its own configuration and the value of the LIBRARY_PATH variable we manually assigned to it (the value of the LIBRARY_PATH variable we manually assigned to it is in a specific location) and takes the value of this new LIBRARY_PATH (in addition to the value of the/usr/libcap (a poem)/lib) plus-Lpass to the compiler
• We explicitly use-LThe specified path is also passed to the compiler and is located in all the-LTop of the options

And for compiler-configured paths such as/usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/The essence of this is that/usr/lib/(This is alsodefault pathPriority greater thanLIBRARY_PATH specifies the path(Reasons for Prioritization).

Therefore, for-L specifies the path，LIBRARY_PATH specifies the pathcap (a poem)default path, all of which are eventually transformed into-Lform is passed to the compiler, and they are prioritized:

-L specifies the path>default path>LIBRARY_PATH specifies the path

Therefore their search priorities are also in line with the above ranking.

validate (a theory)

Finally we can link specific libraries (such as ourlibhello) when the search process validates the above conclusions:

It can be seen that the linker first searches for the link we use with the-LSpecified path/opt/hellolib_LThen search for the path to the compiler configuration/usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/(which is essentially the default path)/usr/lib/), and finally search for the path specified by LIBRARY_PATH/opt/hellolib. It is proved that the priority of the library search path during linking in the compilation process is

-LSpecify the path >Default path >LIBRARY_PATH Specify the path

Default Path > LIBRARY_PATH Cause

As mentioned above, g++ will, for example, depending on its configuration:

/usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/

path is added to the LIBRARY_PATH and is located before the LIBRARY_PATH set by the user. The essence of this path is that the/usr/lib/. This results in ending up with a default path search priority that is greater than the search priority of the path specified by LIBRARY_PATH.

As for manual useldgo to the link.orespond in singing.sofile and test it later when I have a chance.

appendice

Library File Source Code

//file: 
#include <iostream>
void hello()
{
    std::cout << "Hello from the 1.1.0 library!" << std::endl;
}

//file: libhello_alt.cpp
#include <iostream>
void hello()
{
    std::cout << "Hello from the 2.1.0 library!" << std::endl;
}

Main program source code

//file: 
extern void hello();
int main()
{
    hello();
    return 0;
}

makefile

//file: makefile
CXX = g++
CXXFLAGS = -fPIC
LDFLAGS = -shared
DEBUG_MODE ? = 0

ifeq ($(DEBUG_MODE),1)
    DEBUG_OPTS = -v -Wl,--verbose
endif

all: lib/.1.1.0 lib/.2.1.0 obj/

lib/.1.1.0: $(CXX) $(CXXFL)
$(CXX) $(CXXFLAGS) -o $@ $^ $(LDFLAGS) -Wl,-soname,.1

lib/.2.1.0: libhello_alt.cpp
$(CXX) $(CXXFLAGS) -o $@ $^ $(LDFLAGS) -Wl,-soname,.2

obj/.
$(CXX) -c -o $@ $^

# Unable to search in any path
main_none: obj/
$(CXX) $(DEBUG_OPTS) -o $@ $^ -lhello

# Test default path /usr/lib
main_default: obj/
cp . /lib/.1.1.0 /usr/lib
ln -sf /usr/lib/.1.1.0 /usr/lib/
$(CXX) $(DEBUG_OPTS) -o $@ $^ -lhello

# Test using only LIBRARY_PATH
main_library_path: obj/
mkdir -p /opt/hellolib
cp . /lib/.1.1.0 /opt/hellolib
ln -sf /opt/hellolib/.1.1.0 /opt/hellolib/
LIBRARY_PATH=/opt/hellolib $(CXX) $(DEBUG_OPTS) -o $@ $^ -lhello

# Test using only -L
main_l: obj/
mkdir -p /opt/hellolib
cp . /lib/.1.1.0 /opt/hellolib
ln -sf /opt/hellolib/.1.1.0 /opt/hellolib/
$(CXX) $(DEBUG_OPTS) -o $@ $^ -L/opt/hellolib -lhello

# Compare the search priority of the default path and LIBRARY_PATH
cmp_default_libpath: obj/
cp . /lib/.2.1.0 /usr/lib
ln -sf /usr/lib/.2.1.0 /usr/lib/
mkdir -p /opt/hellolib
cp . /lib/.1.1.0 /opt/hellolib
ln -sf /opt/hellolib/.1.1.0 /opt/hellolib/
LIBRARY_PATH=/opt/hellolib $(CXX) $(DEBUG_OPTS) -o $@ $^ -lhello

# Compare the priority of -L and default paths
cmp_l_default: obj/
mkdir -p /opt/hellolib
cp . /lib/.2.1.0 /opt/hellolib
ln -sf /opt/hellolib/.2.1.0 /opt/hellolib/
cp . /lib/.1.1.0 /usr/lib
ln -sf /usr/lib/.1.1.0 /usr/lib/
$(CXX) $(DEBUG_OPTS) -o $@ $^ -L/opt/hellolib -lhello

# Overall comparison test, aggregated to show -L, LIBRARY_PATH and default paths
cmp_all.
mkdir -p /opt/hellolib
mkdir -p /opt/hellolib_L
cp . /lib/.1.1.0 /opt/hellolib
ln -sf /opt/hellolib/.1.1.0 /opt/hellolib/
LIBRARY_PATH=/opt/hellolib $(CXX) $(DEBUG_OPTS) -o $@ $^ -L/opt/hellolib_L -lhello

clean.
rm -f . /lib/* . /obj/* main_* cmp_*
rm -f /usr/lib/*
rm -rf /opt/hellolib*
ldconfig

.PHONY: clean main_none main_default main_library_path main_l cmp_default_libpath cmp_l_default cmp_all