Python requests proxy (Proxy) usage tutorial

Python requests proxy (Proxy) usage tutorial

In Python's requests library, using a proxy server allows you to send HTTP requests over different network routes. A proxy server can help hide real IP addresses, bypass geo-restrictions, or load balancing.

What is an agent?
A proxy server is an intermediate server that is located between the client (your code) and the target server (the server you want to request). Using a proxy server, your request will be sent to the proxy first, and then the proxy forwards the request to the target server, and the target server's response will also be returned to you through the proxy.

The use of agents is very useful in scenarios such as data crawling, accessing restricted websites, and improving privacy protection.

Use proxy in requests

requests supports HTTP, HTTPS, SOCKS and other proxy requests;

Using HTTP and HTTPS Proxy

If you want to use an HTTP proxy or an HTTPS proxy, you can pass it in any request methodproxiesParameters to configure a single request:

import requests

 proxies = {
   "http": "http://proxy1:8080",
   "https": ":8080",
 }

 #Format with authentication
 proxies2 = {
     'http': 'http://user:password@:8080',
     'https': 'https://user:password@:8080',
 }

 response = ("", proxies=proxies)

In this example, the http request will passhttp://proxy1:8080Send, https request will pass:8080send;

Setting up proxy through environment variables

Except for explicitly passing in the codeproxiesParameters, you can also configure proxy through environment variables, the requests library will automatically read these environment variables and apply proxy settings.

Windows environment variable setting command

set HTTP_PROXY=:8080
set HTTPS_PROXY=:8080

Linux and macOS environment variable setting commands

export http_proxy=:8080
export https_proxy=:8080