Using Webshare Proxies with Python: requests, HTTPX and aiohttp

Before you start

You need a Webshare plan (the free plan is fine for testing) and your proxy details from the dashboard. Decide on a connection mode first, because the code differs:

• Direct: a list of IP:PORT:USERNAME:PASSWORD entries; your code rotates.
• Backbone: one gateway (p.webshare.io:80 in Webshare's docs) with rotate or session options in the username.

See backbone vs direct connection. Keep credentials in environment variables, not in code.

requests: one proxy

import os, requests

user, pw = os.environ["WS_USER"], os.environ["WS_PASS"]
proxy = f"http://{user}-rotate:{pw}@p.webshare.io:80"
r = requests.get("https://httpbin.org/ip", proxies={"http": proxy, "https": proxy}, timeout=30)
print(r.status_code, r.json())

The same http:// proxy URL is used for both HTTP and HTTPS targets; for HTTPS, requests opens a CONNECT tunnel through the proxy. General background in Python requests proxies.

requests: rotating through a direct list

import itertools, requests

def load(path):
    for line in open(path):
        ip, port, user, pw = line.strip().split(":")
        url = f"http://{user}:{pw}@{ip}:{port}"
        yield {"http": url, "https": url}

pool = itertools.cycle(list(load("webshare_proxies.txt")))
for url in ["https://example.com/a", "https://example.com/b"]:
    r = requests.get(url, proxies=next(pool), timeout=20)
    print(url, r.status_code)

Download the list from the dashboard, or fetch it with the API so replacements are picked up automatically.

Sticky sessions

Use a session number in the backbone username and reuse one requests.Session, so cookies and IP stay together:

import random, requests

sid = random.randint(1, 999_999)
proxy = f"http://{user}-{sid}:{pw}@p.webshare.io:80"
s = requests.Session()
s.proxies = {"http": proxy, "https": proxy}
s.get("https://example.com/login", timeout=30)

Copy the exact session format from your dashboard. More in Webshare sticky sessions.

SOCKS5

Install SOCKS support with pip install requests[socks], then use socks5h:// so DNS is resolved through the proxy:

proxy = f"socks5h://{user}:{pw}@PROXY_IP:PORT"
requests.get("https://httpbin.org/ip", proxies={"http": proxy, "https": proxy}, timeout=30)

HTTPX

import httpx

proxy = f"http://{user}-rotate:{pw}@p.webshare.io:80"
with httpx.Client(proxy=proxy, timeout=30) as client:
    print(client.get("https://httpbin.org/ip").json())

Older HTTPX versions use proxies= instead of proxy=. HTTPX also has an async client with the same option.

aiohttp for high concurrency

import asyncio, aiohttp

PROXY = f"http://{user}-rotate:{pw}@p.webshare.io:80"

async def fetch(session, url, sem):
    async with sem:
        async with session.get(url, proxy=PROXY, timeout=aiohttp.ClientTimeout(total=30)) as r:
            return url, r.status

async def main(urls):
    sem = asyncio.Semaphore(50)  # keep below your plan's thread limit
    async with aiohttp.ClientSession() as s:
        return await asyncio.gather(*(fetch(s, u, sem) for u in urls))

print(asyncio.run(main([f"https://example.com/p/{i}" for i in range(100)])))

Keep concurrency below your plan's thread limit. See Webshare bandwidth and threads.

Retries and error handling

import time, requests

def get(url, proxies, tries=4):
    for i in range(tries):
        try:
            r = requests.get(url, proxies=proxies, timeout=30)
            if r.status_code in (403, 429, 503):
                raise RuntimeError(f"blocked {r.status_code}")
            return r
        except (requests.RequestException, RuntimeError):
            time.sleep(2 ** i)
    raise RuntimeError(f"failed: {url}")

With backbone rotation, each retry automatically gets a new IP. With a direct list, pick a different entry on each retry. If you see 407, check credentials; see fixing common Webshare errors.

Tips

• Set timeouts on every call. Without one, a dead connection can hang forever.
• Reuse sessions for connection pooling and speed.
• Send realistic headers. Proxies change your IP, not your fingerprint.
• Measure bandwidth per page before you size a residential plan.

A reusable proxy-aware session

Wrap configuration in one helper so every script behaves the same:

import os, requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

def webshare_session(session_id=None, pool=20):
    user = os.environ["WS_USER"] + (f"-{session_id}" if session_id else "-rotate")
    url = f"http://{user}:{os.environ['WS_PASS']}@p.webshare.io:80"
    s = requests.Session()
    s.proxies = {"http": url, "https": url}
    retry = Retry(total=3, backoff_factor=1, status_forcelist=(429, 500, 502, 503, 504))
    s.mount("https://", HTTPAdapter(max_retries=retry, pool_maxsize=pool))
    s.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0 Safari/537.36"
    return s

Note that with connection pooling, a rotating gateway may reuse the same exit for requests on one connection. If you need a new IP every request, create a fresh session or set pool_maxsize=1 and close connections.

Detecting soft blocks

Many sites return status 200 with a challenge or empty page. Check content, not just status:

def looks_blocked(r):
    t = r.text.lower()
    return r.status_code in (403, 429) or "captcha" in t or "access denied" in t or len(t) < 500

Tune the checks to your target.

Logging that helps debugging

Log, for every failed request: time, URL, proxy mode, session ID (if any), status code and a short snippet of the body. When something breaks, these logs tell you whether it's credentials, the proxy, or the target.

Which product from Python

Plain HTTP clients are efficient, so a datacenter list covers most lenient sites cheaply. Use rotating residential for the targets that block it, and a dedicated static IP if your script logs into an account.

A real-world example

A data analyst writes a Python script that downloads public transport timetables from 25 regional websites every night. Twenty sites work through a Webshare datacenter list with simple rotation and a two-second delay per domain. Five sites block datacenter IPs, so the script routes those through the backbone gateway with the rotate option. A small retry helper with exponential backoff handles occasional timeouts, and the whole job finishes in under an hour with fewer than 1% failed requests.

Before you commit to any provider, it helps to see the wider market: our guide to the best proxies in 2026 compares proxy types by job, and cheap proxies explains how to keep costs down without buying IPs that get blocked.

Key takeaways

• Webshare proxies work in requests, HTTPX and aiohttp through standard proxy URLs.
• Use the backbone with rotate or session options, or rotate a direct list in code.
• Use socks5h:// for SOCKS5 so DNS resolves through the proxy.
• Set timeouts, retry with backoff and check content to catch soft blocks.
• Keep concurrency below your plan's thread limit.

FAQ

Does Webshare work with Python requests?

Yes. Use a standard proxy URL with credentials, either a direct IP or the backbone gateway.

Why is my IP not changing in Python?

You're probably using a direct IP (which never rotates) or a session username. Use the rotate option on the backbone, or rotate through your list in code.

Can I use Webshare with Selenium from Python?

Yes; see Webshare with Selenium.

What is socks5h in Python?

It tells the client to resolve DNS through the SOCKS5 proxy, which prevents DNS leaks.

How do I set a Webshare proxy for every request in a script?

Set session.proxies on a requests.Session and use that session for all requests.

Can I use environment variables for proxies in Python?

Yes. requests and HTTPX read HTTP_PROXY and HTTPS_PROXY, but explicit settings in code are clearer.