How to Use Residential Proxies

Credentials, code for curl, Python, Node, Playwright, Puppeteer and Scrapy · checked 3 November 2026
Setup guide

A residential proxy routes your request through a real home internet connection, so the target site sees a consumer ISP address instead of your server's datacenter IP. This guide covers the whole path: where the credentials come from, how targeting is encoded in the username, working code for five stacks, and what to check when a request fails. Every snippet below runs as written — replace the host, port, account and password with the values from your console.

What you need before you start

The one rule that saves hours: the gateway host and port are not the same as your account panel. Copy the endpoint line the console generates, character for character. Most "connection refused" reports are a host or port that was typed from memory.

Step 1 — Read the username, because that is where targeting lives

IPLight uses a single endpoint for the whole pool. Country, region, city and session behaviour are all encoded in the username as hyphen-joined key-value pairs. This is why you never need a different port per country.

ParameterExampleWhat it does
countrycountry-usExit country. Lowercase ISO-3166 alpha-2; omit for global mix.
statestate-nyUS state or region code, when the target pool supports it.
citycity-newyorkCity inside that state, lowercase, no spaces.
sidsid-AbCdEf123456Session ID. Same value keeps the same exit IP; new value rotates.
stimestime-30Minutes to hold the session, 1–120. Applies only with sid.

A typical username looks like this:

# sticky session: United States, New York, hold the same IP for 30 minutes
ACCOUNT-country-us-state-ny-city-newyork-sid-AbCdEfGhIjKl-stime-30

# rotating: United Kingdom, new IP on every request
ACCOUNT-country-gb-sid-ZzYyXxWwVvUu

Two things trip people up. First, codes are lowercase ISO-3166 alpha-2 for countries and the region codes listed in the console for states. Second, once you set sid, the gateway pins that session to one exit IP for stime minutes — if you want a fresh IP, change the sid, do not just wait.

Step 2 — Verify the exit before you write any code

One curl call tells you whether the credential, the endpoint and your network are all correct. Run this first; if it fails, nothing downstream will work.

# replace ACCOUNT, PASSWORD, GATEWAY_HOST and PORT
curl -x "http://ACCOUNT-country-us-sid-Test01-stime-10:PASSWORD@GATEWAY_HOST:PORT" \
     https://ipinfo.io/json

A correct response contains a US city, a consumer ISP in org, and an ip that is not your server. If you see your own datacenter IP, the proxy was not applied: on Linux and macOS check that no HTTP_PROXY / HTTPS_PROXY variable is overriding it, and on Windows check the system proxy settings.

Step 3 — Python with requests

The same credential string works for both schemes. Rotating and sticky behave identically in code; only the username changes.

import requests

USER = "ACCOUNT-country-us-sid-PyTest01-stime-10"
PWD  = "PASSWORD"
PROXY = "http://%s:%s@GATEWAY_HOST:PORT" % (USER, PWD)

session = requests.Session()
session.proxies = {"http": PROXY, "https": PROXY}

r = session.get("https://ipinfo.io/json", timeout=30)
print(r.status_code, r.json())

# rotate: build a new sid per request, reuse the TCP connection pool
import uuid
for i in range(5):
    user = "ACCOUNT-country-de-sid-%s" % uuid.uuid4().hex[:12]
    p = "http://%s:%s@GATEWAY_HOST:PORT" % (user, PWD)
    print(requests.get("https://ipinfo.io/ip", proxies={"http": p, "https": p}, timeout=30).text)
Set a timeout on every call. A dead upstream peer without a timeout will hang your worker pool. 20–30 seconds is a sane default for page requests; lower it for API calls.

Step 4 — Node.js

// npm i undici
import { ProxyAgent, request } from "undici";

const user = "ACCOUNT-country-us-sid-Node01-stime-10";
const agent = new ProxyAgent(`http://${user}:PASSWORD@GATEWAY_HOST:PORT`);

const res = await request("https://ipinfo.io/json", { dispatcher: agent });
console.log(res.statusCode, await res.body.json());

If your project is on axios, pass httpsAgent and httpAgent built with https-proxy-agent instead — do not rely on the deprecated proxy option, which drops credentials on HTTPS.

Step 5 — Playwright and Puppeteer

Browser automation is where credentials usually break: Chromium's --proxy-server flag has no way to carry a username and password. Playwright and Puppeteer solve it in the API, which is why they are the pragmatic choice for headless work behind an authenticated gateway.

# Playwright (Python) — proxy auth is native
from playwright.sync_api import sync_playwright

USER = "ACCOUNT-country-us-city-newyork-sid-Pw01-stime-30"
with sync_playwright() as p:
    browser = p.chromium.launch(proxy={
        "server":   "http://GATEWAY_HOST:PORT",
        "username": USER,
        "password": "PASSWORD",
    })
    page = browser.new_page()
    page.goto("https://ipinfo.io/json", timeout=60000)
    print(page.inner_text("body"))
    browser.close()
// Puppeteer — authenticate the page, not the launch flag
const browser = await puppeteer.launch({
  headless: true,
  args: ["--proxy-server=http://GATEWAY_HOST:PORT"],
});
const page = await browser.newPage();
await page.authenticate({ username: USER, password: "PASSWORD" });
await page.goto("https://ipinfo.io/json", { waitUntil: "domcontentloaded" });
Selenium note. Selenium's Chrome options still cannot send proxy credentials. Use Playwright, or run a local forwarding proxy (for example a small gost instance) that adds the upstream credential, then point Selenium at 127.0.0.1. Do not paste credentials into --proxy-server; Chromium silently ignores them and you end up scraping from your own IP.

Step 6 — Scrapy

# settings.py
DOWNLOADER_MIDDLEWARES = {
    "scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware": None,
    "myproject.middlewares.RotatingProxy": 543,
}
# myproject/middlewares.py
import uuid

class RotatingProxy:
    def __init__(self, settings):
        self.pwd = settings.get("PROXY_PASSWORD")

    @classmethod
    def from_crawler(cls, crawler):
        return cls(crawler.settings)

    def process_request(self, request, spider):
        user = "ACCOUNT-country-us-sid-%s" % uuid.uuid4().hex[:12]
        request.meta["proxy"] = "http://%s:%s@GATEWAY_HOST:PORT" % (user, self.pwd)

Rotating per request at the middleware level is the cheapest way to spread load: each item gets a fresh exit, and retries naturally land on a different IP.

Choosing rotating versus sticky

JobModeWhy
Price monitoring on one marketplaceSticky, 10–30 minThe basket and the product page must come from one session or the price context breaks.
SERP rank trackingRotatingOne query, one IP. Google ties repeated queries to a single address fast.
Bulk product detail crawlRotatingThroughput matters and every page is independent.
Account-based workflowSticky, up to 120 minLogin state and cookies must stay on one exit for the whole run.
Ad verificationRotating by cityYou are sampling what a real user in that city sees.

Troubleshooting, in the order you should check

SymptomMost likely causeFix
407 Proxy Authentication RequiredWrong user or password, or special characters unescapedRe-copy from the console; URL-encode @ : / ? # in the password.
429 or immediate blocksToo many concurrent requests to one hostLower concurrency per domain, rotate per request, add backoff.
Connection timed outOnly one peer in the session was unreachableRetry with a new sid; add a 20–30s timeout so workers do not stall.
403 from the targetThe site blocks residential ranges it has already burnedChange country or city, or move to a different session; check headers for TLS fingerprint mismatch.
502 / 504 from the gatewayUpstream peer dropped the connectionRetry once with a new session; if it persists across countries, check your own egress.
Requests go out from your own IPClient ignored the proxy settingVerify with ipinfo; remove system proxy variables; in Selenium do not use --proxy-server for auth.

Use it responsibly

Proxies change where a request comes from; they do not change what you are allowed to do. Collect only public data you have a right to access, respect the target's terms of service and robots directives, keep request rates at a level a human could plausibly generate, and never use a residential exit for fraud, credential stuffing or evading a ban you have already been given. In most jurisdictions the lawful basis for collection is your responsibility, not your proxy provider's.

Frequently asked questions

What is the proxy username format at IPLight?

The username carries your account ID plus targeting parameters joined by hyphens, for example ACCOUNT-country-us-state-ny-city-newyork-sid-AbCdEfGhIjKl-stime-30. The password is the proxy password issued in the console. You pass this pair to any HTTP or SOCKS5 client.

Do I need to whitelist my server IP?

No. IPLight uses username and password authentication, so you can run the same credentials from several machines, containers or serverless functions without managing allowlists. If you prefer IP-based auth for a fixed fleet, contact support before you scale.

Why do I get 407 Proxy Authentication Required?

Either the username or password is wrong, or your client is sending them in a form the gateway cannot parse. URL-encode special characters, keep the parameter order as generated in the console, and make sure the scheme is http:// even when you are fetching an https:// URL — the tunnel is established with CONNECT.

Why does the same request work in curl but fail in Python?

Most often because the environment variable HTTP_PROXY is set on the host and takes priority, or because requests is given a SOCKS URL without the socks extra installed. Pass proxies explicitly to each call and install requests[socks] if you use the socks5 scheme.

How long can a sticky session last?

Up to 120 minutes per session ID. The session stays on one exit IP for the stime you set and then rotates automatically. If your workflow runs longer, create a new session ID when the old one expires rather than assuming the IP is still alive.

Will one residential IP work for an entire scraping job?

Only for jobs that need a continuous identity, such as a logged-in workflow or a multi-step checkout. For read-only crawling, rotate per request: it spreads load across the pool, keeps per-IP request counts low and produces fewer blocks than a single sticky IP hammering one host.

Test it on your own target list

Residential from $1/GB billed by traffic, static ISP from $2/IP per month from 15 IPs, no monthly commitment. Create an account and check your success rate before you buy volume.

Create account