Rayobyte for Web Scraping: Which Product to Use and How to Set It Up

Why Rayobyte suits scraping

Scraping projects rarely have one kind of target. Some sites accept any IP; some block datacenter ranges; a few defeat almost everything. Rayobyte sells a product for each level, all in one account, so you can match cost to difficulty domain by domain. If you're new to scraping proxies, start with our general web scraping proxies guide.

The tiered setup

For huge untargeted crawls, Scale Residential replaces tier 3 at around a cent per GB.

Routing by domain

Start every new domain at tier 1. If its block rate passes a threshold you choose (5 to 10% is common), promote it a tier. Keep a small table of domain to tier, and re-test promoted domains every few weeks; sometimes a site relaxes.

import os, requests

TIERS = {
    1: os.environ["RAYOBYTE_DC_PROXY"],
    2: os.environ["RAYOBYTE_ISP_PROXY"],
    3: os.environ["RAYOBYTE_RESI_PROXY"],
}
ROUTES = {"easy-shop.example": 1, "medium-shop.example": 2, "hard-shop.example": 3}

def fetch(url, domain):
    proxy = TIERS[ROUTES.get(domain, 1)]
    return requests.get(url, proxies={"http": proxy, "https": proxy}, timeout=30)

Each proxy URL comes from the dashboard's generator. More complete examples are in Rayobyte with Python and Rayobyte with Scrapy.

Sessions

Rotate per request for independent pages like product details. Use a short sticky session for paginated lists or anything that sets cookies you need on the next page. Premium Residential sessions last 1 to 120 minutes and can end early, so retry with a new session on failure. See Rayobyte sticky sessions.

Browsers or HTTP clients?

Use a plain HTTP client whenever the data is in the HTML or a JSON endpoint; it's ten times cheaper in bandwidth and compute. Use a browser only when content needs JavaScript. For browser work, see Rayobyte with Puppeteer, Playwright or the hosted rayobrowse. Our Puppeteer proxies guide covers the full setup.

Cost example

A price intelligence project fetches 1 million pages a month across 40 domains. Average page weight is 150 KB, about 150 GB in total.

• All on Premium Residential at the 50 to 249 GB tier: about $300.
• Routed (70% datacenter, 20% rotating ISP, 10% residential): about $32 + $105 + $53 = roughly $190, before any monthly plan discounts, and less again if the ISP share moves to a monthly plan.

The saving comes almost entirely from moving easy domains off residential.

Mistakes that waste bandwidth

• Loading full pages in a browser when an API endpoint has the data.
• Not blocking images and fonts in browsers.
• Unlimited retries on a blocked domain.
• Ignoring compression. Accept gzip and brotli.
• Fetching unchanged pages. Use conditional requests or sitemaps to skip them.

Staying polite and legal

Cap per-domain concurrency, respect robots rules where they apply, and avoid personal data you don't need. Remember Rayobyte blocks login pages, banks and government sites on residential traffic. See ethical web scraping and compliance review and restricted sites.

How it compares

Geonode offers per-GB residential that doesn't expire, rotating datacenter and a Scraper API; Webshare offers cheap datacenter lists and rotating residential. Many teams combine providers. See Rayobyte vs Geonode for web scraping, Rayobyte vs Webshare for web scraping and all three for web scraping.

Rayobyte, Geonode and Webshare are three options among many. Our best proxies in 2026 guide explains how to match a proxy type to each task, and cheap proxies that work shows where budget providers fit.

Key takeaways

• Route each domain to the cheapest Rayobyte product that works: datacenter, ISP, residential or managed.
• Start every domain at datacenter and promote only on failure.
• Use HTTP clients over browsers wherever possible.
• Tiered routing cuts costs substantially compared with all-residential, mostly by moving easy domains to datacenter.

FAQ

What's the best Rayobyte product for scraping?

There isn't one: use rotating datacenter for easy sites, rotating ISP or Premium Residential for protected ones, and the Web Unblocker or Scraping API for the hardest.

Can I scrape JavaScript sites with Rayobyte?

Yes, with your own headless browser and Rayobyte proxies, with rayobrowse, or with the Scraping API, which includes browsers.

How much bandwidth does scraping use?

It depends on page weight. Light HTML pages may be 50 to 150 KB; full browser loads can be several MB.