What the Scraper API does
Raw proxies give you IPs; everything else (rotation, retries, browsers, block handling, parsing) is your job. A scraping API does more of that for you: you send a URL and options, and get back the page content. Geonode offers a Scraper API that returns pages as HTML or Markdown, with optional JavaScript rendering, plus related APIs:
| API | What it does |
|---|---|
| Scraper API | Fetch one URL and return HTML or Markdown, optionally rendered |
| Map API | Discover a website's URLs before choosing which to extract |
| Crawl API | Discover and extract pages from a site in one job |
| Search API | Return structured web search results for a query |
Geonode also offers an MCP server that exposes these tools to AI assistants, and SDKs for its APIs. At the time of writing it includes a free monthly allowance of API requests and states that a request counts as one request, without multipliers for JavaScript rendering, geo-targeting or batch calls. Check the documentation for current endpoints, parameters and limits.
API or raw proxies?
| Situation | Better choice |
|---|---|
| A few hundred or thousand pages a month from hard targets | API |
| JavaScript-heavy pages and no appetite for running browsers | API |
| Feeding content into LLMs or agents | API with Markdown output |
| Millions of pages from lenient targets | Raw proxies (datacenter) |
| Complex interactions: logins, forms, multi-step flows | Raw proxies with your own browser |
| You need full control over headers, fingerprints and timing | Raw proxies |
Many teams use both: raw proxies for high-volume core targets and the API for awkward or low-volume ones. See scraping API vs proxies.
Why Markdown output matters
HTML is verbose and full of navigation, scripts and styling. Markdown keeps the structure that matters (headings, lists, tables, links) and drops the rest. For AI uses, that means fewer tokens, lower cost and better answers. For humans, it means readable content you can store, diff and search. See Geonode for AI agents.
JavaScript rendering
Some pages only contain their content after scripts run. The Scraper API can render pages with JavaScript before returning them, which saves you running and maintaining headless browsers. Use rendering only where needed: static pages return faster without it.
Proxy tip: Before scaling, run a short pilot on your real targets with Geonode and record success rate, latency and GB per result. Our our complete Geonode guide explains which Geonode product to start with.
Integration patterns
- Direct HTTP calls from any language: send the URL and options, receive content.
- SDKs for supported languages.
- Low-code tools such as n8n through HTTP request nodes. See Geonode with n8n.
- AI hosts through the MCP server.
- Crawl jobs for whole sites, using the Map API first to choose which URLs matter.
Cost thinking
Per-request pricing is easy to predict: count the pages you need. Compare it with raw proxies by including everything a DIY setup costs: bandwidth (especially for browser rendering), retries, CAPTCHA handling and engineering time. For small volumes and hard targets, the API is often cheaper overall. For large volumes on easy targets, raw datacenter proxies usually win on unit cost.
Good practice
The API handles fetching, but the usual rules still apply: respect robots.txt and site terms, avoid collecting personal data you don't need, and keep volumes proportionate. See ethical web scraping.
Budget tip: For jobs where a fixed IP list is enough, Webshare is one of the cheapest options: shared datacenter proxies are sold by the IP, and static residential (ISP) proxies cost well under a dollar per IP per month at the time of writing. More in Webshare for web scraping.
A decision walkthrough
Here's how a small team might decide, target by target, between the Scraper API and raw proxies.
| Target | Volume | Difficulty | Decision | Why |
|---|---|---|---|---|
| Government statistics portal | 50,000 pages a month | Easy | Raw datacenter proxies | Cheap per page, no blocking |
| Two large retailers | 200,000 pages a month | Hard | Raw residential proxies with an HTTP client | Volume justifies engineering; JSON endpoints available |
| A JavaScript-heavy competitor site | 3,000 pages a month | Hard, rendered | Scraper API with rendering | Low volume; no browser to maintain |
| Documentation sites for an AI assistant | A few thousand pages, refreshed weekly | Easy, but want Markdown | Crawl API with Markdown output | Clean input for the model with no parsing |
| Ad-hoc research questions | Unpredictable | Mixed | Search API plus Scraper API | Find sources, then read them |
The pattern: raw proxies for high-volume targets you understand, APIs for low-volume, high-difficulty or AI-facing work. Revisit decisions as volumes change; a target that grows from thousands to hundreds of thousands of pages a month may justify moving to raw proxies, and one that becomes harder may justify the reverse.
Residential tip: If you want to try this on an ethically sourced residential pool, Rayobyte offers a free trial with no credit card, then pay-as-you-go traffic from $3.50 per GB that drops to $0.70 per GB at 1 TB. See the Rayobyte Scraping API.
Getting started
- Create a Geonode account and get an API key.
- Try the Scraper API on two or three of your real target URLs, with and without JavaScript rendering, in both HTML and Markdown output.
- Compare the output with what your own scraper would produce, and note response times.
- Estimate monthly requests and compare the cost with your current or planned proxy setup, including engineering time.
- Start with the targets where the API clearly wins, and keep raw proxies for the rest.
The free monthly allowance listed at the time of writing is usually enough to run this evaluation without paying anything.
If you prefer raw proxies, compare providers in our guide to the best proxies in 2026.
Comparing two popular self-serve providers? Webshare vs Geonode for Scraping APIs breaks down which fits this job and when it makes sense to use both.
Considering Rayobyte instead of, or alongside, Geonode for this? Rayobyte vs Geonode for Web Scraping compares the two, and Rayobyte Scraping API covers the Rayobyte setup.
FAQ
What output formats does the Scraper API return?
Geonode lists HTML and Markdown output.
Does JavaScript rendering cost more?
Geonode states that requests aren't multiplied for JavaScript rendering. Confirm current pricing in the documentation.
Is there a free tier?
At the time of writing, Geonode lists a free monthly allowance of API requests with no credit card required.
Can I crawl a whole website?
Yes. Use the Map API to discover URLs and the Crawl API to extract pages in one job.
Sources and further reading
Independent references that back up the points above, from standards bodies, governments, universities and non-profits. They open in a new tab.
- Bandwidth (computing) (Wikipedia): What bandwidth means, the unit most residential proxies are priced by.
- RFC 9309: Robots Exclusion Protocol (IETF / RFC Editor): The official standard for robots.txt, which polite scrapers and crawlers read first.
- Common Crawl (Common Crawl Foundation): The open web crawl that many AI training datasets are built from.
- urllib.request documentation (Python Software Foundation): Python's standard library documentation, including ProxyHandler.
- RFC 9110: HTTP Semantics (IETF / RFC Editor): The HTTP standard that defines how proxies, tunnels and the CONNECT method behave.
Editorial note: Proxy providers change their pools, features and prices often, so treat any figures here as typical ranges at the time of writing and confirm current terms with the provider before you buy.