What price monitoring involves
Price monitoring means checking the same products on competitors' sites on a schedule and recording price, stock, promotions and sometimes delivery options. Brands use it to enforce minimum advertised prices, retailers use it to reprice, and analysts use it to study markets.
It's a repetitive, long-running job, so two things matter more than anything else: reliability over months, and cost per check. Retailers are among the most protected sites on the web, and many show different prices by region, which is where proxies come in. Geonode fits this job well because you can mix cheap datacenter traffic for small stores with residential traffic for major retailers, and target residential traffic by country, state or city when location changes the price.
Choosing the right Geonode product per retailer
| Retailer type | Typical protection | Geonode product |
|---|---|---|
| Small independent stores | Little or none | Rotating datacenter |
| Mid-sized Shopify or WooCommerce stores | Basic rate limits | Rotating datacenter, residential if blocked |
| Large national retailers | Commercial bot protection | Rotating residential |
| Marketplaces | Strong protection, frequent changes | Rotating residential, or the Scraper API for a few products |
| Retailers with regional pricing | Varies | Residential with state or city targeting |
Test every retailer on datacenter traffic first. If success is above about 90% at your planned rate, keep it there.
Handling regional prices
Many retailers price by location: by country, by state (taxes and shipping), or by store. Getting the right price means making the retailer believe you're in the right place.
- Target the exit location in Geonode: country for national pricing, state or city where the retailer localises.
- Set the retailer's own location if it has a store picker or ZIP code field. Do this once per sticky session, then fetch a batch of products in that session.
- Record the location used with every price, so you can explain differences later.
- Verify a sample manually through a browser using the same exit location.
See geo-targeted proxies for how location targeting works and where it can be inaccurate.
Session settings that work
- Rotating sessions for independent product pages when location doesn't matter.
- Sticky sessions of 5 to 15 minutes when you need to set a store or ZIP first. One session per location per batch.
- Separate sessions per location. Never mix locations in one session.
Proxy tip: You can try every step in this guide with a small balance: Geonode bills residential bandwidth per GB with no expiry, so testing costs very little. See our complete Geonode guide for the full product overview.
Fetching efficiently
Monitoring checks the same pages repeatedly, so efficiency compounds.
- Find the data source. Many product pages load price and stock from a JSON endpoint. Fetching that directly is far lighter than loading the page.
- Use an HTTP client for server-rendered pages, and a browser only where prices are generated by scripts.
- Schedule by volatility. Check fast-moving products often and stable ones daily.
- Skip unchanged pages with conditional requests where supported.
On per-GB residential billing, these choices often make a tenfold difference. And because Geonode's residential bandwidth doesn't expire, a monitoring job with uneven volume (heavier around sales events, lighter in between) doesn't waste unused traffic. See cheap proxies.
Validating the data
Bad prices are worse than missing prices, because they drive bad decisions.
- Detect block pages by checking for the price element, not the status code.
- Flag outliers: a price that moves more than a set percentage since the last check should be re-checked before it's trusted.
- Watch for variants. Size, colour or pack-size variants can change the price shown; make sure you're reading the same variant each time.
- Account for personalisation and A/B tests. Some retailers test prices. Repeating a check from a fresh session helps separate tests from real changes.
- Log context: timestamp, location, session and the raw value before parsing.
A typical setup
A brand monitoring 1,000 products across 15 retailers might run:
- Ten small retailers on rotating datacenter traffic, HTML only.
- Four large retailers on rotating residential, using JSON endpoints where available.
- One retailer with regional pricing on state-targeted sticky sessions, five states.
- Daily checks for most products, hourly for 50 key products during promotions.
Bandwidth for a setup like that is usually modest when fetching HTML or JSON, often a few GB of residential traffic a month. See price monitoring proxies for more patterns.
Budget tip: If you want a cheap proxy to try this with, Webshare has a free plan with 10 datacenter proxies and 1 GB of bandwidth a month, no card required. Paid lists let you pick proxy count, bandwidth and location. See Webshare for price monitoring.
Common problems
- Prices missing on some runs: usually blocks served as normal pages. Add content checks and retries with a new IP.
- Prices from the wrong region: the exit location or the retailer's location setting doesn't match. Set both explicitly.
- Sudden cost increase: a site redesign made pages heavier, or a retry loop is running. Check bytes per successful check.
- One retailer starts failing: it changed its protection. Move it to residential, or to the Scraper API if it's only a few products.
Residential tip: Rayobyte is worth testing when you need precise targeting: its residential proxies can be pinned to a country, state, city, ZIP code or ASN, with sticky sessions from 1 to 120 minutes. More in Rayobyte for price monitoring.
Staying within the rules
Collecting publicly displayed prices is common practice, but check retailers' terms, avoid logging in to accounts you don't own, keep request rates polite, and don't collect personal data such as reviewer names unless you need it and have a lawful basis. See ethical web scraping.
Most price checks run on residential proxies for accurate local prices. To compare Geonode with other providers, see our guide to the best proxies in 2026.
Comparing two popular self-serve providers? Webshare vs Geonode for Price Monitoring breaks down which fits this job and when it makes sense to use both.
Rayobyte is another option here: our Rayobyte vs Geonode for Price Monitoring comparison shows where each provider is stronger, and Rayobyte for Price Monitoring walks through Rayobyte's products for the job.
FAQ
Do I need residential proxies for price monitoring?
For major retailers and marketplaces, usually yes. For small stores, datacenter traffic often works. Test each retailer.
How do I monitor prices in different states or cities?
Use Geonode's state or city targeting, set the retailer's own location in a sticky session, and record the location with each price.
How often should I check prices?
As often as the prices actually change and the decision needs. Hourly checks on stable products waste bandwidth; daily is enough for most.
Can I monitor Amazon prices with Geonode?
Technically yes, but read Amazon's terms and consider its official APIs first. See Geonode for Amazon data.
Sources and further reading
Independent references that back up the points above, from standards bodies, governments, universities and non-profits. They open in a new tab.
- 429 Too Many Requests (MDN Web Docs (Mozilla)): What a 429 response means and how servers use it.
- Automated Threats to Web Applications (OWASP): OWASP's catalogue of automated threats, including scraping and scalping, as site owners see them.
- Bandwidth (computing) (Wikipedia): What bandwidth means, the unit most residential proxies are priced by.
- urllib.request documentation (Python Software Foundation): Python's standard library documentation, including ProxyHandler.
- RFC 9293: Transmission Control Protocol (TCP) (IETF / RFC Editor): The current TCP specification, the transport under HTTP and SOCKS proxies.
Editorial note: Proxy providers change their pools, features and prices often, so treat any figures here as typical ranges at the time of writing and confirm current terms with the provider before you buy.