I’ve seen this commonly from them with residential service. They mostly give you a private v4 address behind CGNAT in addition to IPv6. If you want a public v4 they put you on their old network which has no v6.
It’s very tricky because the IPs are all on normal user ranges you can’t block without blocking those users.
The company behind this blog - spur.us - offer some paid services I think. There is also this project from Wikimedia which uses that data to produce more manageable lists:
Bear in mind the scrapers wouldn’t need to use these proxies were they not being blocked by the sites they are scraping. So it’s being used to evade blocks.
For some content the level of scraping is outweighing real users, driving up costs and pushing them towards more closed models.
Wikipedia for example make content available free, if you start hammering the site they will rate limit you to keep the lights on. If you need the data fast in bulk they have a paid program to get it without scraping. But some prefer to neither adhere to reasonable request limits nor pay for their use of the infra; instead they choose to pay these grifters to avoid the rate limits.
Nah it’s messing up peoples home internet, and massively abused to perform denial of service attacks and scraping of web content by large AI companies who are otherwise blocked.
Your vision of a randomly-routed mesh internet overlay is also not very scalable bandwidth or latency wise.
Implementing v6 without doubt saves money for ISPs. Especially in the CGNAT game as those boxes aren’t cheap.
reply