next-gen-vs-resi
avatar

Gabija Fatenaite

Aug 05, 2020 4 min read

Every year we invest in our products to provide best solutions to our clients, and this time we took it to another level. We gathered great minds and put hundreds of hours to create a new type of proxy – Next-Gen Residential Proxy!

This market innovation has been in testing for a few months now. After finally making all ends meet, we forged it into something we believe will make many of our clients’ lives much much easier. 

What are Next-Gen Residential Proxies?

Next-Gen Residential Proxies are built for heavy-duty data retrieval operations. They enable effortless web data extraction without any delays or errors. The product is as customizable as a regular proxy, but at the same time, it guarantees a much higher success rate. Custom headers and IP stickiness are both supported, alongside reusable cookies and POST requests.

If you have ever used regular proxies for data scraping, integrating Next-Gen Residential Proxies will be a breeze. The only difference is that we require you to accept our certificate, or ignore it altogether with -k or –insecure cURL flags (or equivalent expression in the language of your choice).

The main benefits of Next-Gen Residential Proxies

AI-powered dynamic fingerprinting

  • AI-powered algorithms allow you to imitate an organic user’s behavior more successfully than any other proxies.

AutoRetries system

  • Next-Gen Residential Proxies will automatically retry data extraction in case of unsuccessful attempt.

Adaptive parsing feature 

  • Powered by state-of-the-art ML algorithms.
  • Get high quality and parsed data from specific domains of websites (e.g., e-commerce, search engines).

Render JavaScript heavy websites at scale

  • Tailor your requests to render JavaScript website pages. We will render JS so you don’t need to.

Simple integration using standard proxy protocols

  • Easily switch to Next-Gen Residential Proxies using standard proxy protocols. 

Highly scalable and customizable

  • Supports high volume of requests by utilizing Oxylabs global 100M+ IP proxy infrastructure.
  • Tailor requests on country and city level (100+ countries) or by device.
  • Send your own cookies and headers, or let us take care of them.

The main differences between Next-Gen and Residential Proxies

The main differences between Next-Gen and Residential Proxies

Basic request

Residential Proxies

curl -v -x pr.oxylabs.io:7777 -U "customer-USERNAME:PASSWORD" "https://example.com"

Next-Gen Residential Proxies

curl -v -x -k ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" 

Next-Gen Residential Proxies use a different entry node and are accessible via a different port. It is necessary to use the “-k” flag, which lets cURL to not verify our TLS certificate. Additionally, please note that the Next-Gen Residential Proxies authentication string does not require the user to prepend “customer-”.

Geolocation (country-level)

Residential Proxies

curl -v -x pr.oxylabs.io:7777 -U "customer-USERNAME-cc-DE:PASSWORD" "https://example.com"

Next-Gen Residential Proxies

curl -v -x -k ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H "X-Next-Gen Residential Proxies-Geo-Location: Germany" 

Geolocation preference is sent in a header with Next-Gen Residential Proxies, instead of being inside the auth string (as in Residential Proxies).

Sticky IP (session)

Residential Proxies:

curl -v -x pr.oxylabs.io:7777 -U "customer-USERNAME-sessid-abcde12345:PASSWORD" "https://example.com"

Next-Gen Residential Proxies

curl-v -x -k ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H "X-Next-Gen Residential Proxies-Session-Id: abcde12345"

In both cases, the session ID is arbitrarily chosen  by the customer. The session ID is sent in a header with Next-Gen Residential Proxies and in the auth string with Residential Proxies.

Headers

Residential Proxies

curl -v -x pr.oxylabs.io:7777 -U "customer-USERNAME:PASSWORD" "https://example.com" -H "Some-Header: Value"

Next-Gen Residential Proxies

curl -v -x -k ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H "Some-Header: Value"

Headers are sent exactly the same way with both products, apart from differences outlined in the previous examples.

Cookies

Residential Proxies

curl -v -x pr.oxylabs.io:7777 -U "customer-USERNAME:PASSWORD" "https://example.com" -H "Cookie: NID=1234567890; 1P_JAR=0987654321"

Next-Gen Residential Proxies

curl -k -v -x ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H "Cookie: NID=1234567890; 1P_JAR=0987654321"

Cookies are sent exactly the same way with both products, apart from differences outlined in the previous examples.

POST

Residential Proxies:

 curl -X POST -v -x pr.oxylabs.io:7777 -U "customer-USERNAME:PASSWORD" "https://example.com" -d "@/path/to/file"

Next-Gen Residential Proxies

curl -X POST -k -v -x ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -d "@/path/to/file"

POST requests are sent exactly the same way with both products, apart from differences outlined in the previous examples.

What Next-Gen Proxies have and Residential ones don’t 

JavaScript rendering

Residential Proxies: N/A

Next-Gen Residential Proxies

curl -k -v -x ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H “X-Next-Gen Residential Proxies-Render: html”

Unlike Residential Proxies, Next-Gen Residential Proxies can render JavaScript before passing the result back. This feature may be useful in few cases: 

a) with sites which don’t load any useful content without the use of JavaScript 

b) when  the target site’s bot detection tool uses JavaScript rendering as a way to determine whether the site is being accessed by a bot

Custom Success Status Codes

Residential Proxies: N/A

Next-Gen Residential Proxies

curl -k -v -x ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H "X-Next-Gen Residential Proxies-Status-Code: 500,501,502,503"

Next-Gen Residential Proxies let the user specify custom success status codes. In the example above,  the query specifies that a response from the target website with a response code of 500, 501, 502 or 503 should be passed back to the user and is considered successful. This feature is exclusive to Next-Gen Residential Proxies and does not have an equivalent in Residential Proxies.

Conclusion 

Looking from a business perspective, upgrading to Next-Gen Residential Proxies will help you save up on proxy maintenance and development. Handling captchas, keeping up with website updates, and handling JavaScript rendering will cost you time, and accordingly – money.

Human resources, constant proxy management, and other technical maintenance will come out to be costly. Therefore, the shift from Residential Proxies is a very easy one, and the benefits are clearly beneficial.

If you have any further questions regarding Next-Gen Residential Proxies, please contact your account manager or [email protected]

avatar

About Gabija Fatenaite

Gabija Fatenaite is a Senior Content Manager at Oxylabs. Having grown up on video games and the internet, she grew to find the tech side of things more and more interesting over the years. So if you ever find yourself wanting to learn more about proxies (or video games), feel free to contact her - she’ll be more than happy to answer you.

Related articles

All information on Oxylabs Blog is provided on an "as is" basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Oxylabs Blog or any third-party websites that may be linked therein. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website's terms of service or receive a scraping license.