next-gen-vs-resi
avatar

Gabija Fatenaite

Aug 05, 2020 6 min read

Every year we invest in our products to provide the best solutions to our clients, and this time we took it to another level. We gathered great minds and put hundreds of hours to create a new type of proxy – Next-Gen Residential Proxy!

We have been testing our market innovation for a few months now. After finally making all ends meet, we forged it into something we believe will make many of our clients’ lives much easier.

What are Next-Gen Residential Proxies?

Next-Gen Residential Proxies are built for heavy-duty data retrieval operations. They enable effortless web data extraction without any delays or errors. Our solution is as customizable as any regular proxy, but at the same time, guarantees a much higher success rate. Custom headers and IP stickiness are both supported, alongside reusable cookies and POST requests.

If you have ever used regular proxies for data scraping, integrating Next-Gen Residential Proxies will be a breeze. The only difference is that we require you to accept our certificate, or ignore it altogether with -k or –insecure cURL flags (or equivalent expression in the language of your choice).

Next-Gen Residential Proxy features

AI-powered dynamic fingerprinting

  • AI-powered algorithms automatically generate optimal fingerprints for each target by using browser-specific combinations of user-agents and headers.
  • An optimal fingerprint greatly reduces the chance of getting blocked and delivers a better success rate for each request made.

Auto-Retry system

  • Next-Gen Residential Proxies will automatically retry data extraction in order to achieve the highest possible success rate.
  • In the case of an unsuccessful request, the Auto-Retry system will attempt a variety of tactics to ensure that data extraction eventually succeeds.

Adaptive parsing feature 

  • Powered by state-of-the-art ML algorithms.
  • Get high quality and parsed data from specific domains of websites (e.g., e-commerce, search engines) instead of maintaining many different parsers.
What is adaptive parser

Render JavaScript heavy websites at scale

  • Next-Gen Residential Proxies can render dynamically loaded content on websites.
  • Users will no longer need to use headless browsers to render page content as Next-Gen Residential Proxies will be able to load all data through JavaScript calls before it’s passed sent back.

Simple integration using standard proxy protocols

  • Easily switch to Next-Gen Residential Proxies using standard proxy protocols.
  • Getting started is as easy as typing in our single entry node.

Highly scalable and customizable

  • Supports high volume of requests by utilizing Oxylabs global 100M+ IP proxy infrastructure.
  • Tailor requests on country level (100+ countries) or by device.
  • Send your own cookies and headers, or let us take care of them.

Want to find more about Next-Gen Residential Proxies? Watch our free webinar to get the most in-depth and up-to-date information!

The Future of Web Scraping – Next-Gen Residential Proxies

!

This field is required

Thank you!

Please check your inbox.

We will use your email to send you a link to the webinar. We will also provide you with information on Oxylabs’ services that may be of interest to you. Be sure that you can opt-out from any marketing related communications that we send you at any time. For more information on your rights and data use please read our Privacy Policy.
Comparing Residential Proxy Solutions for Data Acquisition

The main differences between Next-Gen and Residential Proxies

Basic request

Residential Proxies

curl -v -x pr.oxylabs.io:7777 -U "customer-USERNAME:PASSWORD" "https://example.com"

Next-Gen Residential Proxies

curl -v -x -k ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" 

Next-Gen Residential Proxies use a different entry node and are accessible via a different port. It is necessary to use the “-k” flag, which lets cURL to not verify our TLS certificate. Additionally, please note that the Next-Gen Residential Proxies authentication string does not require the user to prepend “customer-”.

Geolocation (country-level)

Residential Proxies

curl -v -x pr.oxylabs.io:7777 -U "customer-USERNAME-cc-DE:PASSWORD" "https://example.com"

Next-Gen Residential Proxies

curl -v -x -k ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H "X-Oxylabs-Geo-Location: Germany" 

Geolocation preference is sent in a header with Next-Gen Residential Proxies, instead of being inside the auth string (as in Residential Proxies).

Sticky IP (session)

Residential Proxies:

curl -v -x pr.oxylabs.io:7777 -U "customer-USERNAME-sessid-abcde12345:PASSWORD" "https://example.com"

Next-Gen Residential Proxies

curl-v -x -k ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H "X-Oxylabs-Session-Id: abcde12345"

In both cases, the session ID is arbitrarily chosen  by the customer. The session ID is sent in a header with Next-Gen Residential Proxies and in the auth string with Residential Proxies.

Headers

Residential Proxies

curl -v -x pr.oxylabs.io:7777 -U "customer-USERNAME:PASSWORD" "https://example.com" -H "Some-Header: Value"

Next-Gen Residential Proxies

curl -v -x -k ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H "Some-Header: Value"

Headers are sent exactly the same way with both products, apart from differences outlined in the previous examples.

Cookies

Residential Proxies

curl -v -x pr.oxylabs.io:7777 -U "customer-USERNAME:PASSWORD" "https://example.com" -H "Cookie: NID=1234567890; 1P_JAR=0987654321"

Next-Gen Residential Proxies

curl -k -v -x ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H "Cookie: NID=1234567890; 1P_JAR=0987654321"

Cookies are sent exactly the same way with both products, apart from differences outlined in the previous examples.

POST

Residential Proxies:

 curl -X POST -v -x pr.oxylabs.io:7777 -U "customer-USERNAME:PASSWORD" "https://example.com" -d "@/path/to/file"

Next-Gen Residential Proxies

curl -X POST -k -v -x ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -d "@/path/to/file"

POST requests are sent exactly the same way with both products, apart from differences outlined in the previous examples.

What Next-Gen Proxies have and Residentials don’t

JavaScript rendering

Residential Proxies: N/A

Next-Gen Residential Proxies

curl -k -v -x ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H “X-Oxylabs-Render: html”

Unlike Residential Proxies, Next-Gen Residential Proxies can render JavaScript before passing the result back. This feature may be useful in few cases: 

a) with sites which don’t load any useful content without the use of JavaScript 

b) when  the target site’s bot detection tool uses JavaScript rendering as a way to determine whether the site is being accessed by a bot

Custom Success Status Codes

Residential Proxies: N/A

Next-Gen Residential Proxies

curl -k -v -x ngrp.oxylabs.io:60000 -U USERNAME:PASSWORD "https://example.com" -H "X-Oxylabs-Successful-Status-Codes: 500,501,502,503"

Next-Gen Residential Proxies let the user specify custom success status codes. In the example above,  the query specifies that a response from the target website with a response code of 500, 501, 502 or 503 should be passed back to the user and is considered successful. This feature is exclusive to Next-Gen Residential Proxies and does not have an equivalent in Residential Proxies.

Conclusion 

Looking from a business perspective, upgrading to Next-Gen Residential Proxies will help you save up on proxy maintenance and development. Handling captchas, keeping up with website updates, and creating JavaScript rendering will cost you time, and accordingly – money.

Human resources, constant proxy management, and other technical maintenance will come out to be costly. Therefore, the shift from Residential Proxies is an easy one, the benefits clearly beneficial.

If you have any further questions regarding Next-Gen Residential Proxies, please contact your account manager or book a call with our team by clicking the “Contact sales” button!

avatar

About Gabija Fatenaite

Gabija Fatenaite is a Senior Content Manager at Oxylabs. Having grown up on video games and the internet, she grew to find the tech side of things more and more interesting over the years. So if you ever find yourself wanting to learn more about proxies (or video games), feel free to contact her - she’ll be more than happy to answer you.

Related articles

How to Detect Bot Traffic?

How to Detect Bot Traffic?

Jan 08, 2021

6 min read

What Is a Bot and How Does It Work?

What Is a Bot and How Does It Work?

Nov 23, 2020

8 min read

What is MAP Monitoring?

What is MAP Monitoring?

Nov 11, 2020

5 min read

All information on Oxylabs Blog is provided on an "as is" basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Oxylabs Blog or any third-party websites that may be linked therein. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website's terms of service or receive a scraping license.