Tune in to the hottest topics on public data gathering
Hear our personal experiences on web scraping
Learn about the best web scraping practices
One of Oxylabs’ core beliefs is knowledge sharing. Listen to our podcast and learn more about all things web scraping. Find out why web scraping exists, discover solutions to daily challenges, and uncover what’s new in the web scraping community.
Episode 4
In the 4th episode of OxyCast, our host, Augustinas Kalvis, along with the special guest who has been at Oxylabs since the very beginning, Eivydas Vilčinskas (Tech Team Lead), discuss the ever-complex yet always relevant topic of scaling. From the explanation of what scaling is and why it’s essential to its unique intricacies, difficulties, and viable approaches, this episode looks at all of it.
In this episode, we answer the following questions:
How do you go from 10 requests per second to tens of thousands?
How do you ensure various bottlenecks are solved, and where might they appear in the first place?
Infrastructure and application bottlenecks - when do you encounter them, and what are some of the solutions you could use?
Vertical vs. Horizontal scaling, what are their differences, and when to use which?
Load balancers, how do they help?
February 2, 2022
An introduction to the web scraping world
The purpose of web scraping
The process & challenges of public data gathering
Buying vs. building a web scraper
February 23, 2022
The most common challenges of web scraping
Ensuring reliability & scalability
Handling bot detection
Tips and tricks on avoiding blocks
The main principles of data parsing
Exploring the process step-by-step
Writing a good selector
Making the parser scalable
Boosting the number of requests
Solutions to bottlenecks
Tips on system design
Share your input
We are always looking for new thematic angles and people to share them with you. Do you have anything particular in mind? Then go ahead and suggest a topic or refer a guest. Let us know to make it a reality by sending an email!
Contact us: events@oxylabs.io
GET IN TOUCH
General:
hello@oxylabs.ioSupport:
support@oxylabs.ioCareer:
career@oxylabs.ioCertified data centers and upstream providers
Connect with us
Resources
Innovation hub