Python Developer at Oxylabs
Tadas started working as a Python Developer at Oxylabs more than two years ago. When he became a team lead of a scraping track, he focused on reworking the whole scraping infrastructure. His goal was to design and develop frameworks that allow developers to quickly and conveniently build scrapers and parsers.
Managing Python projects' dependencies might seem like an easy task nowadays. However, developing your own frameworks and libraries with many dependent projects usually requires more than 10 merge requests for even a minor change. Then, the process of managing Python projects becomes challenging.
Tadas’ presentation will take a quick look at:
A "wannabe" Python monorepo setup;
Issues when trying to manage an increasing number of scrapers and how to solve them without additional tools conveniently;
How to build a "wannabe" Python monorepo setup using Poetry and GitLab.
Coming from different industries and backgrounds but united in common passion and aspirations, these web scraping experts will share their experiences and answer your questions.
GET IN TOUCH
General:
hello@oxylabs.ioSupport:
support@oxylabs.ioCareer:
career@oxylabs.ioCertified data centers and upstream providers
Connect with us
Advanced proxy solutions
Resources
Innovation hub