Python Developer at Oxylabs
Tadas started working as a Python Developer at Oxylabs more than two years ago. When he became a team lead of a scraping track, he focused on reworking the whole scraping infrastructure. His goal was to design and develop frameworks that allow developers to quickly and conveniently build scrapers and parsers.
Managing Python projects' dependencies might seem like an easy task nowadays. However, developing your own frameworks and libraries with many dependent projects usually requires more than 10 merge requests for even a minor change. Then, the process of managing Python projects becomes challenging.
Tadas’ presentation will take a quick look at:
A "wannabe" Python monorepo setup;
Issues when trying to manage an increasing number of scrapers and how to solve them without additional tools conveniently;
How to build a "wannabe" Python monorepo setup using Poetry and GitLab.
Coming from different industries and backgrounds but united in common passion and aspirations, these web scraping experts will share their experiences and answer your questions.
Linux Systems Engineer @ Oxylabs
Python Developer @ Oxylabs
CEO/CTO @ The DataWorks
CEO @ Dataistic
Partner @ Farella Braun + Martel
Moderator @ OxyCon
GET IN TOUCH
Certified data centers and upstream providers
Connect with us