Glen De Cauwsemaecker

Lead Crawler Engineer at OTA Insight

Glen De Cauwsemaecker is the Lead Crawler Engineer at OTA Insight (around 350 employees, of which 67 engineers). With over ten years of experience, he drives the technical roadmap and innovation with a team of four other engineers. Besides researching, developing, and maintaining crawlers, he also maintains and creates new projects related to proxy/networking services, backend services, libraries, automated browser solutions, fingerprinting, tooling, and a trivia of cross-platform and language agnostic reverse engineering activities. 

Glen De Cauwsemaecker

Learn more about Glen's presentation:

How to Continuously Yield High Quality Data While Growing From 100 to 100M Daily Requests

The company where Glen works, OTA Insight, gathers public data from hundreds of different kinds of data sources, with over 100M requests daily, amounting to approximately 9000 GiB (compressed) of raw data. This presentation will cover the story of how OTA Insight grew from 100 daily requests to 100,000,000 requests while continuously yielding high-quality data every single day.

Glen will cover a variety of topics, including: 

  • The beginning of the OTA Insight company and how they scaled up their business, as well as what they're working on today to prepare for tomorrow;

  • Company's infrastructure and how it's involved to help them with the growth;

  • What technology do they use, how did it evolve, as well as what choices did they make at every step;

  • Where they see themselves as a crawler team, what challenges they foresee for the future, and how they aim to stay on top of the game.

Meet OxyCon 2022 speakers

Coming from different industries and backgrounds but united in common passion and aspirations, these web scraping experts will share their experiences and answer your questions. 

Martynas Saulius

Python Developer @ Oxylabs

More details

Denas Grybauskas

Head of Legal @ Oxylabs

More details

Tadas Malinauskas

Python Developer @ Oxylabs

More details

Allen O'Neill

CEO/CTO @ The DataWorks

More details

Ondra Urban

COO @ Apify

More details

Alex Reese

Partner @ Farella Braun + Martel

More details

Gabija Fatėnaitė

Moderator @ OxyCon

More details

Vaidotas Šedys

Moderator @ OxyCon

More details

Eivydas Vilčinskas

Technical Team Lead @ Oxylabs

More details

Ovidijus Balkauskas

Linux Systems Engineer @ Oxylabs

More details

Sanaea Daruwalla

General Counsel @ Zyte

More details

Julius Zaleskis

CEO @ Dataistic

More details

Paul Morgan

Data Collections Team Lead @ Datasembly

More details

Karsten Madsen

CEO @ Morningscore

More details

Sarah McKenna

CEO @ Sequentum

More details

Martynas Saulius

Python Developer @ Oxylabs

More details

Denas Grybauskas

Head of Legal @ Oxylabs

More details

Tadas Malinauskas

Python Developer @ Oxylabs

More details

Allen O'Neill

CEO/CTO @ The DataWorks

More details

Ondra Urban

COO @ Apify

More details

Alex Reese

Partner @ Farella Braun + Martel

More details

Gabija Fatėnaitė

Moderator @ OxyCon

More details

Vaidotas Šedys

Moderator @ OxyCon

More details