Job Description

We are seeking an experienced

Python Developer

with a strong background in

web scraping

to join our growing data engineering team. You will work on building robust, scalable data extraction pipelines using modern scraping frameworks and automation tools.

Key Responsibilities:

Develop and maintain efficient web scraping spiders using

Python

and

Scrapy

Work with

AI Spider Templates

,

Page Object Patterns

, and

Spidermon

for maintainable scraping architecture Extract structured and unstructured data via

public APIs

and

dynamic web content

Handle challenges involving

JavaScript-rendered content

,

HTTP/web protocols

, and

HTML/CSS structure parsing

Implement

IP rotation

and

proxy management

strategies to avoid blocking Clean, normalize, and store extracted data for downstream processing Automate scraping workflows using

Scrapy Cloud

or similar platforms Collaborate with data scientists, analysts, and engineers to deliver high-quality datasets

Required Skills & Qualifications:

Minimum

2 years of hands-on experience in web scraping using Python

Solid understanding of

Scrapy

,

Requests

,

BeautifulSoup

, and/or

Selenium

Familiarity with

dynamic content rendering

,

JavaScript troubleshooting

, and browser automation Working knowledge of

data cleaning

,

workflow automation

, and

cloud scraping environments

Experience with

API integration

and

JSON/XML data parsing

Proficiency in

HTML/CSS

,

HTTP protocols

, and debugging tools Knowledge of

proxy services

,

IP rotation tools

, and

anti-bot handling techniques

Familiar with version control systems (e.g.,

Git

)

Preferred Qualifications:

Experience with

Spidermon

,

Scrapy Cloud

, or similar orchestration tools Exposure to

data storage solutions

like PostgreSQL, MongoDB, or cloud databases Basic understanding of

AI-based scraping templates

or machine learning techniques in data extraction

Benefits:

Competitive salary Flexible working hours Opportunity to work on cutting-edge data projects Learning & development budget Remote working options
Job Type: Full-time

Pay: ?25,000.00 - ?50,000.00 per month

Benefits:

Work from home
Schedule:

Day shift Monday to Friday
Experience:

Python: 2 years (Required)
Language:

English (Required)
Work Location: Remote

Expected Start Date: 30/07/2025