Job Description

Position Title: Web Scraping Specialist
Experience Required: 2+ Years
Location - Hyderabad
Employment Type: Full-Time
Roles and Responsibilities

• Develop and maintain robust web scraping scripts and pipelines using Python and frameworks like Scrapy, Selenium, and Requests.
• Analyze website structures and implement scraping logic using XPath, CSS selectors, and regular expressions.
• Handle complex scenarios including dynamic content, login-based scraping, and session management.
• Implement user-agent rotation, proxy management, and rate-limiting strategies to ensure uninterrupted and ethical scraping.
• Integrate scraping outputs with PostgreSQL databases and write optimized SQL queries for data retrieval and reporting.
• Ensure data accuracy and integrity through data validation, cleansing, and normalization processes.
• Collaborate with cross-functional teams to understand data requirements and deliver actionable datasets.
• Keep up to date with the latest web scraping trends, anti-scraping mechanisms, and best practices.

• Strong command over Python with experience in Scrapy, Selenium, and Requests.
• In-depth knowledge of HTML, CSS, XPath, and DOM parsing.
• Experience in managing dynamic content, solving CAPTCHAs, and using proxy services.
• Good understanding of SQL, especially in PostgreSQL.
• Familiarity with data cleaning and data quality assurance practices.
• Understanding of ethical and legal aspects of web scraping.