Develop and maintain automated web scraping tools using Python, BeautifulSoup, Scrapy, Selenium, or similar frameworks.
Extract structured and unstructured data from public websites and APIs.
Clean, transform, and store raw data into usable formats (e.g., CSV, JSON, SQL databases).
Monitor scrapers for performance issues and make updates when site structures change.
Implement measures to avoid IP bans, such as proxy rotation and user-agent spoofing.
Ensure data accuracy, consistency, and completeness.
Collaborate with data analysts, engineers, and product teams to define data requirements.
Document scraping processes and maintain code repositories (e.g., GitHub, GitLab).
Preferred Skills:
Experience with cloud services (AWS, GCP, Azure) and serverless scraping.
Familiarity with data pipeline tools (e.g., Airflow, Luigi).
Knowledge of regular expressions, data normalization, and ETL workflows.
Ability to manage proxies, headless browsers, and CAPTCHA bypass tools.
Strong debugging and documentation skills.
Job Type: Full-time
Pay: ?15,000.00 - ?35,000.00 per month
Work Location: In person
Beware of fraud agents! do not pay money to get a job
MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.