Job Description

:

- Senior Data Engineer (Cognitive Automation)

The Team: As a member of the Data Transformation - Cognitive Automation team you will work on information retrieval and data sourcing platforms powered by machine learning models for S&P Global Market Intelligence and our clients. You will learn how to develop and maintain production-ready data automation and web extraction pipelines while leading-by-example in a highly engaging work environment. You will work in a (truly) global team and encouraged for thoughtful risk-taking and self-initiative.

What\'s in it for you:

Be a part of a global company and build solutions at enterprise scale Work with a highly skilled and hands-on technical team (including leadership) Contribute to solving high complexity, high impact problems Build and maintain production ready pipelines from ideation to deployment

Responsibilities:

• Develop, Deploy and Maintain data automation & web scraping pipelines in production
• Play a central role in critical stages of the data automation project life cycle, including:

• End-to-end delivery of complex data engineering projects
• Providing technical leadership development on cloud environments including scaling and replicating existing pipelines
• Writing and tuning complex data extraction pipelines, workflows, and data structures as evolving product features
• Leading in data infrastructure, deployment, testing activities with hands on engineering and automation experience including CI/CD
• Strengthening data quality and reliability of the web extraction and data automation pipelines
• Progressing standards and best practices for the platform and operational excellence
• Stakeholder management and documentation for reproducibility

Technical Requirements:

• Expert proficiency in Python, specifically web scraping and API development
• Experience with technologies such as Airflow, Redis, Celery, Selenium, MongoDB, Flask/Django/FastAPI, Docker
• Good knowledge and exposure to AWS environments including EKS
• Some exposure data extraction / data automation pipelines at scale
• Open to learning new technologies and programming languages as required

Good to have:

• A Master\'s degree (minimum Bachelor\'s degree) from a recognized institute
• Around 2-6 years of relevant experience in Data engineering domain
• Some prior work to show on Github, Kaggle, StackOverflow etc.

Location: Gurgaon, Hyderabad, Mumbai, Bangalore (India)

Grade: 9



Equal Opportunity Employer

S&P Global is an equal opportunity employer and all qualified candidates will receive consideration for employment without regard to race/ethnicity, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, marital status, military veteran status, unemployment status, or any other status protected by law. Only electronic job submissions will be considered for employment.

If you need an accommodation during the application process due to a disability, please send an email to: and your request will be forwarded to the appropriate person.

US Candidates Only: The EEO is the Law Poster describes discrimination protections under federal law.

DTMGOP202.1 - Middle Professional Tier I (EEO Job Group)

Job ID: 286738
Posted On: 2023-05-09
Location: Gurgaon, Haryana, India

S&P Global