Job Description

:

Job description - Data Engineering (DE) Lead

Job Location: Gurgaon/Bangalore

Experience: 10+ years

Role:

Data Engineering Lead will be responsible for designing a unified data layer and reporting solution as a DE SME and guide the offshore team at various junctures to ensure timely delivery at best quality. The job holder will be working with technical and business stakeholders at different levels in understanding requirements and building end-to-end data warehousing solutions. Incumbent will be working in an onshore-offshore model to deliver agile, scalable, and efficient data solutions.

Key responsibilities:

• Lead a team of data engineers, analysts & BI developers as a DE SME to design and build efficient & scalable data warehouses for reporting purposes, compliant with best practices like MDM, RDM etc.
• Support the team as DE SME to apply the best approaches for large scale data movement, capture data changes, apply incremental data load strategies, exception handling and support development of reusable ETL modules.
• Gather available details & documentation on existing data feeds by collaborating with business and technical stakeholders; Support the team in building the feeds prioritization, standardization framework and data quality (DQ) plan.
• Oversee and contribute to the creation and maintenance of relevant data artifacts (data lineages, source to target mappings, high level designs, interface agreements, etc.).
• Work with data feeds from multiple geographies and understand their privacy & security requirements, standard contractual clauses (SCC), data sharing agreements and support the team creating amendments as per best practices.
• Mentor the team wherever required and verify that design best practices as well as coding and architectural guidelines, standards, and frameworks are adhered to by offering guidance, communicating risk, and addressing roadblocks as they arise.
• Serves as the primary interface between senior client management and EXL senior leadership (VPs and SVPs)
• Provides thought leadership and delivers business insights to identify and resolve complex issues critical to clients\xe2\x80\x99 success
• Manages communication between senior EXL partners and clients to update project progress & solicit feedback on project deliverables

• 10+ years IT experience in software development for data systems at scale (building data warehouse, data marts, data lakes, ETL, analytic solutions, etc.)
• 5+ years of hands-on experience in data warehouse architecture design and modelling to integrate data from diverse sources.
• At least 5 years of hands-on experience on various ETL tools (Talend - Preferred, Glue, Informatica etc.), Data Warehousing services (Amazon Redshift - Preferred, Snowflake etc.) and other cloud services like S3/Blob, Lambda/Functions etc.
• At least 4 years of experience in leading projects end-to-end and working directly with stakeholders in navigating complexities to enable successful delivery of scalable data pipelines in cloud supporting various applications.
• At least 4 years of proven track record of leading end to end projects and globally distributed teams (onshore-offshore model)
• Strong understanding of CI/CD principles and experience of CI/CD tools like GIT, Jenkins etc.
• Experience with orchestration tools like Airflow to automate and manage complex data pipelines and workflows.
• Strong experience on SQL on a variety of platforms (such as MS SQL, Athena, Redshift) and Python. Good to have experience on PySpark to develop and execute distributed data processing pipelines.
• Experience in encryption of datasets in data warehouses and exposure to data privacy & security regulations.
• Familiarity with infrastructure as a code for cloud resource management as code (CloudFormation, Terraform, AWS CDK).
• Demonstrable superior problem solving, interpersonal and people management skills. Organized and able to effectively communicate progress throughout project lifecycles.
• Bachelor\xe2\x80\x99s or Master\xe2\x80\x99s (preferred) degree in in a quantitative or technical field such as Computer Science, Information Technology, Computer Engineering or equivalent

EXL Service