Job Description

:

The Production Support Data Lead will be an experienced professional responsible for providing expert-level support and incident resolution for mission-critical data pipelines and solutions. This role requires a robust background in data engineering principles, with

extensive hands-on experience in Databricks, PySpark, Azure Data Factory (ADF), and other integral Azure data services

. You will play a pivotal role in ensuring the continuous and reliable operation of data ecosystem, addressing complex data incidents, and driving optimizations for efficiency and data integrity.

Key Responsibilities:

Serve as the Subject Matter Expert (SME) for data engineering solutions within a high-stakes

production support context

. Provide expert-level troubleshooting, diagnosis, and resolution for complex incidents affecting data pipelines built on Azure Databricks, PySpark, SQL, Azure Data Factory (ADF) and other related Azure data services (e.g., Azure Data Lake Storage, Azure Synapse Analytics, Azure SQL Database). Conduct thorough root cause analysis (RCA) for recurring data issues, implementing sustainable long-term solutions to minimize downtime and prevent data discrepancies. Proactively monitor, optimize, and enhance existing data pipelines for performance, scalability, and reliability in a production environment. Collaborate closely with data engineering, development, and business teams to gain a deep understanding of data flows, system interdependencies, and critical business requirements. Develop and maintain comprehensive operational documentation, including detailed runbooks, standard operating procedures, and incident playbooks to streamline support activities. Participate in post-incident reviews, identifying areas for process improvement, automation opportunities, and preventative measures. Contribute to the continuous improvement of production support processes, tooling, and automation capabilities. Actively participate in an rotational support, including weekends and holidays, to ensure support critical business needs.

Required Qualifications:

12-14 years

of direct prior

Production support

experience in a highly complex and critical business environment, managing live data systems. Profound expertise in data engineering concepts, methodologies, and best practices. Exceptional hands-on proficiency with Azure Databricks, PySpark, and Azure Data Factory (ADF). Strong experience with other Azure data services such as Azure Data Lake Storage (Gen2), Azure Synapse Analytics, and Azure SQL Database. Demonstrated ability to troubleshoot and resolve complex data pipeline issues, including data quality anomalies, performance bottlenecks, data ingestion failures, and integration challenges. Solid understanding of SQL, data warehousing concepts, and ETL/ELT processes. Experience with Scheduling tool (Tidal) monitoring tools (e.g., Azure Monitor, Databricks monitoring tools) and incident management systems (e.g., ServiceNow, Jira Service Desk). Excellent analytical, problem-solving, and communication skills, with the ability to articulate complex technical
Job Type: Full-time

Pay: ₹1,556,402.66 - ₹3,071,601.46 per year

Benefits:

Cell phone reimbursement Health insurance Leave encashment Life insurance Paid sick time Provident Fund
Work Location: In person