Job Description

Qualification:

Bachelor's or master's degree in computer science, Information Technology, Engineering, or a related field

Experience:

5 to 10 Years

Job Post Date:

13/11/2025

Job Expiry Date:

13/12/2025

Location:

Pune, Chennai

:

Design & Architecture: Design high-performance, resilient, and scalable data architectures for data ingestion and processing using best practices for Azure Databricks and Spark. Team Leadership: Provide technical guidance and mentorship to a team of data engineers, fostering a culture of collaboration, continuous learning, and innovation. Collaboration: Work closely with data scientists, business analysts, and other stakeholders to understand data requirements and ensure smooth integration of various data sources into the data lake/warehouse. Optimization & Performance Tuning: Ensure data pipelines are optimized for speed, reliability, and cost efficiency in an Azure environment. Conduct performance tuning, troubleshooting, and debugging of Spark jobs and Databricks clusters. Code Quality & Best Practices: Enforce and advocate for best practices in coding standards, version control, testing, and documentation. Integration with Azure Services: Work with other Azure services such as Azure Data Lake Storage, Azure SQL Data Warehouse, Azure Synapse Analytics, and Azure Blob Storage to integrate data seamlessly. Continuous Improvement: Stay current with industry trends and emerging technologies in the field of data engineering and make recommendations for improvements to the team's tools and processes. Ensure Data Quality: Implement data validation and data quality checks as part of the data ingestion process to ensure consistency, accuracy, and integrity of ingested data. * Risk & Issue Management: Proactively identify risks and blockers and resolve complex technical issues in a timely and effective manner.