Job Description

Skill: Pyspark + Databricks

Experience

: 6 to 9 Years

Location : AIA - Gurgaon

Job Summary

We are seeking a highly skilled Technical Lead with 12 to 16 years of experience to join our dynamic team. The ideal candidate will have expertise in Databricks Workflows Unity Catalog and Databricks with a preference for experience in Research and Development. This hybrid role offers the opportunity to work on cutting-edge projects during day shifts with no travel required.

Responsibilities

Lead the design and implementation of scalable data solutions using Databricks Workflows to enhance data processing capabilities. Oversee the integration of Databricks Unity Catalog to ensure seamless data governance and security across platforms. Provide technical expertise in Databricks to optimize performance and drive innovation in data analytics. Collaborate with cross-functional teams to deliver high-quality solutions that meet business requirements and objectives. Develop and maintain comprehensive documentation for data workflows and processes to facilitate knowledge sharing. Mentor and guide junior team members to foster a culture of continuous learning and improvement. Conduct regular code reviews to ensure adherence to best practices and coding standards. Analyze complex data sets to identify trends and insights that support strategic decision-making. Implement robust testing frameworks to ensure the reliability and accuracy of data solutions. Coordinate with stakeholders to gather requirements and translate them into technical specifications. Drive the adoption of new technologies and methodologies to enhance the teams capabilities and efficiency. Ensure compliance with industry standards and regulations to protect data integrity and privacy. Monitor system performance and troubleshoot issues to maintain optimal operation of data platforms.

Qualifications

Possess extensive experience in Databricks Workflows and Unity Catalog demonstrating a deep understanding of their functionalities. Have a strong background in data engineering and analytics with a proven track record of delivering successful projects. Exhibit excellent problem-solving skills and the ability to work effectively in a hybrid work model. Demonstrate proficiency in programming languages such as Python or Scala relevant to Databricks environments. * Show a keen interest in Research and Development with the ability to apply domain knowledge to data solutions.