Job Description

Role:Databricks + Pyspark Developer

Experience: 5 to 9 years

AIA-Chennai

Job Summary

We are seeking a highly skilled Sr. Developer with 7 to 10 years of experience in Databricks and Retail Banking. The ideal candidate will have expertise in Databricks Workspace Admin Databricks CLI Databricks Unity Catalog Databricks SQL Databricks Workflows and PySpark. This hybrid role requires a proactive individual who can contribute to innovative solutions in the Retail Banking domain enhancing our data-driven strategies.

Responsibilities

Develop and maintain scalable data pipelines using Databricks and PySpark to support business objectives in the Retail Banking sector. Collaborate with cross-functional teams to design and implement data solutions that enhance customer experience and operational efficiency. Administer Databricks Workspace ensuring optimal performance and security compliance. Utilize Databricks CLI to automate and streamline data workflows improving overall productivity. Implement and manage Databricks Unity Catalog to ensure data governance and accessibility across the organization. Design and execute complex SQL queries in Databricks SQL to extract valuable insights from large datasets. Oversee the scheduling and execution of data workflows using Databricks Workflows ensuring timely delivery of data products. Provide technical expertise and guidance to junior developers fostering a culture of continuous learning and improvement. Collaborate with stakeholders to understand business requirements and translate them into technical specifications. Monitor and troubleshoot data pipelines resolving issues promptly to minimize downtime. Contribute to the development of best practices and standards for data engineering within the organization. Stay updated with the latest trends and advancements in Databricks and data engineering to drive innovation. Ensure all data solutions align with company goals and regulatory requirements enhancing the companys impact on society. Qualifications Possess strong experience in Databricks Workspace Admin Databricks CLI and Databricks Unity Catalog. Demonstrate proficiency in Databricks SQL and Databricks Workflows for efficient data management. Exhibit expertise in PySpark for developing robust data processing solutions. Have a deep understanding of the Retail Banking domain to deliver relevant data insights. Show ability to work effectively in a hybrid work model balancing remote and in-office tasks. Display excellent problem-solving skills and attention to detail. Hold a bachelors degree in Computer Science Information Technology or a related field.

Certifications Required

Databricks Certified Data Engineer Associate