Job Description

Project Overview:
We are seeking skilled Data Engineers with strong hands-on experience in Databricks for a high-impact data platform project. The project involves building scalable data pipelines, integrating with cloud platforms, and managing data governance using Unity Catalog.
Key Requirements:

• Primary Skills:
• Expertise in Databricks, working on either AWS or Azure (cloud-specific skills are not mandatory unless specified).
• Proficiency in PySpark, Spark SQL, Python, and Delta Lake.
• Hands-on experience in building data pipelines and cloud integration.
• Familiarity with Databricks Unity Catalog and Federated Catalog.

• Cloud Environment:
• Engineers working on Databricks on AWS are not required to have Azure-specific skills.
• Engineers working on Databricks on Azure are not required to have AWS-specific skills.
• Multi-cloud exposure (AWS & Azure) is a plus due to the diverse project environment.

• Infrastructure as Code (IaC):
• Experience with Terraform for infrastructure provisioning and automation.

• Testing and Quality:
• Experience in writing unit tests and integration tests using pytest.