Job Description

Project Overview:
We are looking for a Cloud Data Engineer with hands-on experience in Databricks platform management, cloud networking, and automation. The ideal candidate will be well-versed in managing enterprise-scale Databricks environments across AWS and Azure, with strong skills in infrastructure provisioning, access control, and CI/CD pipeline development.
Key Responsibilities & Requirements:

• Databricks Platform & Security:
• Experience with Databricks Unity Catalog, RBAC (Role-Based Access Control), and Databricks APIs.
• Knowledge of Databricks Service Principals and Azure AD group integration.
• Strong understanding of Network Connectivity Configuration (NCC) across AWS & Azure.
• Cloud Platforms:
• Working knowledge of both AWS and Azure networking and services.
• Programming & Scripting:
• Proficient in Python and PowerShell scripting for automation and orchestration.
• Infrastructure as Code (IaC):
• Hands-on experience with Terraform for deploying cloud infrastructure.
• CI/CD & DevOps:
• Experience in creating and managing Azure DevOps pipelines for deployment and automation.
• Testing & Validation:
• Capable of writing robust unit and integration tests using pytest.