Job Description

JD:

EXP:10 & Above

Manage and optimize Databricks workspaces, clusters, jobs, pools, and notebooks. Configure cluster policies, pool autoscaling, runtime versions, and job scheduling. Monitor cluster performance, costs, autoscaling, and job execution. Implement and manage CI/CD integration for Databricks (Repos, GitHub/Azure DevOps integration). Oversee Unity Catalog administration: catalogs, schemas, permissions, external locations. Troubleshoot issues related to jobs, clusters, libraries, and notebook execution. Manage libraries, secret scopes, environment configurations, and workspace permissions. Implement fine-grained access controls (ACLs), delta lake permissions, and data governance policies. Implement data access controls following enterprise security and compliance frameworks. Manage ADF pipelines, data flows, linked services, integration runtimes (IR), and triggers. Monitor pipeline execution, performance, and operational health. Manage and optimize Self-Hosted Integration Runtime (SHIR) and Azure IR. Handle pipeline deployments and promote ADF artifacts across environments (Dev/Test/Prod). Troubleshot pipeline failures, connectivity issues, and performance bottlenecks. Configure ADF security: RBAC, managed identities, and key vault integrations. Ensure high availability, SLAs, and operational reliability of ADF workflows.
Improve ADF pipeline performance through caching, IR optimization, and parallelization.

Location

Bengaluru

Job Function

TECHNOLOGY

Role

Developer

Job Id

391002

Desired Skills

Azure | DevOps
Desired Candidate Profile

Qualifications

: BACHELOR OF ENGINEERING