Associate Manager Data Engineering

Year    Bangalore, Karnataka, India

Job Description

:
The AWS Databricks Platform administrator is a technical role to design, implement and maintain processes used to manage the organization's Databricks platform. The administrator will be responsible for facilitating the work of data analysts, data engineers, data scientists and data governance/stewards while maintaining best practices for security and compliance. Unity catalog will be used to implement access and identity policies. Infrastructure will be maintained using Terraform. The administrator will be responsible for assisting with compute issues, monitoring and alerting, and assuring the platform functions smoothly. This role requires great organization and communication skills.
Job responsibilities:

  • Manage and maintain role-based access to data and features in the Databricks Platform and using Unity Catalog as applicable.
  • Implement external access controls for outside teams using service principles, SQL warehouses and Delta Sharing
  • Work with platform users to solve problems associated with the Databricks platform, job deployments, job failures, and facilitate work, improve processes and systems used to manage infrastructure, users and external access
  • Create infrastructure for AWS and Databricks using Terraform: S3, IAM roles, Instance Profiles, KMS Keys, Secrets, VPCs, Athena, CloudTrail and other required resources.
  • Understand and implement best practices for security and compliance. Create documentation for users and admins
  • Understand how to utilize Databricks APIs to automate administrative tasks
  • Create queries and dashboards to monitor critical systems and processes, use Databricks system tables to monitor optimal usage of resources.
  • Manage tags and naming convention enforcement for all resources as applicable in the environment.
  • Manage open-source spark cluster (open-source version of Cloudera).
  • Manage integrations with other tools in the data lake such as Denodo, Collibra, Redshift, Kafka, Protegrity, etc.
  • Comply with all agreed SLA requirements for all service requests, incidents and changes.
Required qualifications:
  • 2 or more years of experience administering the Databricks platform (on AWS preferred).
  • Strong understanding of data engineering needs and a willingness to take up new integrations, requirements and challenges.
  • Experience of designing and creating AWS infrastructure with Terraform, via GitHub Actions or similar Devops tools.
  • Strong understanding of AWS infrastructure required by Databricks
  • Experience with compliance audits/audit documentation
  • Experience using Databricks APIs with tools like Postman
  • Understanding of the Databricks workspace and development environment
  • Strong SQL skills for creating admin related queries and dashboards
  • Strong debugging skills and ability to do risk analysis for 3rd party integrations with AWS resources.
  • Knowledge of Python a plus
At DXC Technology, we believe strong connections and community are key to our success. Our work model prioritizes in-person collaboration while offering flexibility to support wellbeing, productivity, individual work styles, and life circumstances. We're committed to fostering an inclusive environment where everyone can thrive.
Recruitment fraud is a scheme in which fictitious job opportunities are offered to job seekers typically through online services, such as false websites, or through unsolicited emails claiming to be from the company. These emails may request recipients to provide personal information or to make payments as part of their illegitimate recruiting process. DXC does not make offers of employment via social media networks and DXC never asks for any money or payments from applicants at any point in the recruitment process, nor ask a job seeker to purchase IT or other equipment on our behalf. More information on employment scams is available .

Beware of fraud agents! do not pay money to get a job

MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.


Job Detail

  • Job Id
    JD4003501
  • Industry
    Not mentioned
  • Total Positions
    1
  • Job Type:
    Full Time
  • Salary:
    Not mentioned
  • Employment Status
    Permanent
  • Job Location
    Bangalore, Karnataka, India
  • Education
    Not mentioned
  • Experience
    Year