Data Engineer(data Bricks)

Year    Bangalore, Karnataka, India

Job Description

Title: Data Engineer(Data Bricks)
Location: Bangalore
Exp: 4+ Yrs
Job Descrition:
Key Responsibilities:

  • Design and develop ETL/data pipelines using Databricks and Apache Spark.
  • Optimize and manage Spark-based workloads for scalability and performance.
  • Work with structured and unstructured data from multiple sources.
  • Implement Delta Lake for data reliability and ACID transactions and Unity Catalog for centralized governance and cataloging service in Databricks.
  • Develop and maintain SQL-based transformations, queries, and performance tuning.
  • Collaborate with data engineers, analysts, and business teams to meet data requirements.
  • Implement job orchestration using Airflow, Databricks Workflows, or other scheduling tools.
  • Ensure data security, governance, and compliance best practices.
  • Monitor, debug, and resolve performance bottlenecks in Databricks jobs.
  • Work with cloud storage solutions (AWS S3).
Required Skills & Experience:
  • Strong experience in Databricks and AWS.
  • Hands-on experience with Python, PySpark, Scala, SQL.
  • Experience with Spark performance tuning and optimization.
  • Knowledge of Delta Lake, Lakehouse Architecture, and Medallion Architecture.
  • Familiarity with orchestration tools (Airflow, Databricks Workflows).
  • Hands-on experience with data modeling and transformation techniques.
  • Experience with cloud platform AWS.
  • Proficiency in CI/CD for Databricks using Git, DevOps tools.
  • Strong understanding of data security, governance, and access control in Databricks
  • Good knowledge of APIs, REST services, and integrating Databricks with external systems.
Preferred Qualifications:
  • Databricks Certification (Databricks Certified Associate/Professional).
  • Knowledge of BI tools like Power BI, Looker, ThoughtSpot.
  • Bachelor's or Master's degree in Computer Science, Information Systems, Engineering or equivalent.
  • 4 to 6 years of experience

Beware of fraud agents! do not pay money to get a job

MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.


Job Detail

  • Job Id
    JD4107328
  • Industry
    Not mentioned
  • Total Positions
    1
  • Job Type:
    Full Time
  • Salary:
    Not mentioned
  • Employment Status
    Permanent
  • Job Location
    Bangalore, Karnataka, India
  • Education
    Not mentioned
  • Experience
    Year