Job Description

5 - 7 Years
2 Openings
Pune

Role description

Job Title: Senior Data Engineer - Azure & Databricks

Overview

We are seeking a highly skilled

Senior Data Engineer

with

5+ years of hands-on experience

in building scalable, high-performance data solutions within the

Azure ecosystem

. The ideal candidate will have strong expertise in

Databricks, Apache Spark (Java), modern data lake architectures

, and end-to-end ETL pipeline development. This role involves designing, developing, and optimizing data systems that support business-critical analytics and operational use cases.

Key Responsibilities

Design, develop, and maintain

scalable data pipelines

using Azure services and Databricks. Build efficient

data ingestion and transformation workflows

leveraging Databricks, Spark, and Delta Lake. Develop, enhance, and optimize

ETL pipelines using Java, Spark, and SQL

. Implement industry-best practices for

data modeling, data quality, governance, and pipeline performance

. Work with relational and lakehouse data platforms, including

PostgreSQL and Delta Lake

. Collaborate with cross-functional teams to ensure smooth execution, deployment, and support of production data workloads. Troubleshoot performance issues and optimize workloads for reliability and scalability.

Required Skills & Experience

5+ years

of hands-on experience in

Data Engineering

. Strong working knowledge of the

Azure ecosystem

, specifically:

+ Azure Databricks
+ Azure Data Lake Storage (ADLS)
+ Azure Functions
+ Azure Data Factory (preferred but not mandatory)
Expertise in

Apache Spark with Java (Spark Core, Spark SQL, Spark Streaming)

. Advanced

SQL

skills including performance tuning and complex query optimization. Strong understanding of

ETL/ELT methodologies

, data warehousing concepts, and lakehouse architectures. Experience with

Delta Lake features

such as schema evolution, ACID transactions, and time travel. Hands-on experience with

PostgreSQL

or similar relational databases. Excellent analytical, problem-solving, and debugging skills. Strong communication skills with the ability to work with technical and non-technical stakeholders. Self-motivated, proactive, and comfortable working in

fast-paced environments

.

Preferred Skills

Experience with

CI/CD tools

such as Azure DevOps and Git. Knowledge of

Infrastructure-as-Code (IaC)

tools like Terraform. Exposure to

streaming technologies

such as Kafka or Azure Event Hub. Familiarity with

data governance tools

such as Azure Purview.

Skills

Azure Cloud,Java,Pyspark

About UST

UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world's best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients' organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact--touching billions of lives in the process.