Job Description

Role Overview

We are looking for an experienced

Data Engineer

with strong expertise in

Azure Databricks, Azure Data Factory (ADF), and PySpark

. The ideal candidate should have hands-on experience in building scalable data pipelines, integrating diverse data sources, and working on end-to-end data engineering solutions in the Azure ecosystem. This role involves working with structured, semi-structured, and unstructured data sources including

SAP, SharePoint, JSON, XML, APIs, and files from Azure Blob Storage

.

Key Responsibilities

Design, develop, and maintain

data pipelines

using

Azure Databricks, ADF, and PySpark

.


Ingest, transform, and process data from multiple sources including

SAP, SharePoint, APIs, JSON, XML, and unstructured files

.


Implement efficient data ingestion from

Azure Blob Storage

and other Azure-native services.


Collaborate with stakeholders to understand business requirements and translate them into scalable technical solutions.


Optimize performance of

PySpark scripts

and Databricks workflows for large-scale data processing.


Ensure

data quality, governance, and security compliance

throughout the pipeline lifecycle.


Support integration of data into downstream systems for analytics and reporting.


Troubleshoot data pipeline issues and provide timely resolutions.

Required Skills & Qualifications

3-7 years

of hands-on experience in

Data Engineering

.


Strong expertise in:

Azure Databricks

(PySpark, Delta Lake, notebooks, cluster management).

Azure Data Factory

(pipelines, triggers, linked services, monitoring).

Azure Blob Storage

(data ingestion, partitioning, schema management).


Proficiency in

PySpark

for ETL/ELT development and large-scale data transformations.


Experience working with

SAP, SharePoint, APIs, JSON, XML, and unstructured files

as data sources.


Strong SQL skills for querying and transformation.


Familiarity with

data lake / data warehouse concepts

.


Understanding of

CI/CD practices

and version control (Git/Azure DevOps).


Excellent problem-solving and communication skills.

Good to Have (Optional)

Exposure to

Power BI or Synapse Analytics

.


Knowledge of

data security, access management, and compliance

in Azure.


Experience with

real-time/streaming ingestion (Kafka/Event Hub)

.

Why Join Us?

Opportunity to work on

cutting-edge Azure data projects

.


Exposure to

diverse enterprise data sources

including SAP and SharePoint.


Career growth towards

solution architect / lead roles

.


Collaborative and innovative work culture.