Job Description

Job Summary:

We are looking for a talented and experienced

Azure Data Engineer

to join our data engineering team in

Bangalore

. As an Azure Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines, and working with Azure-based data services to ensure the seamless flow of data for analytical and operational purposes. The ideal candidate will have a strong understanding of

Azure data platforms

, data engineering practices, and be proficient in creating and optimizing data solutions using

Azure Data Factory

,

Azure Synapse Analytics

,

Azure SQL Database

, and other related Azure services.

Key Responsibilities:

Data Pipeline Development:

Design, develop, and maintain efficient and scalable data pipelines in

Azure Data Factory (ADF)

to integrate data from multiple sources, process data, and load it into target systems.

Data Integration:

Work with various data sources like

SQL

,

NoSQL

,

Blob Storage

, and

data lakes

to ingest and process data in an efficient and scalable manner.

Azure Data Services:

Utilize

Azure Synapse Analytics

,

Azure Data Lake

,

Azure SQL Database

,

Cosmos DB

, and other Azure services to build and optimize data architecture and storage solutions.

ETL/ELT Development:

Implement both

ETL (Extract, Transform, Load)

and

ELT (Extract, Load, Transform)

processes using Azure tools, ensuring data is processed accurately and efficiently.

Data Warehousing:

Design and maintain cloud-based

data warehousing

solutions using

Azure Synapse Analytics

(formerly Azure SQL Data Warehouse) to support reporting, analytics, and data-driven decision-making.

Performance Tuning:

Optimize performance of data pipelines and processes, ensuring they run efficiently, and troubleshoot performance issues.

Data Modeling & Management:

Develop data models for large datasets, ensuring proper schema design, partitioning, indexing, and data governance.

Automation & Monitoring:

Set up automated processes for data pipeline monitoring, error handling, and logging to ensure high data quality and reliability.

Security & Compliance:

Ensure that the data solutions are secure and comply with data privacy regulations (e.g., GDPR, HIPAA) using

Azure Security

features and best practices.

Collaboration:

Collaborate with data scientists, business analysts, and stakeholders to understand data needs and ensure alignment of data engineering solutions with business objectives.

Documentation:

Maintain comprehensive documentation for data architecture, pipelines, data models, and best practices.

Required Skills & Qualifications:

Bachelor's degree

in Computer Science, Engineering, or a related field (or equivalent experience).

5+ years

of experience in

data engineering

, with at least

3 years

of hands-on experience working with

Azure data platforms

. Strong proficiency in

Azure Data Factory (ADF)

,

Azure Synapse Analytics

,

Azure SQL Database

, and

Azure Data Lake

. Solid understanding of

ETL

and

ELT

processes and experience with building data pipelines on the cloud. Proficient in

SQL

and experience with query optimization and performance tuning. Experience with

Azure Databricks

,

HDInsight

,

Cosmos DB

, and other Azure-based data technologies is a plus. Familiarity with

Data Lake

architectures and the ability to implement solutions for large-scale data processing. Strong knowledge of

data modeling

,

data warehousing

, and

data governance

practices. Experience working with

cloud storage solutions

, including

Blob Storage

and

Data Lake Storage

. Knowledge of

Power BI

,

Azure Analysis Services

, or similar tools for data visualization and reporting is a plus. Experience with

Python

,

Spark

, or

Scala

for data processing is highly desirable. Familiarity with

DevOps

practices and tools (e.g.,

Azure DevOps

) for data engineering deployments is a bonus. Understanding of

data security

and

compliance

frameworks in cloud environments.

Preferred Skills (Nice to Have):

Azure Certification

such as

Microsoft Certified: Azure Data Engineer Associate

. Experience with

machine learning models

and

AI workflows

integrated with Azure data services. Exposure to

Big Data

technologies such as

Hadoop

,

Spark

, or

Kafka

. Familiarity with

Agile

development methodologies. Knowledge of

CI/CD pipelines

for automating deployments of data engineering solutions. Experience with

DataOps

for better management of data pipelines.
Job Type: Full-time

Pay: ?900,000.00 - ?1,200,000.00 per year

Work Location: In person