Job Description

Project Role :

Data Engineer

Project Role Description :

Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.

Must have skills :

Microsoft Azure Data Services

Good to have skills :

Microsoft Azure Databricks

Minimum

3

year(s) of experience is required

Educational Qualification :

15 years full time education



Summary: As a Data Engineer, you will be responsible for designing, developing, and maintaining data solutions for data generation, collection, and processing. Your role involves creating data pipelines, ensuring data quality, and implementing ETL processes to migrate and deploy data across systems. Roles & Responsibilities: o Data Integration: Develop and implement data pipelines using Azure Data Factory, Azure Fabric, and MDMF(Meta data Management framework) to ingest, transform, and store data from various sources. o Data Modeling: Maintain data models, ensuring data quality and consistency across different databases and systems. o Database Management: Manage Azure SQL Databases and other storage solutions to optimize performance and scalability. o ETL Processes: Design and optimize ETL (Extract, Transform, Load) processes to ensure efficient data flow and availability for analytics and reporting. o Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide actionable insights. o Documentation: Maintain clear documentation of data architectures, data flows, and pipeline processes. Professional & Technical Skills: o Proficiency in Azure services such as Azure Data Factory,Azure Fabric, MDMF Azure SQL Database. o Good knowledge of SQL and experience with programming languages such as Python and Pyspark. o Familiarity with data modeling techniques and data warehousing concepts. o Experience with cloud architecture and data architecture best practices. o Understanding of data governance and security principles. o Excellent problem-solving skills and attention to detail. o Strong communication skills for collaborating with technical and non-technical stakeholders. Additional Information: - The candidate should have a minimum of 3 years of experience in Microsoft Azure Data Services. - This position is based at our Bengaluru office. - A 15 years full-time education is required.




15 years full time education