Hands-on with Microsoft Fabric: Data Factory Gen2, Lakehouse, Notebooks, Power BI integration
Azure Databricks + Delta Lake expertise
Medallion Architecture implementation experience
CI/CD in Azure DevOps, Git, and Purview for governance
Certifications: Microsoft DP-203, DP-600, Databricks DE Associate
Required Skills & Experience:
Skill Area Description
Microsoft Fabric Deep hands-on with Data Factory (Pipelines), Dataflows Gen2, Lakehouse, Notebooks, and Power BI Integration.
Databricks Expert in Azure Databricks, Delta Lake, PySpark, Notebooks, and ML integration (optional).
Data Engineering Concepts Solid knowledge of data modeling, orchestration, data warehousing, data lakes, CDC, partitioning, and schema evolution.
Cloud & Storage Azure Data Lake Storage Gen2, Azure SQL, Azure Synapse, OneLake, and blob storage.
Job Summary:
We are seeking a highly skilled Senior Data Engineer to join our team, with deep expertise in Microsoft Fabric, Azure Data Engineering services, and Databricks. The ideal candidate will have strong experience in building scalable data pipelines, integrating modern data platforms, and enabling advanced analytics solutions across the enterprise.
Key Responsibilities:
Area Responsibilities
Solution Architecture Design and implement scalable and secure data pipelines and architectures using Microsoft Fabric (Data Factory, Dataflows, Lakehouse) and Databricks.
Data Integration Ingest data from various structured and unstructured sources, including SQL, APIs, SAP, flat files, and real-time streams.
Development Develop ELT/ETL processes using PySpark/Scala in Databricks and Synapse/Fabric pipelines.
Lakehouse & Medallion Architecture Implement bronze, silver, and gold layer data models in Lakehouse environments using Delta Lake and Fabric's OneLake.
Performance Tuning Optimize Spark jobs, Fabric pipelines, and Delta tables for performance and cost.
DevOps & CI/CD Enable CI/CD pipelines for Fabric and Databricks notebooks using Azure DevOps/Git.
Governance & Security Implement role-based access control (RBAC), data lineage, and security best practices within Fabric and Databricks.
Collaboration Work closely with data analysts, data scientists, architects, and business stakeholders.