Job Description

Project Role :

Custom Software Engineer

Project Role Description :

Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs.

Must have skills :

Microsoft Azure Analytics Services, Microsoft SQL Server, Microsoft Azure Databricks, PySpark, Azure Data Factory, Databricks REST API

Good to have skills :

NA

Minimum

2

year(s) of experience is required

Educational Qualification :

15 years full time education



Summary: : Azure Data Engineer - ADF, Databricks, ADLS Role Overview We are seeking a skilled and detail-oriented Azure Data Engineer to join our data engineering and governance team. The ideal candidate will have hands-on experience in designing, developing, and maintaining scalable data pipelines using Azure Data Factory, Azure Databricks, and Azure Data Lake Storage (ADLS). This role involves working closely with cross-functional teams to deliver robust data solutions that support analytics, reporting, and initiatives. Roles & Responsibilities: -Design, build, and maintain ETL/ELT pipelines using Azure Data Factory and Databricks. -Develop workflows that ingest, transform, and load data from diverse sources into ADLS, ensuring data quality and consistency. -Implement Bronze, Silver, and Gold layer transformations in Databricks using PySpark and SQL. -Optimize pipeline performance and cost efficiency through monitoring and tuning. -Collaborate with data analysts, and business stakeholders to understand data requirements and deliver solutions. -Maintain metadata, lineage, and audit trails for all data processes. -Manage access controls and security across Databricks workspaces and ADLS folders. -Document architecture, data flows, and transformation logic for compliance and future reference. -Support Power BI integration by enabling reporting layers and sandbox environments within Databricks. Professional & Technical Skills: - Must To Have Skills: Proficiency in Microsoft Azure Analytics Services, Microsoft Azure Databricks, PySpark, Microsoft SQL Server. -5+ years of experience in Azure Data Factory, Azure Databricks, and ADLS Gen2. -Strong proficiency in SQL, PySpark, and Spark-SQL. -Experience with Databricks REST API for workflow orchestration. -Understanding of data governance, security (RBAC, Azure AD), and compliance frameworks. -Exposure to data modeling, incremental transformations, and data mart integration. Nice to Have -Knowledge of open-source lakehouse formats like Delta Lake, Iceberg, and Parquet. -Experience with machine learning workflows and AI model deployment in Databricks. -Familiarity with Agile/Scrum methodologies and DevOps practices. Work Environment -Multiple Databricks workspaces including production sandboxes -Collaboration with ADLS and Databricks teams for access provisioning and data governance. -Integration with SAP HANA and other enterprise systems for data ingestion. Additional Information: - The candidate should have minimum 2 years of experience in Microsoft Azure Analytics Services. - This position is based at our Pune office. - A 15 years full time education is required.




15 years full time education