Job Description

Role: Data Engineer

Location: Bangalore Chennai, Pune, Hyderabd

Responsibilities

The candidate must be excited by the prospect of optimizing or even re-designing data pipelines to support the modernization of data platform Be curious and eager to work across a variety and volume of data. Design and build scalable and reliable data pipelines with the latest tech stack. Facilitate real-life actionable use cases leveraging our data with a user- and product-oriented mindset. Support teams without data engineers with building decentralized data solutions and product integrations. Conceptualize, design and implement improvements to ETL processes and data through independent communication with data-savvy stakeholders. Designing, building, and operating a Data Lake or Data Warehouse. Knowledge to ingest, cleanse, transform and load data from varied data sources in the above Azure Services (Databricks and DataFactory) Strong knowledge of Medallion architecture Consume data from source with different file format such as XML, CSV, Excel, Parquet, JSON Strong problem-solving skill such as backtracking of dataset, data analysis etc. Strong Knowledge of in advanced SQL techniques for carrying out data analysis as per client requirement.

Mandatory Skills

The candidates need to understand different data architecture patterns and parallel data processing.


1) S/he should be proficient in using the following services to create data processing solutions:

Azure Databricks Azure Data Lake Storage Azure Data Factory



2) Strong Knowledge in

PySpark SQL

3) Should be familiar to be built data pipelines and data analysis using Python

Desired Skills

Ability to query the data from serverless SQL Pool in Azure Synapse Analytics. Knowledge of Azure DevOps. Knowledge to configure any dataset with Vnet, Subnet Networks * Knowledge of Microsoft Entra ID, to create App registration for single and multitenant for security purpose.