Job Description

Client: Our client is an international investment company.

Project overview: Our team is currently developing a new Data Warehouse and maintain a several applications and services that are tightly integrated with each other and ETL processes support.

Position overview: Our client is looking for a skilled Data Engineer with strong experience in Databricks and Azure to join their team. This role focuses on both new development (70%) and maintenance of these developments (30%) within Azure. The position is fully remote, based in India, with work hours that overlap with U.S. Eastern Time, requiring availability until 2 PM EST.

Responsibilities: Create and manage scalable data pipelines to collect, process, and store large volumes of data from various sources Integrate data from multiple sources, ensuring consistency, quality, and reliability Design, implement, and optimize database schemas and structures to support data storage and retrieval Develop and maintain ETL (Extract, Transform, Load) processes to accurately and efficiently move data between systems Build and maintain data warehouses to support business intelligence and analytics needs Optimize data processing and storage performance for efficient resource utilization and quick retrieval Create and maintain comprehensive documentation for data pipelines, ETL processes, and database schemas Monitor data pipelines and systems for performance and reliability, troubleshooting and resolving issues as they arise Stay up to date with emerging technologies and best practices in data engineering, evaluating and recommending new tools as appropriate


Requirements: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field (Engineering or Math preferred) 5+ years of experience with SQL, Python, .NET, SSIS, and SSAS 2+ years of experience with Azure cloud services, particularly SQL Server, ADF, Azure Databricks, ADLS, Key Vault, Azure Functions, and Logic Apps, with an emphasis on Databricks 2+ years of experience using Git and deploying code using a CI/CD approach Strong analytical and problem-solving skills Excellent communication and interpersonal skills Ability to work independently and as part of a team * Attention to detail and a commitment to quality