Job Description

Job Overview

We are seeking an experienced Lead Data Engineer to architect, design, and lead the development of our enterprise data infrastructure. The ideal candidate will have strong technical leadership skills, deep expertise in data engineering technologies, and experience managing data engineering teams across the globe.

Key Responsibilities

Technical Leadership

Architecture Design:

Design and implement scalable, robust data architectures for enterprise-level data processing

Data Pipeline Leadership:

Lead the development of complex ETL/ELT pipelines processing terabytes of data

Technology Strategy:

Evaluate and recommend cutting-edge data technologies and tools

Performance Optimization:

Optimize data processing performance and implement best practices

Data Quality:

Establish and maintain data quality standards and governance framework.

Team Management

Team Leadership:

Lead and mentor a team of 3-5 Data Engineers

Code Reviews:

Conduct thorough code reviews and establish coding standards

Technical Mentoring:

Guide junior engineers in advanced data engineering concepts

Project Management:

Manage multiple data engineering projects simultaneously

Stakeholder Communication:

Collaborate with product managers, analysts, and business stakeholders

Data Infrastructure

Cloud Architecture:

Design and implement cloud-native data solutions (AWS/Azure/GCP)

Big Data Processing:

Lead implementation of distributed computing frameworks (Spark, Hadoop)

Real-time Systems:

Architect streaming data pipelines using Kafka, Flink, or similar

Data Warehousing:

Design and optimize data warehouses and data lakes

Technical Requirements

Core Skills (Must Have)

Experience:

5-8 years in data engineering with 2+ team leader role

Programming:

Proficient in level Python, Spark and equivalent programming with experience in data engineering libraries

SQL Mastery:

Advanced SQL skills with experience in PostgreSQL, MySQL, SQL Server, Oracle

Cloud Platforms:

Deep experience with Databricks, Snowflake, AWS, Azure, or Google Cloud Platform

ETL/ELT:

Expert in designing and implementing complex data pipelines

Big Data:

Strong experience with Apache Spark, Hadoop ecosystem

Version Control:

Advanced Git workflows and CI/CD practices

Advanced Skills (Highly Preferred)

Streaming:

Experience with Apache Kafka, Apache Flink, or similar streaming platforms

Orchestration:

Apache Airflow, Prefect, or similar workflow orchestration tools

Containerization:

Docker, Kubernetes for data engineering workloads

Monitoring:

Data pipeline monitoring and observability tools

Data Modeling:

Advanced data modeling techniques (Star Schema, Data Vault, etc.)

Data Governance:

Good understanding on Data Governance and Security implementation on Data Solutions.

Leadership Skills

Team Management:

Proven experience leading data engineering teams

Technical Communication:

Ability to explain complex technical concepts to non-technical stakeholders

Project Management:

Experience managing multiple concurrent projects

Mentoring:

Strong mentoring and coaching abilities

Problem Solving:

Advanced troubleshooting and problem-solving skills

Think you are apt for this role?