Job Description

Attention All Data Folks!!!
We are Hiring Sr Data Engineering Lead (Azure/AWS) in Indore, MP (Hybrid Model)
Below is the JD for the reference
Sr Data Engineering Lead
Indore, MP (Hybrid 3 Days a Week)
Full Time
Exp Level- 8-15+ years
About the Role
Ccube is seeking a highly skilled Data Engineering Lead to architect, build, and scale our enterprise-grade data infrastructure. This role demands a strong technical foundation in distributed data systems, deep hands-on experience with modern big data and cloud technologies, and proven leadership in building and mentoring high-performing data engineering teams. The ideal candidate will shape the data engineering roadmap, drive innovation, and ensure robust, scalable, and efficient data solutions across the organization.
Key Responsibilities

• Data Strategy & Architecture

• Define the end-to-end data engineering strategy and technical vision aligned with Ccube's business goals.

• Architect scalable, high-performance, and cost-efficient data platforms on AWS, Azure, or GCP.

• Design and optimize data lakehouse medallion architectures integrating batch and streaming pipelines (Spark, Kafka, Delta Lake, Iceberg, Hudi, etc.).

• Build reusable frameworks for data ingestion, transformation, and orchestration across heterogeneous systems.

• Data Engineering Execution

• Lead the development and optimization of ETL/ELT pipelines using PySpark, Scala, SQL, and Airflow or equivalent orchestration tools.

• Oversee the implementation of real-time streaming solutions leveraging Kafka, Kinesis, or Pub/Sub.

• Guide the team in integrating structured, semi-structured, and unstructured data sources.

• Drive adoption of DataOps and DevOps best practices -- CI/CD for data pipelines, automated testing, and monitoring.

• Cloud & Infrastructure

• Design and manage cloud-native data solutions (AWS Glue, EMR, Redshift, Snowflake, BigQuery, Databricks, etc.).

• Optimize infrastructure for scalability, performance, and cost using Terraform, CloudFormation, or Pulumi.

• Lead initiatives for data platform modernization and cloud migration strategies.

• Governance, Security & Observability

• Define and enforce standards for data quality, lineage, governance, and metadata management (e.g., Great Expectations, Apache Atlas, or Collibra).

• Implement robust data security, compliance, and privacy frameworks aligned with industry standards (GDPR, HIPAA, etc.).

• Establish observability frameworks for data pipelines -- logging, monitoring, and anomaly detection.

• Leadership & Collaboration

• Lead and mentor a team of senior and mid-level data engineers, fostering a culture of excellence, ownership, and innovation.

• Collaborate cross-functionally with AI/ML, Analytics, and Product Engineering teams to enable data-driven decision-making.

• Evaluate emerging technologies (e.g., VectorDBs, GraphDBs, RAG frameworks) and drive their adoption for advanced data-driven use cases.

• Represent the data engineering practice in architecture reviews and executive technology forums.

Required Qualifications

• Experience:

• 10+ years in data engineering and architecture roles, with 3+ years in technical leadership or data platform lead roles.

• Technical Expertise:

• Deep proficiency in Spark, PySpark, Scala, SQL, and distributed data processing.
• Proven hands-on work in cloud data platforms - AWS (Glue, EMR, Redshift), GCP (Dataflow, BigQuery), or Azure (Synapse, Data Factory).

• Experience with workflow orchestration (Airflow, Dagster, Prefect) and containerization (Docker, Kubernetes).

• Expertise in modern data storage systems - Snowflake, Databricks, Iceberg, Hudi, or Delta Lake.

• Familiarity with Graph Databases (Neo4j, AWS Neptune) and Vector Databases (Pinecone, Weaviate, Milvus) for AI/RAG systems.

• Exposure to data observability, data mesh, and feature store frameworks.

• Leadership:

• Strong people management, mentorship, and cross-functional collaboration skills.

• Demonstrated success in building or scaling a data engineering function or CoE (Center of Excellence).

• Certifications (Preferred):

• AWS Certified Data Analytics - Specialty / GCP Professional Data Engineer / Databricks Certified Data Engineer / Snowflake Architect.

Why Join Ccube?

• Build the Future - Be part of a mission-driven company shaping the next generation of AI and digital solutions
• Collaborative Culture - Work with kind, brilliant people who value transparency, experimentation, and integrity
• Career Mobility - We invest in your learning and promote from within
• Compensation & Equity - Competitive base + commission + stock options
• Professional Growth - Annual stipend for certifications, courses, and industry events

Are you ready to help companies unlock the power of technology--and build something extraordinary in the process?
Feel Free to reach out to me at "rahul " for more details.
?
Powered by JazzHR

Skills Required