Job Description

Senior Data Engineer - GCP Native Platform
Location: Fully Remote (Pan - India)
Role Summary
Seeking an experienced Senior Data Engineer to design and build scalable data pipelines on Google Cloud Platform. You will implement data solutions using BigQuery, Dataflow, Cloud Composer, and modern engineering practices while mentoring junior team members and driving technical excellence.
Key Responsibilities
Pipeline Development & Engineering
Build and optimize batch and streaming data pipelines using Dataflow (Apache Beam).
Develop and maintain Airflow DAGs in Cloud Composer for workflow orchestration.
Implement ELT processes using Dataform (SQLX) for transformations, testing, and documentation.
Design and maintain BigQuery datasets following medallion architecture (Bronze/Silver/Gold).
Create reusable pipeline templates and frameworks to accelerate development.
Data Platform Implementation
BigQuery: Optimized schemas, partitioning strategies, clustering, materialized views, and performance tuning.
Dataflow: Build Apache Beam pipelines in Python and/or Java for complex transformations.
Cloud Composer: Manage Airflow workflows with dependency management and scheduling.
Dataform: Author SQLX transformations, tests, and documentation.
Pub/Sub: Implement event-driven ingestion and CDC patterns.
Cloud Storage (GCS): Architect data lake structures, access patterns, and lifecycle policies.
Technical Delivery
Partner with architects on technical design and platform standards.
Conduct code reviews and enforce engineering best practices.
Troubleshoot and optimize pipeline performance and cost.
Implement data quality checks, monitoring, and observability.
Support production deployments and incident resolution.
Collaboration & Mentorship
Mentor junior engineers on GCP best practices and data engineering patterns.
Collaborate with analysts, data scientists, and cross-functional teams to enable data consumption.
Document technical solutions and maintain knowledge-base artifacts.
Participate in agile ceremonies and coordinate with onshore/offshore teams.
Required Qualifications
Experience
5+ years of Data Engineering experience; minimum 2 years working with GCP.
Hands-on experience with BigQuery at enterprise scale and production pipelines.
Strong background in building production-grade batch and streaming pipelines.
Development experience in Python and/or Java.
Proven track record of delivering complex data solutions.
Core Technical Skills
BigQuery (SQL optimization, scripting, stored procedures, partitioning).
Apache Beam / Dataflow (batch & streaming).
Airflow / Cloud Composer (DAG development).
SQL and Python programming proficiency.
Git, CI/CD pipelines and basic DevOps familiarity.
Additional Skills
Dataform or dbt experience.
Infrastructure-as-Code (Terraform basics).
Data modeling and schema design.
API integration (REST) and system interfacing.
Cloud Monitoring, logging, and data quality frameworks.
Preferred / Nice-to-Have
GCP Professional Data Engineer certification.
Experience with Vertex AI for ML pipeline integration.
Cloud Functions for serverless processing.
Dataproc / Spark workloads.
Real-time streaming architectures and low-latency processing.
Familiarity with data governance tools (Atlan, Collibra, Dataplex).
Experience with legacy system migrations.
BI tools experience (Power BI, Looker).
Containerization (Docker) and orchestration (Kubernetes).
What We're Looking For
Strong problem-solving and analytical skills.
Self-motivated and able to work independently in a remote environment.
Excellent communication, documentation, and stakeholder management skills.
Passion for data engineering, continuous learning, and mentoring others.
Collaborative team player with a delivery-focused mindset.

Skills Required