Job Description

:

Education:

Engineer (or equivalent) degree in computer engineering or computer science.

Experience:

6-8 Years of Experience

-------------------

Develop and maintain data pipelines for efficient data extraction, transformation, and loading (ETL/ELT) processes, utilizing

PySpark

for distributed big data processing and

Polars/Rust

for

maximal performance and memory safety

in single-node bottlenecks.

Design, implement, and own internal process improvements: automating manual processes,

optimizing data delivery latency

using high-speed language components, and re-designing infrastructure for greater scalability and

cost efficiency

.

Work on the data pipeline operations to operate, maintain, and evolve our decoding pipelines, proposing improvements to automatize and industrialize all processes and ways of working with a focus on

data quality and platform stability

.

Support the ramp-up and installations of data pipeline for future airline/aircraft deployment.

Integrate high-performance Rust-compiled routines

(e.g., UDFs) into Python and PySpark workflows to resolve critical performance issues.

Technical Skills

--------------------

Core High-Performance & Distributed Computing

Rust:

Strong proficiency in Rust for developing

memory-safe, highly concurrent, and low-latency

data processing microservices or core pipeline components. Familiarity with the

Cargo

package manager.

Polars (Expert Level):

Mastery of Polars for high-speed,

multi-threaded data manipulation

on single machines. Deep understanding of the

Lazy API, Apache Arrow

columnar format, and query optimization techniques (e.g., predicate and projection pushdown).

Python:

Deep expertise in writing

production-grade, modular, and reusable code

(including Python packaging/wheels). Proven ability to orchestrate complex workflows and tooling.

PySpark (Expert Level):

Mastery of PySpark internals, including:


+

Advanced Performance Tuning:

Expertise in diagnosing and resolving bottlenecks using the

Spark UI

. Deep understanding of

Adaptive Query Execution (AQE)

, data skew mitigation, and optimizing shuffles.
+

Transactional Data:

Experience with

Delta Lake, Apache Hudi, or Apache Iceberg

for building reliable, ACID-compliant Data Lakehouse architectures (handling UPSERTs, Time Travel).

SQL (Expert Level):

Expert proficiency in

analytical SQL

(window functions, CTEs) and

database optimization

(indexing, partitioning, query plan analysis).

Data Orchestration & Platform Ownership

Workflow Orchestration (e.g., Apache Airflow, Dagster):

Proven experience in designing, deploying, and

maintaining complex, dependency-driven DAGs

in production environments, including failure recovery and alerting.

Infrastructure-as-Code (IaC):

Hands-on experience with

Terraform or CloudFormation

to provision, manage, and secure cloud data resources and compute clusters.

Cloud Platforms (AWS/Azure/GCP):

Hands-on experience with core cloud data services and

cost-management practices

(e.g., EMR/Dataproc, S3/ADLS/GCS, and serverless compute).

Data Governance & Software Engineering Practices

Data Quality & Lineage:

Experience implementing robust

data quality checks

(e.g., Great Expectations) and integrating with

Data Catalog/Lineage tools

.

Security & Access Control:

Practical experience implementing

least-privilege access (IAM/RBAC)

, data encryption, and

data masking/tokenization

for sensitive data (PII/PHI).

CI/CD & Testing:

Strong knowledge in building automated test, lint, and deployment pipelines (Jenkins, GitLab CI) for data services. Ability to write comprehensive

Unit Tests and Data Validation Tests

.

Domain Specific:

Palantir (Skywise) experience good to have.

Soft Skills

---------------

Good communication skills, including the ability to articulate the

technical trade-offs

between distributed (PySpark) and high-speed single-node (Polars/Rust) solutions to technical and non-technical stakeholders.

Sound knowledge in Agile/SAFe methodologies.

Ownership Mentality:

Ability to handle a subject from demand management, to development, and long-term production support.

Ability to solve complex networking, data, and software issues.

Good to have exposure in aviation industries.

This job requires an awareness of any potential compliance risks and a commitment to act with integrity, as the foundation for the Company's success, reputation and sustainable growth.

Company:

Airbus India Private Limited

Employment Type:

Permanent
-

Experience Level:

Professional

Job Family:

Digital
By submitting your CV or application you are consenting to Airbus using and storing information about you for monitoring purposes relating to your application or future employment. This information will only be used by Airbus.


Airbus is committed to achieving workforce diversity and creating an inclusive working environment. We welcome all applications irrespective of social and cultural background, age, gender, disability, sexual orientation or religious belief.


Airbus is, and always has been, committed to equal opportunities for all. As such, we will never ask for any type of monetary exchange in the frame of a recruitment process. Any impersonation of Airbus to do so should be reported to emsom@airbus.com .


At Airbus, we support you to work, connect and collaborate more easily and flexibly. Wherever possible, we foster flexible working arrangements to stimulate innovative thinking.