Job Description

Job Title:

Senior Data Integration Engineer (AI ETL & Enterprise Integrations)

Overview:

We are building an advanced AI Orchestration and Integration Platform for financial and enterprise environments -- designed to connect, build, orchestrate, and monitor multi-agent AI workflows. The platform is deployed securely within customer environments, enabling seamless integrations and intelligent automation across business systems.

We are looking for a

Senior Data Integration Engineer

to lead the design and implementation of robust ETL/data infrastructure pipelines and enterprise tool integrations. This is a hands-on role for a technically strong professional who can architect, build, and scale high-performance data and integration workflows across distributed systems.

Key ResponsibilitiesETL Design, Evolution & Scaling

Build and manage Python-based ETL pipelines, containerized for deployment in Kubernetes or similar environments. Orchestrate data workflows using Airflow DAGs. Implement distributed computing with Ray, Polars, or PySpark for scalable embedding generation. Manage fault-tolerant data processing with checkpointing, retries, and parallel writes to Postgres, Vector DBs, and graph databases like Neo4j.

Messaging & Event-Driven Processing

Design event-driven ETL workflows using RabbitMQ or Kafka (e.g., events like etl.file.uploaded, etl.process.failed). Develop microservices for decoupled and asynchronous data processing. Enable real-time system notifications and user updates through message queues.

Enterprise Integrations

Build and maintain integrations with:

Communication tools:

Slack, Teams, Outlook, Gmail.

Storage systems:

SharePoint, Google Drive, OneDrive, S3.

Project management tools:

Jira, Asana, ADO, Confluence.

DevOps & Source Control:

GitHub, GitLab, Bitbucket.

Industry APIs:

Salesforce, ServiceNow, Databricks, and other legacy enterprise systems. Develop SDKs or plugin frameworks for rapid integration of new systems. Implement secure authentication and authorization (OAuth, SAML, API key management) and ensure proper RBAC propagation.

Observability & Reliability

Integrate monitoring and observability tools such as Prometheus, Grafana, and tracing frameworks. Deliver dashboards tracking ETL latency, throughput, and error rates. Ensure enterprise-grade production readiness with audit logging and compliance integrations.

AI & Data Alignment

Support AI pipelines by feeding structured data into retrieval-augmented generation (RAG) and multi-agent workflows. Optimize GPU-based processing for embedding and model-serving workloads. Collaborate with AI teams to align data flows with model orchestration and automation processes.

Ideal Candidate Profile

7-10 years of experience in data engineering, integrations, or distributed systems. Proficiency in Python ETL frameworks and distributed compute technologies (Ray, Polars, PySpark, Airflow). Strong hands-on experience with containerization and orchestration (Kubernetes/Rancher or similar). Proven expertise in building enterprise integrations with communication, storage, and project management platforms. Deep understanding of authentication and identity protocols (SAML, OAuth, API key management). Experience with relational (Postgres), vector, and graph databases (Neo4j). Familiarity with observability and monitoring stacks (Prometheus, Grafana, tracing tools). Bonus: Background in financial services or regulated enterprise environments.
Job Type: Contractual / Temporary
Contract length: 6 months

Pay: ₹80,000.00 - ₹100,000.00 per month

Work Location: Remote