Client Consulting Manager(data Engineer)

Year    KA, IN, India

Job Description

Company Description



Visa is a world leader in payments technology, facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories, dedicated to uplifting everyone, everywhere by being the best way to pay and be paid.


At Visa, you'll have the opportunity to create impact at scale -- tackling meaningful challenges, growing your skills and seeing your contributions impact lives around the world. Join Visa and do work that matters -- to you, to your community, and to the world.


Progress starts with you.



We are seeking an experienced Manager Data & AI Engineer to join our CEMEA team. This is a Staff-level individual contributor role for someone who thrives on solving complex technical challenges, architecting scalable data platforms, and driving engineering excellence. You'll lead critical data engineering initiatives, mentor talented engineers, and build the data infrastructure that powers insights and AI-driven solutions for Visa's global clients.


What You'll Do:


Data Platform Architecture & Development:

Design and build enterprise-scale data platforms using modern big data technologies including Spark, Hadoop, Kafka, and cloud-native services. Architect robust, scalable data pipelines that process petabytes of data for batch, streaming, and real-time analytics Drive technical decisions on architecture, tooling, and engineering practices that impact multiple projects and teams Establish and enforce engineering standards, best practices, and code quality across data engineering initiatives

Build Production-Grade Data Pipelines:

Develop and optimize large-scale ETL/ELT pipelines for data ingestion, transformation, quality assurance, and feature engineering Implement streaming data pipelines using Kafka and Spark Streaming for real-time analytics and decision-making Design data models, partitioning strategies, and optimization techniques for distributed systems Ensure data quality, reliability, and observability across all data workflows

Enable AI/ML & Advanced Analytics:

Build data infrastructure that supports AI/ML workloads including feature stores, training pipelines, and model serving infrastructure Collaborate with data scientists to productionize machine learning models through robust MLOps practices Design and implement data pipelines for GenAI applications including embeddings generation, vector storage, and retrieval systems Support deployment of AI/ML models with scalable inference pipelines and monitoring

Drive Cloud Infrastructure & DevOps Excellence:

Manage and optimize AWS/Azure cloud infrastructure (S3, EMR, EC2, Lambda, Glue, Redshift, SageMaker) Build CI/CD pipelines and automate deployments using Jenkins, Git, Docker, and Kubernetes Implement workflow orchestration using Airflow, Prefect, or Control-M Design for high availability, disaster recovery, and system reliability

Technical Leadership & Collaboration:

Mentor junior data engineers, fostering a culture of continuous learning and innovation Code reviews and technical discussions to elevate team capabilities Partner with product managers, data scientists, and business stakeholders to translate requirements into technical solutions Stay current with emerging technologies and drive adoption of best practices in data engineering and AI/ML infrastructure

This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager.

Qualifications



8+ years of hands-on data engineering experience with a Bachelor's degree, or 6+ years with a Master's degree in Computer Science, Engineering, Statistics, or related technical field

Proven track record of building and leading complex data engineering projects at scale



Must-Have Technical Skills:

Core Data Engineering Expertise:

Expert proficiency in Python and Scala/Java for building production data systems

Deep hands-on experience with Apache Spark (Spark SQL, DataFrames, Streaming) including performance tuning and optimization

Strong expertise in Hadoop ecosystem: HDFS, Hive, HBase, YARN

Production experience with Kafka for building event-driven and streaming architectures

Advanced SQL skills with experience in both RDBMS and NoSQL databases (Cassandra, MongoDB, Redis)

Proven experience designing and deploying large-scale ETL/ELT pipelines processing terabytes of data

Strong AWS/Azure experience: S3, EMR, EC2, Lambda, Glue, Redshift, SageMaker

Solid understanding of data modeling, partitioning strategies, and distributed systems optimization



DevOps & Infrastructure:

Experience with CI/CD pipelines (Jenkins, GitLab CI, GitHub Actions)

Hands-on experience with Docker and Kubernetes for containerization and orchestration

Proficiency with workflow orchestration tools like Airflow, Prefect, or Control-M

Experience with infrastructure as code and automation



MLOps & AI Infrastructure:

Experience building feature engineering pipelines and feature stores

Understanding of MLOps workflows: model deployment, versioning, monitoring, and automation

Experience building data pipelines that support ML/AI workloads

Familiarity with model lifecycle management and productionization



Preferred Skills:

Advanced Data Engineering:

Experience with real-time processing frameworks (Flink, Spark Streaming, Kafka Streams)

Familiarity with modern data platforms (Databricks, Snowflake)

Experience with data quality frameworks and observability tools (Great Expectations, Datadog, Prometheus)

Knowledge of DR/HA architectures and reliability engineering

Multi-cloud experience (Azure, GCP)

Understanding of data governance, security, and compliance



AI/GenAI Infrastructure:

Experience with vector databases (Pinecone, Weaviate, Milvus, ChromaDB, FAISS)

Understanding of RAG (Retrieval-Augmented Generation) system architectures

Familiarity with Model Context Protocol (MCP) for LLM integrations

Experience with embeddings generation and semantic search pipelines

Exposure to model serving frameworks (TensorFlow Serving, Triton, SageMaker endpoints)

Knowledge of cloud AI services (AWS Bedrock, Azure OpenAI, Vertex AI)

Experience with LLM orchestration frameworks (LangChain, LlamaIndex)



What Makes You Stand Out:

Strong architectural thinking with ability to design systems for scale, reliability, and maintainability

Proven ability to drive technical initiatives independently with minimal supervision

Deep problem-solving skills and comfort navigating ambiguity in complex technical environments

Excellent communication and stakeholder management abilities

Passion for mentoring and elevating engineering teams

Curiosity and adaptability to stay ahead of emerging technologies in data engineering and AI/ML

Additional Information



Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Beware of fraud agents! do not pay money to get a job

MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.


Job Detail

  • Job Id
    JD5150766
  • Industry
    Not mentioned
  • Total Positions
    1
  • Job Type:
    Full Time
  • Salary:
    Not mentioned
  • Employment Status
    Permanent
  • Job Location
    KA, IN, India
  • Education
    Not mentioned
  • Experience
    Year