Job Description

As a Foundation Model Engineer at ZenteiQ, you will lead the development of large-scale Scientific Foundation Models that integrate deep learning with physical and scientific principles. This role blends AI research, systems engineering, and architectural innovation to create models capable of reasoning, prediction, and scientific understanding.
Your work will directly contribute to ZenteiQ's mission to build sovereign, ethical, and high-performance scientific AI systems that transform industries such as aerospace, materials science, and energy.
Key Roles & Responsibilities:
Model Architecture: Design and implement scalable foundation model architectures (10B-100B+ parameters) using transformer and hybrid attention mechanisms.
Distributed Training: Build high-efficiency training pipelines with FSDP, DeepSpeed, and Megatron-LM to optimize multi-node GPU workloads.
Multimodal Integration: Develop cross-modal models combining text, image, simulation, and sensor data for scientific applications.
Performance Optimization: Enhance inference and training performance using quantization, KV-cache management, mixed precision, and CUDA kernel optimization.
Research and Experimentation: Conduct ablation studies, evaluate new architectures (MoE, GQA, MQA), and publish results in internal and external forums.
Collaboration: Work closely with the data, infrastructure, and HPC teams to ensure scalable deployment and compliance with security standards.
Mentorship: Guide junior engineers in model design, training efficiency, and research best practices.
Project Goals:
Develop and deploy ZenteiQ's Scientific Foundation Model (SFM) capable of reasoning over scientific data and equations.
Achieve breakthroughs in physics-informed deep learning, integrating domain constraints with AI-driven learning.
Advance India's sovereign AI initiative by building a national-scale scientific AI system optimised for precision, interpretability, and scalability.
Contribute to open research publications, ethical AI practices, and reproducible model design frameworks.
Requirements
Required Skills & Qualifications:

• Master's or Ph.D. in Computer Science, AI/ML, or related field.

• Expertise in transformer-based architectures (GPT, LLaMA, T5, etc.)

• Proficiency in PyTorch, CUDA, and distributed training frameworks (DeepSpeed, FSDP, Megatron-LM)

• Experience with HPC environments, multi-GPU scaling, and parallelization strategies (tensor, pipeline, data)

• Strong understanding of numerical methods, scientific computing, or physics-informed ML

• Proven track record in large-model research, optimization, or deployment

• Publications or open-source contributions in foundation models preferred.

Tech Stack:
PyTorch o CUDA o Flash Attention o DeepSpeed o FSDP o Megatron-LM o JAX o Triton o MoE o RoPE / ALiBi o xFormers o HPC o Linux
Benefits
What We Offer:
Work on cutting-edge national-scale AI systems.
Access to H100/H200 GPU clusters and next-gen compute infrastructure.
Competitive compensation, performance bonuses, and equity options.
Opportunity to publish and present at global AI conferences.
A collaborative, ethical, and innovation-driven work environment.

Skills Required