Job Description

Senior AI Engineer (Regression Evaluation & Model Quality Systems)
Location: Flexible (India)
Type: Full-time
Company: Zscaler
About Zscaler For over a decade, Zscaler has been redefining cloud security by delivering a globally distributed, cloud-native platform that secures users, applications, and data across 185+ countries. Protecting more than 7,000 enterprises and detecting over 150 million threats daily, Zscaler's mission is to make secure digital transformation seamless. We thrive in a fast-paced, innovative culture built on collaboration, creativity, and accountability. Our teams consist of some of the brightest minds in technology-driven by the challenge of building intelligent, scalable, and secure systems that power the modern enterprise. About the Team The AI and Data Science team leads Zscaler's enterprise data intelligence strategy, focusing on building AI-driven systems that enhance automation, trust, and decision-making. We design and implement scalable ML pipelines, evaluation frameworks, and generative AI tools to measure, monitor, and optimize model performance across the company's ecosystem. Our group bridges machine learning research, data engineering, and platform development, creating the foundation for reliable and explainable AI across predictive analytics, cybersecurity, and enterprise automation. Role Overview We are seeking a Senior AI Engineer with a strong foundation in regression modeling, evaluation systems, and AI infrastructure. In this role, you will design and build the pipelines that measure and ensure the reliability of large-scale AI systems-including large language models (LLMs), generative AI services, and predictive ML models. You will collaborate with data scientists, ML engineers, and DevOps teams to develop scalable, automated evaluation systems that quantify model performance, detect drift, and ensure statistical consistency across diverse business and technical use cases.
Key Responsibilities
Design, implement, and automate regression-based evaluation systems for large language models and predictive ML services.
Develop statistical and ML pipelines to monitor model drift, bias, and performance trends across large-scale datasets.
Analyze regression outputs and experiment data to identify key quality drivers and actionable insights for model improvement.
Partner with ML and product teams to benchmark and fine-tune models for applications including generative AI, anomaly detection, and customer analytics.
Collaborate with DevOps and platform teams to deploy evaluation and monitoring tools on AWS or GCP infrastructure (EKS, ECS, SageMaker, Vertex AI).
Build dashboards and reporting frameworks to visualize regression performance metrics (RMSE, MAPE, R , stability indices, etc.).
Document evaluation methodologies and standardize reproducible model validation frameworks across teams.
Stay current with advances in regression evaluation, LLM testing, and scalable ML performance measurement.
Qualifications
Bachelor's or Master's degree in Computer Science, Data Science, Statistics, or a related technical field.
5+ years of hands-on experience in AI/ML engineering, data science, or model evaluation, with emphasis on regression analysis and model reliability. Strong proficiency in Python (pandas, NumPy, scikit-learn, PyTorch/TensorFlow).
Solid understanding of machine learning evaluation techniques, including performance metrics, A/B testing, and model drift analysis.
Experience with cloud platforms (AWS or GCP) and deploying evaluation systems using EKS, ECS, SageMaker, or Vertex AI.
Familiarity with LLM frameworks (LangChain, RAG architectures, vector databases) or generative AI evaluation is a strong plus.
Demonstrated ability to build automated regression testing and monitoring systems for production ML pipelines.
Preferred Skills
Experience with MLOps, CI/CD for ML pipelines, and monitoring platforms (e.g., MLflow, Weights & Biases, or Prometheus).
Strong statistical background with expertise in time-series regression, panel data, or causal inference.
Knowledge of data visualization and reporting tools (Plotly, Streamlit, Power BI).
Excellent collaboration and communication skills with cross-functional teams.
A passion for reliable, explainable, and measurable AI systems.

Skills Required