Job Description

Role and responsibilities Collaborate with data engineers, data scientists, and stakeholders to understand data requirements, problem statements, and system integrations Utilize, apply & enhance GenAI models using state-of-the-art techniques like transformers, GANs, VAEs, and LLM models Implement and optimize GenAI models for performance, scalability, and efficiency Integrate GenAI models, including LLMs, into production pipelines, applications, and existing analytical solutions Develop user-facing interfaces and APIs to interact with GenAI models, including LLMs Utilize prompt engineering techniques to enhance model performance, including LLM models Apply software engineering principles to develop robust, scalable, and maintainable GenAI applications Build and deploy GenAI applications on cloud platforms Integrate GenAI applications with other applications, tools, and analytical solutions to create a cohesive user experience and workflow Continuously evaluate and improve GenAI models and applications based on data, feedback, and user needs Stay up-to-date with the latest advancements in GenAI research, development, software engineering practices, and integration tools Document code, models, and processes for future reference Build and maintain tools and infrastructure for data processing for AI/ML development initiatives. Technical skills requirementsThe candidate must demonstrate proficiency in, Collaborate with data engineers, data scientists, and stakeholders to understand data requirements, problem statements, system integrations, and RAG application functionalities. Use, apply, enhance GenAI models using state-of-the- art techniques like transformers, GANs, VAEs, LLMs (including experience with various LLM architectures and capabilities), and vector representations for efficient data processing. Implement and optimize GenAI models for performance, scalability, and efficiency, considering factors like chunking strategies for large datasets and efficient memory management. Integrate GenAI models, including LLMs, into production pipelines, applications, existing analytical solutions, and RAG workflows, ensuring seamless data flow and information exchange. Develop user-facing interfaces and APIs (RESTful or GraphQL) to interact with GenAI models and RAG applications, providing a user-friendly experience. Utilize LangChain and similar tools (e.g., PromptChain) to facilitate efficient data retrieval, processing, and prompt engineering for LLM fine-tuning within RAG applications. Apply software engineering principles to develop robust, scalable, maintainable, and production-ready GenAI applications. Build and deploy GenAI applications on cloud platforms (AWS, Azure, or GCP), leveraging containerization technologies (Docker, Kubernetes) for efficient resource management. Integrate GenAI applications with other applications, tools, and analytical solutions (including dashboards and reporting tools) to create a cohesive user experience and workflow within the RAG ecosystem. Continuously evaluate and improve GenAI models and applications based on data, feedback, user needs, and RAG application performance metrics. Stay up-to-date with the latest advancements in GenAI research, development, software engineering practices, integration tools, LLM architectures, and RAG functionalities. Document code, models, processes, and RAG application design for future reference and knowledge sharing.Nice-to-have skills Experience working with RAG applications Experience with cloud-based data warehousing solutions (e.g., BigQuery, Redshift, Snowflake) Experience with cloud-based workflow orchestration tools (e.g., Airflow, Prefect) Familiarity with Kubernetes (K8S) is a welcome addition Google Cloud certification Unix or Shell scripting