Job Description

What does day-to-day look like:

Produce high-quality, robust Python code designed to verify specific requirements against complex LLM conversational inputs. This code must scale efficiently to handle a wide range of input variations. Go through intricate inputs provided by LLMs, understand the complex problems and reasoning present, and develop optimal verification code that accurately checks a certain requirement against the input data. Create extensive and rigorous test cases to validate the performance and accuracy of your verification code against the diverse LLM outputs. Your competitive programming experience will be essential in conceptualizing tests that validate the complexity and correctness of LLM conversations. Identify common underlying patterns and reusable logic within verification requirements. Develop modular, reusable code components to increase efficiency and consistency across different validation problems. Understand, modify, and implement changes to the verification requirements using both Python code and LLMs, based on the provided inputs. This ability to iterate on the verification logic is a must to accurately test and assess the client's LLM models. Collaborate closely with researchers and engineers to ensure validation methodologies align perfectly with model training goals. Incorporate feedback to refine verification logic and maintain clarity and consistency in all test and code documentation. Design innovative evaluation strategies and processes to improve the model's alignment with user needs and ethical guidelines. Create and refine optimal responses to improve AI performance, emphasizing clarity, relevance, and technical accuracy. Conduct thorough peer reviews of code and documentation, providing constructive feedback and identifying areas for improvement. Collaborate with cross-functional teams to improve model performance and contribute to product enhancements. Continuously explore and integrate new tools, techniques, and methodologies to enhance AI training processes.

Requirements

Experience:

3+ years of professional experience in software development, with a strong focus on Python.

Code Scalability and Optimization:

Proven ability to produce production-quality, robust code and a deep understanding of writing solutions that scale well and maintain performance under varied input conditions.

Analytical Prowess:

Excellent analytical and problem-solving skills, with the ability to quickly break down complex LLM conversational inputs into verifiable, structured requirements.

Competitive Programming Insight:

While not creating new competitive coding problems, prior competitive coding experience (e.g., knowledge of data structures, algorithms, time/space complexity) is highly desired and will be leveraged to validate the complexity and edge cases of LLM outputs.

Communication:

Proficient in English, with excellent written communication skills for articulating complex technical requirements and verification logic clearly.

Demonstrated Aptitude (Plus):

A profile on platforms like LeetCode, HackerRank, or GitHub demonstrating advanced algorithmic and problem-solving skills is a plus
Job Type: Contract
Contract length: 2 months

Benefits:

Work from home
Application Question(s):

Are you a track record via LeetCode / HackerRank / GitHub or similar? Are you fine with this Short term contract (1-2 Months)?
Experience:

Python: 4 years (Required) LLM: 3 years (Required)
Work Location: Remote