Join a leading research organization at the forefront of LLM development! As an LLM Research Evaluator, you will play a crucial role in shaping the future of artificial intelligence. Your expertise in research, content, and analytical thinking will directly impact the development and refinement of cutting-edge language models. This is an exceptional opportunity to contribute to a dynamic field and make a tangible difference in how AI interacts with the world.
This role calls for a meticulous and insightful individual with a passion for both research and AI. You will be responsible for rigorously evaluating the output of LLMs, ensuring accuracy, identifying biases, and ultimately contributing to the creation of more reliable and effective AI models. This position offers a unique blend of intellectual challenge and real-world impact.
Responsibilities:

• Conduct in-depth research across various domains, including finance, mathematics, and science, to assess the accuracy and depth of LLM outputs.
• Critically evaluate the LLM’s ability to synthesize information from multiple sources, identify credible sources, and fact-check information.
• Assess the quality of LLM-generated content, focusing on clarity, coherence, and adherence to research standards.
• Employ prompt engineering techniques to test the LLM’s research capabilities and identify potential weaknesses.
• Analyze LLM outputs for biases, logical inconsistencies, and factual errors.
• Collaborate with cross-functional teams to provide clear feedback and contribute to LLM development.

Must-Have Qualifications:

• Native English speaker.
• Ph.D./Master’s degree in Political Science, Linguistics, Creative Writing, Content Writing, Arts, Essay Writing, Scripts, or equivalent experience as an English Teacher.
• Excellent written, verbal, and interpersonal communication skills.
• Strong analytical and critical thinking skills.
• Proficiency in using computers.
• Knowledge of research methodologies and data gathering techniques.
• Ability to work independently and as part of a team.

Nice-to-Have Qualifications:

• Knowledge in Finance domains.
• Experience with data curation and annotation.
• Familiarity with AI tools and platforms.
• Understanding of LLM limitations and common failure modes.

The target hiring compensation range for this role is $48.69 to $54.10 an hour. Compensation is based on several factors including, but not limited to education, relevant work experience, relevant certifications, and location.
About Aquent Talent:
Aquent Talent connects the best talent in marketing, creative, and design with the world’s biggest brands.
Our eligible talent get access to amazing benefits like subsidized health, vision, and dental plans, paid sick leave, and retirement plans with a match. More information on our awesome benefits!
Aquent is an equal-opportunity employer. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, and other legally protected characteristics. We’re about creating an inclusive environment-one where different backgrounds, experiences, and perspectives are valued, and everyone can contribute, grow their careers, and thrive.