$38-$43 per hour
Hourly contract
Remote
Mercor is looking for a Bilingual German STEM Expert with experience in biology/physics/chemistry theory and problem-solving to develop and refine high-quality reasoning questions that evaluate AI models. Your knowledge in biology/physics/chemistry will ensure the accuracy, rigor, and instructional quality of these test items. You will create and evaluate German/English prompts and responses, ensuring scientific precision while maintaining clear, natural German phrasing and alignment with English where needed.
Design and Optimize STEM-Based Prompts (Bilingual): Create detailed prompts in German and/or English with multiple constraints and scientific instructions.
Define and Document Evaluation Standards: Establish high-level expectations for correct responses in STEM contexts, and develop comprehensive rubric that accounts for scientific rigor and—when in German—linguistic clarity and appropriate academic tone.
Conduct Model Testing and Grading (Bilingual): Run prompts through models and assess preliminary outputs against expectations for scientific accuracy, reasoning quality, and clarity, comparing German vs. English where needed.
Support Benchmarking and Quality Assurance: Collaborate in QA review processes to ensure prompt tasks and rubrics meet scientific rigor, maintaining consistency and reliability across German-language benchmarks before integration into official benchmarks.
Native-level fluency in German (written) with strong reading/writing ability in English
BS or BA in Biology/Physics/Chemistry
Familiarity with undergraduate-level biology/physics/chemistry topics
Strong writing and critical thinking skills
Ability to work independently and meet deadlines
Complete an AI-led interview, this should take around 15 minutes.
Complete a 45-minute written assessment that will guide you through writing rubrics.
If selected, you will be invited to work on the project.
Expect to contribute at least 20 hours per week.
Expect a commitment of around 1 month.
You'll be working in a structured project environment with clear goals and tools.
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Mercor partners with leading AI labs and enterprises to train frontier models using human expertise. You will work on projects that focus on training and enhancing AI systems. You will be paid competitively, collaborate with leading researchers, and help shape the next generation of AI systems in your area of expertise.
Tagged as: Chemistry, Computer Science, Data Science, Earth Science, Engineering, Environmental Science, Life Sciences, Mathematics, Physics
Lilly Health Physicist – Radiation Safety Professional At Lilly, we unite caring with discovery to make life better for people...
ApplyPostdoctoral Research Associate The Postdoctoral Research Associate will contribute to an NIH-funded project investigating age-related changes to episodic memory for...
ApplyPostdoctoral Research Associate Dr. Masatoshi Inoue's lab is currently searching for a postdoctoral research associate. Our lab focuses on exploring...
ApplySr. Scientist We are seeking a highly skilled, motivated and innovative Sr. Scientist to contribute to antiviral immunological therapies. In...
ApplyAssistant Research Scientist This project focuses on documenting the Assessing Risks and Impacts of AI (ARIA) dataset, which is used...
ApplyScientist (Oligo Synthesis) – Sbx Molecular Engineering At Roche, you can show up as yourself, embraced for the unique qualities...
ApplyPlease visit work.mercor.com.
Don't forget to mention that you found the position on jobRxiv!
