Member of Technical Staff - Research, AI Safety at Vals AI in San Francisco, CA

Job description

About the role We are looking for an exceptional researcher to design and build the next generation of AI benchmarks for catching misaligned behavior in frontier models and agents, e.g., reward hacking, scheming, deception, loss of control, multi-agent safety, etc. You will create high-impact, challenging evaluations that push the boundaries of what we can measure in foundation models. This role is ideal for someone with deep research expertise who wants to see their work directly influence how the world evaluates AI systems. You will lead the design and development of novel benchmarks and infrastructure that help assess misalignment risks in LLMs. Our benchmarks shape how foundation models are developed and generative AI applications are built. We work with every major foundation model lab - along with leading financial institutions and the application-layer companies pushing the frontier forward. Our work has been featured by the Wall Street Journal, New York Times, Washington Post, and Bloomberg. We are building the standard for evaluating the ability of LLMs to perform real-world tasks. You will be at the forefront of defining what that standard looks like. What you'll do • Design and develop novel, high-impact benchmarks and environments that assess real-world (mis)behavior, including reward hacking, scheming, deception and other misaligned behavior, in frontier models and agents (including multi-agent systems) • Conduct research to ensure our benchmarks are valid, reliable, and meaningful • Collaborate with foundation model labs and enterprises to understand evolving safety evaluation needs • Analyze model behavior across benchmarks and articulate results • Publish research findings and contribute to the broader safety and evaluation research communities • Stay current with the latest developments in LLM capabilities and alignment evaluation methodologies Requirements • Safety experience: Substantial, hands-on work in safety evaluations, monitorability, scheming, deception, or multi-agent safety. You may have worked on an AI safety or alignment team, or done a PhD focused on AI safety. You've published safety research, perhaps papers at leading conferences, or high-quality write-ups. • Deep ML/AI background: An understanding of neural networks and the fundamentals behind the model training process. • Strong engineering: Excellent coding skills and the ability to design and build complex evaluation environments yourself, mostly in Python. • Communication : Ability to clearly communicate complex research ideas to both technical and non-technical audiences. • Location: We are an in-person team in San Francisco and will support relocation or transportation as needed. Nice to haves • Mechanistic interpretability experience • Contributions to open-source safety or evaluation tools and datasets What we offer • Highly competitive salary and meaningful ownership. Excellence is well rewarded. • Relocation and transportation support • Health/dental insurance coverage • Lunch and dinner provided, free snacks/coffee/drinks • Unlimited PTO • $1,500 housing stipend (within one-mile radius) • Collaborate with leading AI labs and work on the front lines of AI development and evaluations • Opportunity to publish and present your work About us Founding team : The core methodology behind this platform comes from NLP evaluation research we conducted at Stanford. We raised a $5M seed from some of the top institutional and angel investors in the valley. Our team has prior work experience at NVIDIA, Meta, Microsoft, Palantir and HRT. Collectively, we have over 300 citations in our published work. Our early team includes Stanford PhDs, ex-Jane Street quants, and the first designer at Snorkel. We recently announced our $40M Series A at a $400M valuation, led by Andreessen Horowitz, with participation from existing investors 8VC, Pear VC, and Bloomberg and new investors Hudson River Trading and NextLadder Ventures. Tech stack : We use Python for m