Staff AI Research Engineer

Distributional is looking for a research engineer to pursue novel research methodologies around testing of AI-powered systems. Distributional is building the modern enterprise platform for AI testing to make AI safe, reliable, and secure. AI is complex, unpredictable and constantly changing. Better AI testing is required to build confidence in non-deterministic AI-enabled applications, but AI testing is broken today. We’re building the modern AI testing stack for enterprises to fix this. We’re a Series A company with deep experience backed by the best investors.

Distributional is powered by computational statistics and machine learning to facilitate in depth and extensible analysis and testing.  Our research engineers collaborate with the Product team to identify valuable opportunities to improve Distributional through these advanced methodologies.  Then Research is off to the races: reviews of scholarly content; quantifying success; experimenting to identify positive outcomes; rigorously arguing amongst the other research engineers; moving the successes into the product roadmap, and eventually into production; working with GTM to develop sales and marketing collateral.  Because of our fast pace and emphasis on research mentality, you will have an opportunity to contribute to fundamental elements of Distributional’s success.

• Explore viable paths to improve/expand our computational offerings
• Conduct experiments to understand the potential for proposed Research initiatives
• Present and discuss your findings to the Research team for rigorous review and movement into the product roadmap
• Collaborate with product and engineering teams to adapt and iterate on research solutions for a robust and scalable implementation in production.
• Share Distributional values: Purpose, Humility, Optimization, Quality, Experimentation, Transparency

• Applied research experience in Machine Learning, Statistics, Mathematics, Operations Research or related field
• 7+ years of experience, including 3+ years of industry experience conducting product-focused research, including expertise in computational experimentation with Python
• Ability to rapidly prototype research ideas into code.
• Ability to guide your own research projects, organize experimental results concisely, present them for peer review within the company, and rigorously analyze other engineers' experiments.
• Communication: Ability to operate independently and as part of a team; able to communicate outwards and upwards to keep yourself unblocked.
• Effectiveness: Ability to work as part of a fast paced and fully remote team prioritizing your own time effectively.

• Deep understanding of statistical analysis and statistical testing
• Expertise with LLM tooling and evaluation in production settings
• Expertise with visualizing data in higher dimensions, including developing new visualizations

If this sounds like you, apply! If you don’t meet all of the qualifications but think you could be a match, we’d still love the chance to review your application. At Distributional, we encourage people from all ages, abilities, and experiences to apply. Distributional does not discriminate on the basis of race, color, ancestry, religion, national origin, sexual orientation, age, citizenship, marital or family status, disability, gender, gender identity or expression, pregnancy or caregiver status, veteran status, or any other legally protected status.

Distributional is a remote-first company, with hubs in Toronto, NYC, and the Bay Area. The whole company gets together in person for 3 off sites each year. Distributional offers comprehensive health coverage (with options including 100% company-paid premiums), flexible time off (in addition to two annual company-wide breaks in July and December), wellness stipends, learning & development stipends, as well as access to 401K.

We offer competitive compensation aligned to tier one markets. Our estimated salary for this role at Distributional ranges from USD $200,000 to $300,000. Actual pay is determined by multiple factors such as skills, qualifications, experience and market demand.