[Remote] Director of Product Management, AI Evals
Note: The job is a remote job and is open to candidates in USA. Harness is the AI Software Delivery Platform company, led by technologist and entrepreneur Jyoti Bansal. They are seeking a Director of Product Management to lead the AI Evaluation module, focusing on how engineering teams measure, validate, and improve AI applications in production.
Responsibilities
- Design AI-assisted experiences for eval authoring, failure analysis, and remediation recommendations
- Own the vision, strategy, roadmap, and P&L for Harness's AI Evals module
- Partner with engineering to deliver scalable, differentiated solutions that developers love
- Work closely with GTM teams (sales, marketing, customer success) on positioning, adoption, and revenue growth
- Engage directly with customers to shape the roadmap, improve adoption, and ensure measurable business impact
- Drive competitive positioning and track product metrics
Skills
- 8+ years of product management experience, including AI/ML technologies, developer tooling, or platform products
- Experience owning business outcomes including strategy, positioning, and revenue impact
- Strong communication and stakeholder management skills across product, engineering, and GTM teams
- Obsessed with customer success and outcome-driven
- Strong understanding of: LLM evaluation methodologies, CI/CD pipelines and developer workflows, AI observability, monitoring, and quality assurance, Agentic AI systems and the unique evaluation challenges they present
- Experience building AI/ML evaluation or observability platforms at scale
- Experience in building products from the ground up
- Familiarity with prompt engineering, fine-tuning pipelines, and dataset versioning
- Exposure to AI governance, responsible AI frameworks, or automated remediation
- Cloud-native and enterprise SaaS experience
Benefits
- Competitive salary
- Comprehensive healthcare benefits
- Flexible Spending Account (FSA)
- Flexible work schedule
- Employee Assistance Program (EAP)
- Flexible Time Off and Parental Leave
- Monthly, quarterly, and annual social and team building events
- Monthly internet reimbursement
Company Overview
Company H1B Sponsorship