Data Scientist

About Us:

At Sauce Labs, we empower the world's top enterprises - like Walmart, Bank of America, and Indeed - to deliver quality web and mobile applications at speed. Our industry-leading platform ensures continuous quality across the SDLC, using AI-powered analytics to identify key quality signals from development through production. With our unified solution, teams can release and innovate with confidence, knowing their apps will always look, function, and perform exactly as they should. Backed by TPG and Riverwood Capital, we are shaping the future of digital confidence - join us!

The Role:

At Sauce Labs, we’re looking for a Data Scientist / GenAI Engineer to join our team and work directly with our engineering crew on the next generation of AI-powered products. You’ll be right in the mix of building, evaluating, and refining our new AI Assistant, helping our customers unlock deeper, smarter insights from their testing data. If you love collaborating across teams to turn complex data into helpful AI features, we’d love to meet you!

Responsibilities:

• Collaborate with the engineering team to execute experiments and provide insights
• Prompt engineering and optimization for accuracy, relevance, and hallucination reduction
• Research new use cases for AI-powered features
• Monitor the accuracy of AI solutions over time
• Collect and analyze data across Sauce Labs
• Manage the data directory across Sauce Labs - work with the data engineering team
• Analyze time-series testing datasets to identify patterns and insights
• Analyze telemetry data for performance and usage patterns
• Analyze logs and traces for root cause analysis
• Discover actionable insights from the data
• Evaluate model performance using GenAI evaluation frameworks
• Design and maintain golden datasets for GenAI evaluation
• Build evaluation pipelines using MLflow and LLM-as-judge frameworks
• Develop deterministic and LLM-based scoring rubrics for answer validation

Required Skills:

• Strong Python skills (Pandas, data manipulation, LLM frameworks)
• Experience with GenAI evaluation metrics (recall@k, MRR, faithfulness, F1)
• Proficiency in prompt engineering (few-shot, grounding, structured outputs)
• Familiarity with RAG techniques (hybrid retrieval, re-ranking, chunking strategies)
• SQL proficiency (Snowflake or PostgreSQL)
• Understanding of LLM-as-judge evaluation and scoring rubrics
• Knowledge of data governance (bronze/silver/gold data tiers)
• Experience with experiment tracking tools (MLflow, Weights & Biases, LangSmith)
• Experience with agentic frameworks (MCP, tool calling, ReAct patterns)

Nice to Have:

• Knowledge of fine-tuning techniques (SFT, LoRA, DPO)
• Familiarity with vector databases (Pinecone, Weaviate, Chroma)
• Understanding of LLM security (prompt injection defense, tool safety)
• Experience with advanced RAG (Graph-RAG, Self-RAG, Corrective RAG)
• Knowledge of Snowflake Cortex AI features

Please note our

privacy terms

when applying for a job at Sauce Labs.

Sauce Labs is proud to be an Equal Opportunity employee and values diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender identity/expression/status, sexual orientation, age, marital status, veteran status or disability status.

Security responsibilities at Sauce

At Sauce, we will commit to supporting the health and safety of employees and properties, partnering with internal stakeholders to learn and act on ever-evolving security protocols and procedures. You’ll be expected to fully comply with all policies and procedures related to security at the department and org wide level and exercise a ‘security first’ approach to how we design, build & run our products and services.