Source: CrawlJobs

Data Scientist, Agent

Lovable Stockholm, Sweden
Full-time On-site work

Publication date: 16/11/2026

TL;DR: You own how we measure and improve Lovable's AI agent. You build the eval systems and experiments that tell us whether a change makes the agent better or worse, and you turn agent telemetry into the fixes that raise success rates and cut errors. At Lovable, data scientists are not isolated model-builders; they sit close to the product, experimenting continuously with how intelligence changes user behavior and product dynamics.

Responsibilities

Define and own the metrics for agent quality: success, completion, error rates, and the behaviors that drive them; Build the eval systems and experiment framework that decide whether an agent change ships, such as an A/B-tested rollout; Turn agent traces and telemetry into concrete fixes, working directly with the agent engineering team; Build the tooling and agents that produce evaluations continuously as the agent evolves; Set the bar for how to judge agent behavior where there is no answer key to check against

Requirements

Experience or strong interest in LLM evaluation and observability: building evals, scoring outputs, tracing agent behavior, and catching regressions; Strong SQL and Python; Applied statistics and experimentation; Comfortable designing A/B tests for agent changes where outcomes are noisy; Ability to build systems and agents that produce insight continuously rather than one-off analyses; Instinct for what "good" looks like in agent behavior (success, error rates, task completion) and how to measure it when there is no clean answer key; Entrepreneurial, thrives in ambiguity, and works closely with engineers
Apply with employer

Similar offers

All →

Cookie settings

Each purpose is optional and off by default. You can withdraw consent at any time.

Showing offers closest to you