AI Data Quality Analyst (Paradox)

Join our Tel Aviv team as an AI Data Analyst and help shape the future of AI. We're looking for analytical minds to contribute to our cutting-edge Large Language Model (LLM) project. As an AI Data Quality Analyst, you will evaluate LLM-generated conversations and other text-based data, identify trends and regressions, and partner with AI Engineers and Product Managers to improve model performance.

Responsibilities

Review and evaluate LLM-generated conversations and other text-based outputs against task-specific quality criteria and success metrics; Perform high-quality annotation and data labeling to create reliable datasets for model training, benchmarking, and evaluation; Monitor real production data to identify quality trends, regressions, and areas for improvement; Identify qualitative patterns, recurring failure modes, regressions, and opportunities to improve model behavior and user experience; Analyze evaluation results and clearly communicate insights, risks, and recommendations to cross-functional stakeholders; Work closely with our AI Engineers and Product Managers to understand the data needs and ensure data quality processes align with goals, ensuring Paradox and Workday products remain top in class.

Requirements

Experience with data labeling or annotation, ideally involving textual data, conversational data, or generative AI; Strong qualitative research and analytical skills, with the ability to detect meaningful patterns in large volumes of textual data; Strong attention to detail, sound judgment, and a high bar for consistency and quality; Creative, out-of-the-box thinking and a proactive approach to investigating ambiguous or unexpected model behavior; Technical curiosity and confidence working across web-based tools, labeling platforms, dashboards, and evolving internal interfaces; Strong English fluency, both written and verbal communication; A research-oriented background, such as Linguistics, Human-Computer Interaction, Cognitive Science, Psychology, or a related field; Ability to work as from Workday Tel Aviv, Israel office location.

Nice to have

Experience working directly with LLM analysis, model evaluation, conversation quality assessment, or related AI quality workflows; Hands on experience using AI tools such as Claude Code, Cursor, or similar tools.