Research Engineer, Content Understanding

Exa is an applied AI lab building a search engine. The team builds massive-scale infrastructure to crawl the web, trains embedding models to process it, and designs vector databases to retrieve over it. The research engineer will work on document understanding for search: parsing pages into content and furniture, classifying page type and topic, judging usability, publication date, quality and trustworthiness, and identifying novel information. The role involves working on every page on the web, in every language, including credibility, misinformation, AI-generated content, and pages written for search engines rather than readers.

Responsibilities

Make parsing work on pages where it currently does not, and prove the improvement; Teach a model to judge page quality and agree on what quality means well enough to supervise it; Work on credibility and misinformation as a modelling problem; Decide whether two documents are semantically the same or genuinely different for web deduplication; Build classification and extraction that is accurate at web scale and cheap enough to run on all of it; Design the supervision for something without labels and find out whether it is learnable; Trace a bad search result back to the page-level prediction that caused it and fix it at the source

Requirements

Graduate-level ML experience (Master's or PhD with at least 2 years of relevant experience), or an exceptionally strong undergrad; Ability to build a transformer from scratch in PyTorch; Experience training models that had to be cheap enough to run everywhere; Enjoys building large-scale datasets and working with data, with most gains coming from supervision rather than architecture; Comfortable with problems where ground truth does not yet exist and defining it is part of the job; Cares about finding high quality knowledge and recognizes its importance for the world