The Role

You'll design and stand up human-in-the-loop data systems for frontier AI labs and enterprise ML teams: SFT, RLHF, evaluation, red teaming, multimodal annotation. The work is concrete: defining task schemas, quality frameworks, and data specs, then making sure they actually work in production.

This sits across pre-sales, solution design, and early delivery. You'll need to be technically credible with ML researchers, comfortable getting hands-on with data and tooling, and able to make good tradeoffs across quality, cost, and speed.

What You'll Do

Solution Design & Pre-Sales

• Translate ambiguous GenAI use cases into scoped, buildable data workflows
• Design end-to-end workflows: task schemas, guidelines, quality frameworks, sampling strategies, delivery models
• Lead technical discovery; surface constraints in data availability, task design, talent requirements, and tooling
• Produce clear proposals that align scope, pricing, and delivery assumptions

Data Quality & Evaluation

• Build quality frameworks: rubrics, acceptance criteria, audit models
• Design evaluation approaches for subjective, high-ambiguity tasks (reasoning, multimodal, safety/policy)
• Contribute to red teaming and adversarial testing tied to deployment risks

Hands-On Execution

• Analyze datasets; validate outputs; debug workflows
• Prototype task designs and evaluation pipelines (Python, Ango Hub, or client tooling)
• Partner with Delivery to stand up workflows; stay engaged through pilot and early production

Client & Internal Leadership

• Act as technical counterpart to ML researchers, product leads, and data teams
• Drive alignment across Sales, Solutions, Product, and Delivery
• Contribute to reusable patterns for workflows, pricing, and quality frameworks

What We're Looking For

Experience

• 5–10 years in AI/ML data systems or human-in-the-loop (or advanced degree + 3–5 years)
• Experience across at least two of: annotation, data generation/collection, evaluation, RLHF, red teaming
• Client-facing experience in a technical role

Technical

• Strong Python; can analyze datasets, prototype workflows, and debug issues
• Comfortable with unstructured and multimodal data (text, image, PDF, audio, video)
• Working understanding of ML workflows, especially training data and evaluation

GenAI & Quality

• Experience with GenAI data programs (SFT, preference data, evals, red teaming)
• Strong grasp of quality frameworks: rubrics, scoring, sampling, audit, adjudication
• Ability to reason about ambiguity, subjectivity, and model behavior

Judgment & Execution

• Strong decision-making under uncertainty; can balance quality, cost, and speed
• Comfortable operating hands-on in ambiguous, early-stage workflows

Communication

• Clear, concise communicator with technical and non-technical audiences
• Effective working across cross-functional, global teams

Nice to Have

• Research background (PhD or equivalent) in linguistics, STEM, or social science
• Experience working directly with frontier AI labs
• Experience with multimodal or agentic/interactive systems