[Hiring] AI/ML Data Annotation Engineer REMOTE USA
Position: AI/ML Data Annotation Engineer
Date Posted: September 9, 2026
Industry: Artificial Intelligence / Machine Learning / Data Operations
Employment Type: Full Time
Experience: 3+ Years in Data Operations, ML Operations, Evaluation, or Annotation Engineering
Qualification: Bachelor’s Degree in Computer Science, Artificial Intelligence, Data Science, Engineering, or a related field
Salary: $30–$50/hour
Location: REMOTE, United States
Company: Rex.zone
Description:
Rex.zone is seeking a Remote AI/ML Data Annotation Engineer to support advanced AI and machine learning training workflows across the United States. This role focuses on creating high-quality annotation processes, improving training data quality, and supporting evaluation programs that enhance the performance of large language models (LLMs).
The ideal candidate will have experience with AI/ML data operations, annotation workflows, RLHF-style preference data, and quality evaluation processes. The successful applicant will work closely with engineering teams to develop scalable labeling systems, improve model performance, and ensure reliable training data outcomes.
Key Responsibilities:
• Build and maintain high-quality annotation workflows for AI/ML training projects.
• Develop labeling schemas, annotation guidelines, and ambiguity-resolution processes.
• Monitor training data quality metrics, error patterns, and continuous improvement plans.
• Design and manage QA evaluation programs, including review queues, spot checks, gold tasks, and agreement analysis.
• Perform RLHF preference labeling and ranking to support AI model alignment and performance.
• Conduct prompt evaluations and regression testing to measure model improvements.
• Support NLP tasks including named entity recognition and intent classification.
• Perform computer vision annotation tasks such as bounding boxes and segmentation when required.
• Execute content safety labeling to support responsible AI development.
• Collaborate with engineering teams to integrate annotation tools and datasets into LLM training pipelines.
Requirements:
• 3+ years of experience in data operations, ML operations, evaluation, or annotation engineering.
• Strong understanding of NLP, LLM evaluation, and human feedback signals such as RLHF.
• Experience with QA evaluation methods, sampling strategies, and disagreement analysis.
• Familiarity with annotation tools and workflow automation.
• Strong technical writing skills for creating clear guidelines and repeatable processes.
• Excellent analytical, problem-solving, and communication skills.
Strong knowledge of:
• AI/ML Data Annotation
• Large Language Model (LLM) Evaluation
• RLHF & Human Feedback Systems
• NLP Tasks & Model Testing
• QA Evaluation Programs
• Annotation Tools & Workflow Automation
• Data Quality Management
Disclaimer: The job details above are structured for clarity and based on publicly available content from recruiters/Company pages. All rights remain with the original source; names may be withheld for confidentiality. We are not involved in the hiring process.
