Rlhf jobs

I’m assembling a training corpus for a math-focused large-language model and need a calculus specialist to enrich selected research papers with clear, accurate explanations of every sophisticated step. You’ll read each PDF, insert concise LaTeX-formatted comments that unravel dense derivations, and flag any ambiguous notation so the model grasps the intended meaning. Deliverables • Annotated PDF (or .TEX) with inline clarifications • Companion JSON that maps each comment to its passage and supplies a one-sentence paraphrase Acceptance criteria • All non-trivial calculus moves are explained in ≤3 sentences, free of factual error • Original math notation preserved; added text compiles in LaTeX • File names and JSON keys match the template I...

...SFT, RLHF/RLAIF, DPO, and reward modeling. Methodology Pressure-Testing: Scrutinize and refine our evaluation framework to ensure our scholar-led datasets stand up to the technical standards of frontier lab eval teams. Pipeline & Environment Development: Lead the development of pilot training environments and benchmarking tools that test model capabilities beyond traditional STEM domains. Cross-Domain Collaboration: Interface directly with Bentham CEO Marcus Heal and doctorate domain experts to translate qualitative human reasoning into rigorous, verifiable alignment signals. Qualifications Prior experience in model post-training, preference tuning, or evaluation design at a major AI lab (e.g., OpenAI, Anthropic, Google DeepMind, Meta etc). Deep technical understanding of S...

Subject: Alignerr RL Task Setup + Validation | Python, Docker, Hi I can set up your Alignerr RL task template end-to-end and deliver a validated PR-ready package today. I’ve worked with Dockerized ML evaluation harnesses, GitHub PR flows, and deterministic scoring rubrics for RLHF/Alignerr-style tasks. **Here’s exactly how I’ll deliver your requirements:** **1. Repo Setup & Task Creation** Clone and run the template. Create `problems/<your_task_name>/` from the starter template. Configure `` metadata + Dockerfile with pinned Python/ML deps to ensure reproducibility. **2. Scoring & Validation Logic** Implement `` with a deterministic, idempotent rubric. No randomness, no external API calls. Clear error handling for malformed inputs. I’ll ...

A MuJoCo task in this repo is a robotics/simulation benchmark where the agent must produce a verifiable artifact, usually an MJCF model file like /tmp/output/ or a controller like /tmp/output/policy.py. The artifact is graded by scorer/ using deterministic MuJoCo checks: compile the model, inspect joints/bodies/sensors, run fixed rollouts, measure stability/control/o...robotics challenge for an AI to solve, plus an automatic judge that proves whether the solution really works in simulation. Example: examples/mujoco-pendulum asks the agent to create a damped pendulum MJCF with specific physical properties; the scorer checks that it compiles, has the right hinge/body/sensors/mass/COM, and remains stable during rollout.

...required by the platform. • Target goal: Successfully complete and pass at least 10 interviews, assessments, certifications, or qualification screenings per week. Covered Work Types The contractor may be assigned work including, but not limited to: • AI response evaluation • AI model evaluation • LLM evaluation • Prompt engineering • Prompt writing • Prompt optimization • Prompt evaluation • RLHF (Reinforcement Learning from Human Feedback) • AI safety evaluation • AI reasoning evaluation • Fact-checking • Hallucination detection • Response ranking • Response comparison • Multi-turn conversation evaluation • Chatbot evaluation • Creative writing evaluation • Technical writing evaluat...

Overview: We are looking for an AI evaluation specialist with experience in OpenClaw, Outlier, LLM evaluation, Python, and rubric design. The role involves designing complex OpenClaw agent tasks, evaluating model traje...especially pytest/unit testing - Ability to write clear rubrics and evaluation criteria - Strong English reading and writing skills - Detail-oriented and able to follow complex project instructions - Understanding of AI safety, privacy, and prompt injection risks - Ability to work remotely and asynchronously Nice to Have: - Prior OpenClaw Atlas or OpenClaw RL project experience - Experience with RLHF, model ranking, or trajectory evaluation - Familiarity with browser tools, APIs, spreadsheets, email/calendar workflows - Experience reviewing safety failures or AI ta...

...reviewing responses produced by Large Language Models (LLMs) and evaluating them for quality, accuracy, relevance, and alignment with UK English language, culture, and local context. Candidates should have strong analytical skills and a deep understanding of UK language usage, slang, culture, and everyday experiences. We are particularly interested in freelancers with experience in LLM evaluation, RLHF, AI response rating, data annotation, content moderation, GenAI projects, or similar AI training and evaluation work. Applicants must be based in the United Kingdom and possess excellent UK English proficiency. Prior experience with AI evaluation projects is highly preferred. This is an urgent requirement with immediate onboarding for qualified candidates. *Payment for ...

...manually researched and validated leads ________________________________________ Preferred Decision-Makers • CEO • Founder • COO / Operations Head • Vendor Manager • Localization Manager • HR Head • Business Development Head • AI Program Manager • Language Operations Lead ________________________________________ Target Industrial Domains 1. AI Annotation Companies 2. LLM / Generative AI Companies 3. RLHF & Human Feedback Companies 4. Multilingual AI Companies 5. Speech AI & Transcription AI Companies 6. AI Localization & Linguistic QA Companies 7. Content Moderation AI Companies 8. E-learning / EdTech Companies 9. Media Localization / Subtitling / Dubbing Companies 10. Gaming Localization Companies 11. Translation & Loc...

...Bonus Qualifications: * Experience with RLHF or AI model evaluation * Familiarity with Handshake AI, Scale AI, Outlier AI, DataAnnotation, Surge AI, or similar platforms * Experience evaluating AI-generated text, code, images, presentations, or structured data * Familiarity with tools such as Figma, Excel, PowerPoint, GitHub, VS Code, Jupyter, or AI workflows Important: This role is focused on quality evaluation, usability, aesthetics, structure, and consistency. Candidates should be comfortable making nuanced judgment calls using detailed rubrics and calibration systems. If interested, please send: 1. Resume or LinkedIn 2. Relevant specialization(s) 3. Examples of previous work or evaluation experience 4. Any experience with AI annotation, RLHF, AI evaluation, or data...

...Bonus Qualifications: * Experience with RLHF or AI model evaluation * Familiarity with Handshake AI, Scale AI, Outlier AI, DataAnnotation, Surge AI, or similar platforms * Experience evaluating AI-generated text, code, images, presentations, or structured data * Familiarity with tools such as Figma, Excel, PowerPoint, GitHub, VS Code, Jupyter, or AI workflows Important: This role is focused on quality evaluation, usability, aesthetics, structure, and consistency. Candidates should be comfortable making nuanced judgment calls using detailed rubrics and calibration systems. If interested, please send: 1. Resume or LinkedIn 2. Relevant specialization(s) 3. Examples of previous work or evaluation experience 4. Any experience with AI annotation, RLHF, AI evaluation, or data...

...* Ability to evaluate outputs objectively using rubrics * Strong written English communication * Comfortable reviewing AI-generated artifacts * Ability to explain WHY one response is stronger than another * Familiarity with modern software, design, or AI workflows depending on specialization Preferred Qualifications: * Familiarity with Handshake AI or AI evaluation platforms * Experience with RLHF or AI model evaluation * Experience with design QA, engineering QA, or AI review workflows * Familiarity with tools such as Figma, PowerPoint, Excel, GitHub, VS Code, Jupyter, or multimodal AI systems Important Notes: * This role focuses on quality, usability, aesthetics, structure, and editability * Evaluators should NOT bias scoring based on assumptions about the hidden model/sour...

...aesthetically * Ability to explain WHY one design is stronger than another * Strong written English communication * Comfortable reviewing AI-generated artifacts and following evaluation rubrics * Familiarity with Microsoft Office for inspecting editability and formatting quality Bonus Qualifications: * Familiarity with Handshake AI or AI evaluation platforms * Experience with AI model evaluation or RLHF workflows * Experience with presentation QA or design QA * Familiarity with modern product/design systems from companies such as Apple, Stripe, Linear, Framer, Notion, Airbnb, etc. * Experience with dashboards, enterprise reporting, or data visualization systems Important Notes: * This role focuses ONLY on aesthetics, usability, and editability * Evaluators do NOT primarily fa...

...Strong analytical and critical thinking skills Ability to follow detailed instructions carefully Experience using AI tools such as ChatGPT, Claude, Gemini, or similar Strong internet research skills High attention to detail Ability to work independently and meet deadlines Preferred Qualifications Python or SQL experience Data science or computer science background Experience with AI annotation or RLHF projects Experience with prompt engineering Familiarity with machine learning concepts Previous experience on AI evaluation platforms is a plus Ideal Candidate We are looking for someone who is: Highly intelligent and fast-learning Organized and dependable Comfortable working with AI systems Able to think critically and logically Interested in artificial intelligence and emerging ...

...across backend, infrastructure, and AI layers Required Skills Strong Python + production experience Hands-on with LLMs (OpenAI, Anthropic, etc.) Experience with: RAG systems & embeddings Vector databases (FAISS, Pinecone, Weaviate, Milvus) LangChain / LangGraph / LlamaIndex Solid understanding of: APIs & microservices Distributed systems Nice to Have Multi-agent systems experience Fine-tuning / RLHF knowledge Real-time / async systems Evaluation metrics (NDCG, BLEU, etc.) Docker, Kubernetes, CI/CD Search / recommendation systems background What We’re Looking For Someone who thinks in systems, not just models Builder mindset — able to go from concept → production Focus on real-world execution, not just theory Comfortable working in a fast-moving startu...

...resources for remote AI model evaluation Technical Specifications Required Skills: -2+ years of Software Development or DevOps experience -Proficiency in Python and experience with large-scale frameworks like Django -Linux Terminal experience -Extensive experience reading stack traces and navigating unfamiliar, "messy" codebases to locate root causes Preferred Skills: -Experience in Tool-Calling -RLHF (Reinforcement Learning from Human Feedback) experience -Code Quality Assurance background What We're Providing: Two sample files for annotation High-level guidelines document: SWE-bench Expert Annotation Guidelines (attached). Sample Zip Folder (attached) While the guidelines are provided as a reference, we encourage you to apply your own judgment and domain exper...

I’m running an RLHF pipeline and need a sharp, data-driven review of the training code to understand exactly where time and compute are being lost. The sole focus is algorithm efficiency during the model-training stage; everything else in the codebase is stable for now. By surfacing and fixing the slow spots we should see cleaner gradients, faster convergence, and ultimately better decision-making accuracy from the model. What I’ll hand over • A self-contained repository (Python, PyTorch or other language) with reward model, PPO loop, and evaluation scripts • A brief outline of the current hardware limits and expected throughput What I expect back • A profiled breakdown highlighting hotspots in the training loop, dataloaders, and reward computation...

...diverse domains, ensuring alignment with predefined criteria. Develop comprehensive explanations and rationales for evaluations, showcasing excellent reasoning and technical expertise. Lead efforts in Supervised Fine-Tuning (SFT), including creating and maintaining high-quality, task-specific datasets. Collaborate with researchers and annotators to execute Reinforcement Learning with Human Feedback (RLHF) and refine reward models. Design innovative evaluation strategies and processes to improve the model's alignment with user needs and ethical guidelines. Create and refine optimal responses to improve AI performance, emphasizing clarity, relevance, and technical accuracy. Conduct thorough peer reviews of code and documentation, providing constructive feedback and identifying...

...Requirements Git repo: Candidate should have: Strong Python experience Familiarity with ML workflows, datasets, or model evaluation Knowledge of data parsing, grading logic, or tool-building Ability to write clean, well-documented code Ability to produce deliverables fast Having experience with LLM evaluation, SWE-bench style tasks, RLHF, or prompt engineering is a plus. Deliverables A GitHub repo or drive link containing: Task prompt Grading script Any dataset, tool, or helper function needed Documentation: How to run the task Why it is challenging Expected failure/success modes At least 10 test runs with model outputs + pass rate analysis...

...You’re curious and adversarial: you instinctively push systems to breaking points - You’re structured: you use frameworks or benchmarks, not just random hacks - You’re communicative: you explain risks clearly to technical and non-technical stakeholders - You’re adaptable: thrive on moving across projects and customers Nice-to-Have Specialties - Adversarial ML: jailbreak datasets, prompt injection, RLHF/DPO attacks, model - extraction - Cybersecurity: penetration testing, exploit development, reverse engineering - Socio-technical risk: harassment/disinfo probing, abuse analysis - Creative probing: psychology, acting, writing for unconventional adversarial thinking What Success Looks Like - You uncover vulnerabilities automated tests miss - You deliver rep...

Robust Speech-to-Text Pipeline for Sales Agent Training Goal: Build a robust speech-to-text pipeline us...annotations (summaries, sales techniques) for richer context. Step 6: Analyzing patterns – what works in sales calls? Analyze which phrases and features correlate with success. Use sentiment trajectories and feature importance. Visualize results. Improvement: Use LLMs (GPT-4/5) to generate insights in natural language. Can be applied in RLHF. Step 7: Training and fine-tuning the automated sales agent Use supervised fine-tuning on an LLM with dialogue data. Complement with RLHF to optimize response strategies. Simulate conversations and include human feedback. Improvement: Apply insights from Step 6 to shape training goals and generate synthetic conversations to st...

...including Docker containers for environment isolation and reproducibility. - Excellent written communication and documentation skills. - Experience working with structured QA or annotation workflows. - English proficiency at B2, C1, C2, or Native level. Preferred Qualifications - Experience in AI training, LLM evaluation, or model alignment. - Familiarity with annotation platforms. - Exposure to RLHF (Reinforcement Learning from Human Feedback) pipelines. - Working knowledge of Docker for replicating or reviewing complex code environments is a plus. Why Join Us? Join a high-impact team working at the intersection of AI and software development. Your Python expertise will directly influence the accuracy, safety, and clarity of AI-generated code. This role offers remote flexibil...

...Propose and implement architectural improvements to the custom VAE-HMM-Transformer backbone and other components of the Cerebrum Optionally: help design better rule-based or learned reward functions Required Skills Strong experience with PyTorch and transformer architectures Deep understanding of variational models, autoregressive decoding, and sequence generation Experience with reward modeling, RLHF, or reinforcement-style fine-tuning Comfortable reading and modifying research-style codebases Bonus If You Have Familiarity with probabilistic graphical models or HMMs Experience tuning custom tokenizers, loss functions, or LLM adapters Contributed to open-source ML or LLM frameworks If you are interested please look over the Architecture first, then send

...Propose and implement architectural improvements to the custom VAE-HMM-Transformer backbone and other components of the Cerebrum Optionally: help design better rule-based or learned reward functions Required Skills Strong experience with PyTorch and transformer architectures Deep understanding of variational models, autoregressive decoding, and sequence generation Experience with reward modeling, RLHF, or reinforcement-style fine-tuning Comfortable reading and modifying research-style codebases Bonus If You Have Familiarity with probabilistic graphical models or HMMs Experience tuning custom tokenizers, loss functions, or LLM adapters Contributed to open-source ML or LLM frameworks If you are interested please look over the Architecture first, then send

...at rest and in transit Training Methods Base Model Selection: Start with the Llama 4 model variant (70B parameters ideally) Use the chat-tuned version as the foundation as it's already optimized for dialogue Training Techniques: Supervised Fine-Tuning (SFT): Initial training on your PCB dataset LoRA/QLoRA: Low-rank adaptation to efficiently train while preserving most base model knowledge RLHF: Reinforcement Learning from Human Feedback with PCB domain experts Context window utilization: Train with full examples showing input-to-output flow Training Data Requirements You'll need several types of datasets: 1. PCB Design Corpus (500K examples) KiCad project files: Thousands of open-source PCB designs Circuit schematics: With component lists and connections Design ...

...enhance foundational LLM models for ServiceNow scripting languages, focusing on improving co-pilot features for accurate and context-aware automation support. Conduct regular evaluations and feedback loops to refine LLM-based integrations, ensuring alignment with business objectives. Fine-tune models using techniques such as Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) to optimize ServiceNow’s scripting capabilities, including JavaScript, GlideScript, and advanced querying techniques like GlideQuery and Dot-walking. Develop and implement automated processes using ServiceNow's scripting languages (JavaScript, GlideScript, etc.), leveraging advanced techniques for data manipulation and workflow automation. Optimize existing scripts a...

I'm in need of skilled AI data annotators and Reinforcement Learning with Human Feedback (RLHF) contributors for a project involving the annotation of text data from scientific articles. Your tasks will include: - Labeling text content - Identifying entities - Classifying sentiments Ideal candidates should have: - Experience in data annotation, particularly with text - Familiarity with scientific articles - Skills in sentiment analysis and entity recognition Your expertise will help me train a robust AI model.

...storytelling. This AI should be future-proof, incorporating the latest AI research, including: • Fine-tuned LLMs (LLaMA 3, Mixtral 8x7B, DeepSeek) • Retrieval-Augmented Generation (RAG) • Vector Search (ChromaDB, FAISS, Weaviate) • Semantic Search for Contextual Relevance • Multi-Agent AI (Self-Improving AI) • Graph-Based Memory Systems (Neo4j, LangGraph) • Reinforcement Learning with Human Feedback (RLHF) • LoftQ for Low-Cost Fine-Tuning • FlashAttention-2 for Efficient Model Training • RWKV for Adaptive Long-Form Writing • Graph Databases for Deep Storyline Memory • Multi-Agent AI with Adaptive Self-Improvement • ReAct Prompting for Dynamic Decision-Making • In-Context Learning (ICL) for Adaptive Style Matchi...

...developing large language models and implementing advanced techniques such as retrieval-augmented generation and reinforcement learning from human feedback. The main goal of this project is to create a sophisticated conversational AI and content generation tool. The scope of the work is to develop an LLM application using an open-LLM integrated with a Reinforcement Learning from Human Feedback (RLHF) mechanism to improve performance. The expected level of implementation is that one of a to Minimum Value Product. Since the LLM must be specialized on a specific industrial sector, it is necessary that it is trained on a set of additional documentation, for example using a RAG (Retrieval-Augmented Generation) mechanism. Consider that I have approximately 10,000 documents in pdf and...

1. Hi, I'm looking for a Ph.D. level (or equivalent) person with an understanding of math, AI, or ML. It would be great if we could read AI, ML, diffusion, RLHF, and reinforcement learning related academic papers together during [Removed by Freelancer.com Admin for offsiting - please see Section 13 of our Terms and Conditions]. 2. Another important thing is that I would like to have the meeting at 7pm EST. If you are from Europe or Pakistan, that time may NOT work for you. We will have a [Removed by Freelancer.com Admin for offsiting - please see Section 13 of our Terms and Conditions] chat for two hours on weekdays, for example. 3. If you read until here, please include links to some papers that you wrote in the related fields. Please include the link in the first sentence ...

I am looking for a skilled developer to create an LLM-based recommendation engine. The ideal candidate should have experience in generative AI and be able to implement a real-time LLM based recommendation system to generate personalized shopping feeds ba...recommendation system to generate personalized shopping feeds based on a given user's profile. The specific requirements for this project include: 1. Evaluate and select an appropriate LLM model to deploy as the foundation model for our recommendation system 2. Deploy the selected model into production to generate recommendation feed 3. Create a workflow for fine-tuning / improving the model outputs using RLHF techniques The task is very urgent and we require someone with the bandwidth to focus on a rapid deployment of an...

...experience are up-to-date to minimise the risk of this project going wrong or taking more time than it should. * Scope * 1. Guide the CEO how to fine-tune LLMs Work with us step-by-step on live video calls with screen sharing throughout the process. Educate us on what you've learned so far and guide on us what is good/not good based on our needs We will try three different ways: PEFT + LoRA QUORA RLHF 2. Help us understand how to prep our own datasets to enable fine-tuning 3. Provide training and video walkthroughs of any areas that we cannot cover on the live calls together (where requested) * Out of Scope 1. Any development work - we have the solutions already 2. Documentations - unless you are being proactive in providing exceptional value * Expected timeline: 1...

...salga mal o tome más tiempo del necesario. * Alcance * 1. Guíe al CEO sobre cómo ajustar finamente LLMs. Trabaje con nosotros paso a paso en llamadas de video en vivo con uso compartido de pantalla durante todo el proceso. Eduque sobre lo que ha aprendido hasta ahora y guíenos sobre lo que es bueno/no es bueno según nuestras necesidades. Probaremos tres formas diferentes: PEFT + LoRA QUORA RLHF údenos a comprender cómo preparar nuestros propios conjuntos de datos para habilitar el ajuste fino. capacitación y explicaciones en video de áreas que no podamos cubrir en las llamadas en vivo juntos (si se solicita). *Fuera del alcance * trabajo de desarrollo, ya que ya tenemos las soluciones. ón, a menos qu...

Goal: Build a closed-book QA generative model to answer astrology questions. It should be built so that it only answers from the data it was fine-tuned and not any previous knowledge(like from Wikipedia). Dataset Available: About 70 pdfs (some contain Sanskrit which needs to be filtered out). Transc...not any previous knowledge(like from Wikipedia). Dataset Available: About 70 pdfs (some contain Sanskrit which needs to be filtered out). Transcripts of about 1000 videos from an astrology channel(can skip these if pdfs are sufficient). We can either use GPT-3 models(provided it isn't too costly) or any open-source alternative like GPT-J or Flan-t5. It'll be good if it can be integrated with RLHF(Reinforcement Learning with Human Feedback) to get improvements over time. ...

...salga mal o tome más tiempo del necesario. * Alcance * 1. Guíe al CEO sobre cómo ajustar finamente LLMs. Trabaje con nosotros paso a paso en llamadas de video en vivo con uso compartido de pantalla durante todo el proceso. Eduque sobre lo que ha aprendido hasta ahora y guíenos sobre lo que es bueno/no es bueno según nuestras necesidades. Probaremos tres formas diferentes: PEFT + LoRA QUORA RLHF údenos a comprender cómo preparar nuestros propios conjuntos de datos para habilitar el ajuste fino. capacitación y explicaciones en video de áreas que no podamos cubrir en las llamadas en vivo juntos (si se solicita). *Fuera del alcance * trabajo de desarrollo, ya que ya tenemos las soluciones. ón, a menos qu...