Toloka

Toloka builds training and evaluation data for AI agents and large language models, combining human expertise with tooling rather than selling software a team runs itself. The work splits three ways on the vendor's own description: generating environments for agents to act in, producing training datasets, and running evaluation and red-teaming against finished models. That shape matters for buyers, because Toloka is engaged as a data partner on a scoped programme rather than licensed as a self-serve product.

The agent side is the part the vendor leads with. It covers trajectory demonstrations and step-by-step evaluations across tool-use workflows, plus virtual environments and reinforcement learning gyms built with MCP replicas and computer-use testbeds. Safety work is called out separately, aimed at injection vulnerabilities and policy compliance. Toloka names the agent types it builds data for, including conversational agents, corporate assistants and deep research agents, which is a useful signal of where its expertise is concentrated.

Alongside agent work, Toloka produces creative and multimodal datasets: expert human evaluation and feedback, content collection across text, image, video and audio, and professional annotation with quality filtering. Domain-specific demonstrations and preference data for language and vision language models round out the catalogue. Buyers should expect a consultative engagement, since the vendor does not publish pricing and positions itself as an extension of an in-house research team.