Field Trials
Field Trials
Word of Lore AI Field Trials are tournament-style, one-on-one competitions between various AI tools solving real-world problems. When planning future trials, we consider all available industry tools capable of solving specific use cases. Each trial is structured around a single use case, and only tools and workflows capable of addressing that particular challenge are enrolled.
Nominations are open to members. Comment on the trial nomination page.
Ongoing Field Trials
The proceeding rating period ends on November 30, 2024.
Field Trials for your favourite AI vendors
Field Trials for your favourite AI vendors
Cohere
OpenAI
you.com
Perplexity
including open-source/-weights models from
Meta
Mistral AI
As an independent publication, Word of Lore.ai facilitates fair, real-world, and unbiased field trials and is not affiliated with any particular AI tech vendor.
Trial Face-Offs
Trial Face-Offs
Trials have a one-on-one format where AI tools or workflows compete on a variety of practical tasks. We determine winners based on performance. Results are published in the face-offs section along with the trial card.
Leaderboard and Ratings
Leaderboard and Ratings
The outcome of AI trials is compiled and reflected in AI ratings. These trials are designed for AIs to accumulate ratings over time. When one AI outperforms another, we adjust ratings accordingly:
- The prevailing AI gains rating points.
- The defeated AI loses rating points.
This rating carries over to subsequent trials and determines future face-offs using the Swiss pairing system (a method that matches competitors based on similar performance records).
Insights
Insights
Coming soon: We will share the best and most useful content with our community, including:
- Top-performing AI workflows described with easy-to-follow instructions
- Updates on AI tools that impact how users apply existing workflows
- Insightful stories from leading AI practitioners
Why it matters
Why it matters
Field Trials
The tech industry often suffers from excessive marketing and hype. More importantly, there's a lack of clarity on how to apply AI in business or personal life. Our approach addresses these issues. We design field trials for real-world, useful use cases. These trials involve hand-crafted evaluation datasets and meticulous execution for reliable results. Our team combines editorial expertise with data science knowledge to provide valuable insights.
Face-Offs
Our trial execution process involves careful scrutiny of workflow pairings and execution methods. We manage everything from pairing nominees and scheduling face-offs to ensuring consistent execution and publishing results. This comprehensive effort culminates in a trial card that clearly demonstrates how different AI tools compare in direct competition.
Ratings
Ratings form the foundation of any competition, representing a competitor's performance. This principle applies equally to our AI trials. We have carefully selected a rating system designed to converge toward an accurate representation of each AI tool's true capabilities.
Leaderboard
Top-performing AI tools earn their place at the summit of our leaderboard. This ranking system allows our community to identify and build confidence in the AI solutions that excel in areas of greatest importance.
Insights (Coming Soon)
Insights will feature top-performing workflows, practitioners' stories, and practical news from the AI industry. We curate content based on its usefulness and applicability, ensuring you receive valuable information without unnecessary noise.
Newsletters
Subscribers gain priority access to all the benefits mentioned above, with some content available exclusively to our newsletter audience.
Empowering Your AI Journey
Empowering Your AI Journey
Join Our Community of AI Enthusiasts and unlock AI's potential in your work, without the confusion.
Free Membership
Word of Lore members enjoy several complimentary benefits. These include newsletters featuring new trial releases and AI insights, participation in community discussions, and the ability to nominate field trials. Sign up for free to become a member and access these perks.