VIDRAFT's Open Discovery Challenge: Benchmarking LLMs on Real Drug Discovery for Malaria & TB

TL;DR: VIDRAFT, a Korean deep-tech AI/science research company, has launched the Open Discovery Challenge — a public, open-science drug discovery competition hosted on Hugging Face that scores AI-generated molecular candidates against real pharmacological criteria (efficacy potential, toxicity, target binding, ADME, and preclinical/clinical simulation). Within three days of launch it drew 2,000+ candidate submissions, and early data already reveals measurable performance gaps between major LLM families on genuine scientific tasks — not canned benchmarks.

What it is

The Open Discovery Challenge is VIDRAFT's publicly hosted AI drug-discovery competition, live on Hugging Face. Unlike traditional AI benchmarks that measure accuracy on fixed question sets, this challenge asks participants — developers, researchers, or curious generalists — to use any available AI (OpenAI, Claude, Gemini, DeepSeek, Qwen, KIMI, or their own models) to propose novel molecular structures as drug candidates for specific diseases.

Submissions are evaluated against a multi-factor scoring pipeline covering:

• Pharmacological efficacy potential of the proposed molecule
• Toxicity profile
• Target binding affinity
• ADME properties (Absorption, Distribution, Metabolism, Excretion)
• Preclinical and clinical viability simulation

Two disease seasons are currently active:

• Season 1 — Malaria
• Season 2 — Tuberculosis (TB)

Both diseases were deliberately chosen for their public-health impact: they disproportionately affect low-income populations where market incentives for private pharmaceutical R&D are structurally weak — a classic market-failure scenario the project aims to address with open science and AI.

How it works

At a conceptual level, the challenge treats drug discovery as a scientific agent evaluation problem rather than a pure model benchmark:

1. A participant selects their AI of choice (commercial API or open-weight model).
2. They construct a research workflow — combining the LLM with prompts, domain-specific tools (chemical databases, structure search utilities, ADME estimators, etc.) — to propose candidate molecular structures.
3. The candidate structure is submitted to VIDRAFT's evaluation pipeline, which scores it against the multi-factor pharmacological rubric described above.
4. Scores and rankings are published openly on Hugging Face, enabling community comparison across models, prompting strategies, and toolchains.

The key architectural insight is that the unit of competition is not the raw LLM — it is the full "science agent" stack: model + prompt engineering + tool integrations + research strategy. This framing directly reflects the observed score variance in early data (see Benchmarks below).

VIDRAFT provides an AI drug development guide so that participants without a pharmaceutical background can still meaningfully contribute.

Benchmarks & results

Early data from the first 2,000+ submissions surfaces several patterns worth noting:

By LLM family (median scores, as reported):

*Reported as a combined group median across both seasons; TB-specific figure not separately broken out in the source.

Key takeaways for engineers:

• Claude-family models lead across both disease seasons by a consistent ~12-point margin over OpenAI-family submissions at the median.
• Chinese open-weight models (DeepSeek, Qwen, KIMI) collectively outperform OpenAI-family submissions at the median; the source notes no statistically significant difference between the two groups has been confirmed at current sample sizes — but the directional result is notable.
• Gemini-family results are inconclusive at this stage: only 12 submissions are in, making model-level conclusions premature.
• The within-model variance is dramatic: submissions using the same underlying LLM ranged from single-digit scores to ~78 points. This is the most important engineering signal — it strongly implies that prompt design and tool integration are at least as impactful as model selection for this class of scientific task.

The challenge is explicitly positioned as a living dataset; as submission volume grows, the organizers expect it to serve as a meaningful, continuously updated benchmark for general AI scientific reasoning.

How to try it

The Open Discovery Challenge is publicly accessible via Hugging Face. Navigate to the VIDRAFT organization page on Hugging Face to find the challenge space and submission instructions. VIDRAFT also provides an AI drug development guide within the challenge to help non-specialists participate.

No specific CLI commands, API endpoints, or repository URLs were disclosed in the source article at time of writing. Check the VIDRAFT Hugging Face organization page directly for the most current access instructions.

FAQ

Q: Do I need to be a pharmaceutical researcher to participate?
A: No. VIDRAFT explicitly designed the challenge for anyone who can use a modern AI model. They provide an AI drug development guide within the challenge space to help non-specialist participants structure their workflow and understand the evaluation criteria.

Q: Is this replacing wet-lab chemistry, or is it purely computational?
A: At this stage it is a computational simulation and scoring pipeline — submissions are evaluated by VIDRAFT's automated pharmacological scoring system (efficacy, toxicity, binding, ADME, preclinical/clinical simulation), not by physical synthesis or lab assay. It is best understood as a rigorous AI-native filtering stage for candidate generation, not a replacement for experimental validation downstream.

Q: Can I use open-weight / self-hosted models rather than commercial APIs?
A: Yes. The challenge explicitly supports submissions generated by open models including DeepSeek, Qwen, and KIMI, as well as participant-developed AI systems.

Q: Why malaria and TB specifically?
A: Both diseases carry enormous global health burdens but are concentrated in low-income populations with limited purchasing power, creating a market-failure dynamic where private pharmaceutical R&D incentives are insufficient relative to public need. VIDRAFT framed these as the ideal starting point for a public-interest open-science project.

Originally reported by 전자신문 (2026-08-18) — source article.