Our public plan for evaluating frontier proprietary models and open-weight models, building local and cloud testing capacity, and publishing repeatable safety results.
We want a model testing program that is technically credible, reproducible, and public-facing. That means testing both proprietary and open-weight systems across the same safety lenses, documenting the methodology, and publishing the results freely.
Too much frontier-model discussion still depends on vendor claims, one-off anecdotes, or unpublished internal evaluations. Our goal is to add independent evidence: repeatable benchmark runs, failure-case documentation, and public reporting that compares models across safety, reliability, and agentic risk dimensions.
Our planned test tracks include jailbreak resistance, prompt injection, harmful-output refusal, factual grounding, bias and fairness, honesty and deception, agentic tool-use boundaries, and corrigibility under pressure.
Current model list below is current as of June 23, 2026 and will evolve as vendors ship new versions.
We are approaching this in three phases so the testing program grows in a disciplined way instead of trying to do everything at once.
Acquire an NVIDIA DGX Spark as our first dedicated local model-evaluation workstation. NVIDIA positions DGX Spark as a desk-side platform for prototyping, fine-tuning, and inference, with 128 GB of unified memory, fine-tuning support up to 70B-parameter models, and inference/testing support up to 200B-parameter models.
Lease GPU capacity from cloud providers for larger open-weight models, longer benchmark sweeps, multimodel comparison runs, and ablation studies that exceed our local lab footprint.
Accept in-kind donations of current and prior-generation GPU or inference cards with 24 GB or greater VRAM so we can build out a broader local testing cluster over time.
These are planning targets for the first wave of evaluation infrastructure and research capacity. They are meant to fund capability, not excess: enough to build a credible independent testing workflow, staff core evaluation work, and keep the program running.
Target budget for an NVIDIA DGX Spark workstation plus shipping, tax, external storage, cables, and setup overhead. This gives us a dedicated local platform for repeatable open-weight testing and small-scale fine-tuning experiments.
Initial fund for burst testing on rented GPU infrastructure, including larger open-weight models, repeated red-team runs, long-context sweeps, and reproducibility checks across providers.
Integration budget for donated GPUs and inference cards, including host hardware, power, cooling, networking, replacement parts, and configuration work needed to make donated equipment usable in practice.
Dedicated support for researcher time spent designing evaluations, running benchmark campaigns, reviewing failure cases, documenting methodology, and publishing clear public reports. Hardware alone does not create an independent testing program; people do.
We plan to test a mix of frontier API models and open-weight models. The goal is not to pick winners, but to compare safety behavior across different architectures, access models, and deployment environments.
These give us coverage across flagship, mid-tier, and lower-cost variants that many developers actually deploy.
The local lab is mainly for repeatable open-weight work; cloud GPUs let us scale that work to larger or more demanding checkpoints.
Model descriptions and hardware capability references on this page were checked against vendor documentation on June 23, 2026, including OpenAI, Anthropic, Google, Meta, Mistral, and NVIDIA.
If you want to help us build this testing program, financial contributions and in-kind hardware support both make a meaningful difference.
General support helps us acquire testing hardware, pay for cloud GPU time, and keep methodology and reporting public.
Support The RoadmapWe are actively open to donated current or prior-generation GPU and inference cards with 24 GB or greater VRAM, along with supporting workstation or networking gear.
Offer HardwareWe welcome collaboration from researchers, model developers, safety teams, and infrastructure partners who want more rigorous public model evaluations.
Get In Touch