signal_tag · 2_broadcasts
#model-evaluation
// 2 transmissions tagged with #model-evaluation
TX_852901· AI
Test cheaper models with open-weight benchmark harness
A benchmark harness evaluates open-weight, closed, and local models on real product tasks, preventing hidden costs when traffic is switched, by computing a cost-per-success metric [Dev.to].
TX_671286· AI
OpenAI and Hugging Face disclose model‑evaluation security breach
OpenAI and Hugging Face released a joint statement detailing a security breach that occurred during model evaluation and outlining mitigation steps for engineers.