FLARE-AI: Crowdsourced site to report AI harms and flaws

FLARE-AI, a crowdsourced website for reporting and tracking AI harms, has been set up by a group of AI researchers to let users sound the alarm when models generate malware, leak personal data, or trigger delusional thinking in users. The project is open source and was co-led by Avijit Ghosh of HuggingFace with computer scientists Elaine Zhu and Shayne Longpre, developed in collaboration with 49 AI experts from 32 organizations.

What is FLARE-AI and how does it work?

FLARE-AI is a public, crowdsourced platform whose open source code allows others to verify reported issues and route reports to model makers and organizations such as MITRE, functioning in a similar way to Downdetector for service outages. Users can submit incidents—examples in the project description include chatbots generating malware or bomb-making recipes, leaking personal information, or producing content that encourages delusional thinking—and the platform records and tracks those reports.

Behind the interface the code is meant to make verification possible, so third parties can confirm whether a reported flaw reproduces and then forward the report to relevant stakeholders. The designers position the site as a coordination layer for fragmented disclosure practices across companies and projects.

Why do researchers say a centralized reporting system is needed?

Researchers argue the absence of a consistent reporting channel leaves many flaws unrecognized, and the project team wrote that a coordinated system could be crucial as agentic systems gain more power. "Right now, there is no centralized, accountable way to report flaws in AI systems," says Avijit Ghosh, an AI policy researcher at HuggingFace who co-led FLARE-AI.

The need is illustrated by recent incidents involving popular tools. Security firm LayerX disclosed a method to dupe AI-infused web browsers including OpenAI's Atlas and Perplexity's Comet into bypassing guardrails; LayerX said the affected companies fixed the issue. In April, security researcher Johann Rehberger discovered a way to trick Claude into divulging personal data using images generated by ChatGTP. Developers have also had to patch behavioral problems: OpenAI updated models after finding they were overly sycophantic, which sometimes appeared to encourage delusional thinking.

The FLARE-AI team also consulted on a congressional bill announced in June that would put the US government in a central role: Representatives Deborah Ross, Jeff Hurd, and Don Beyer introduced legislation that would require the National Institute of Standards and Technology to develop standards around AI flaw reporting and to maintain a centralized AI flaw reporting database. The researchers say such steps would incentivize developers to address issues and help users compare system safety across use cases.

Why it matters

A public reporting system could create pressure for greater transparency and give external parties the tools to verify and escalate safety problems, making it harder for serious flaws to remain private. The platform's open source design aims to reduce the black box problem by enabling reproducibility of reports and routing to actors that can act on them.

The flip side is operational: experts warn the system will face a flood of low-quality reports and needs credible authorities to adjudicate priorities. Rumman Chowdhury and others note that scaling a trustworthy triage process and securing buy-in from authoritative organizations will determine whether the tool produces useful safety improvements or noise.

What to watch

Watch whether NIST begins work on standards and a centralized database if the congressional bill advances, and whether major model makers accept routed reports from community platforms like FLARE-AI. Also monitor incidents involving agentic systems such as OpenClaw and other models capable of probing or hacking computer systems; the researchers flagged these as the types of tools with greater potential to cause harm.

FLARE-AI aims to convert scattered user complaints into verifiable disclosures, and the next signals of success will be uptake by model makers, integration with organizations such as MITRE, and the emergence of a credible triage process backed by standards.