VeriSpeak

VeriSpeak is a spoken-statement factual-verification benchmark. Each example is a short synthesized speech clip of a single declarative sentence about a public figure, labeled correct or incorrect depending on whether the spoken statement is factually true.

The task: given the audio (and optionally its transcript), decide whether the claim it makes is accurate. It targets speech-native fact-checking / hallucination detection.

Dataset at a glance

Examples	3,879
Split	test only (evaluation benchmark)
Label balance	1,973 correct / 1,906 incorrect
Categories	location 2,226 · year 1,451 · relation 202
Unique subjects	659 (each a distinct public figure)
Audio	mono WAV, 22,050 Hz, ~10 s average, ~10.9 h total
Language	English

Data fields

field	type	description
audio	Audio(sampling_rate=22050)	the spoken clip; decodes to {array, sampling_rate}
transcript	string	the exact sentence that was spoken
label	string	"correct" or "incorrect" — is the statement factually accurate
category	string	kind of fact asserted: "location", "year", or "relation"
name	string	the public figure the statement is primarily about
key	string	stable id for the subject entity; shared by every clip about the same person
file_name	string	original filename, formatted {key}_{category}_{label}_{index}.wav

incorrect clips are minimal perturbations of a true statement — swapping the year, the place, or the related entity — so the surface form stays close to the correct version and the label hinges on the fact, not the phrasing (e.g. "…served as Prime Minister of Belgium from 2008 to 2009." → "…from 2017 to 2011.").

Usage

from datasets import load_dataset

ds = load_dataset("abhiram4572/VeriSpeak", split="test")
ex = ds[0]

ex["audio"]        # {'array': array([...]), 'sampling_rate': 22050}
ex["transcript"]   # 'Herman Van Rompuy served as Prime Minister of Belgium from 2008 to 2009.'
ex["category"]     # 'year'
ex["label"]        # 'correct'

Decoding the audio column needs an audio backend: with datasets >= 4.0 install torchcodec, or pin pip install "datasets<4.0" soundfile. The raw bytes are embedded in the Parquet files, so no separate audio download is required.

Filter by category or label

years = ds.filter(lambda r: r["category"] == "year")
wrong = ds.filter(lambda r: r["label"] == "incorrect")

# turn `label` into an integer class column if a model needs it
ds = ds.class_encode_column("label")

Source

VeriSpeak is derived from the publicly available KVQA knowledge base, whose public-figure information is sourced from Wikidata. It does not include private or user-provided personal data.

Ethics & intended use

The benchmark contains synthetic false claims solely for controlled evaluation. Each claim is paired with a veracity label and is clearly identified as benchmark-generated in this documentation and in the dataset metadata. VeriSpeak is intended for evaluating speech-based fact-verification systems and should not be treated as a source of factual claims about individuals.

License

MIT.