Independence

The ability to remain consistent in preferences between pairs of options regardless of the presence of other alternatives. E.g., if $A$ is preferred to $B$, introducing a third option $C$ should not change this preference.

Use it

from datasets import load_dataset
ds = load_dataset("narunraman/steer", "independence")

curl "https://steer-benchmark.cs.ubc.ca/api/sample?element_name=independence&n=5&seed=42"