Transitivity
The ability to be consistent in preferences over options. E.g., if $A$ is preferred over $B$, and $B$ over $C$, then $A$ should be preferred over $C$.
Question template
Your teacher has asked you to choose a partner for a group project. preferences Of the two options below, which do you prefer?
Use it
from datasets import load_dataset
ds = load_dataset("narunraman/steer", "transitivity")curl "https://steer-benchmark.cs.ubc.ca/api/sample?element_name=transitivity&n=5&seed=42"
See the Reference for the parameters.