Researchers have published a new AI debate protocol called FBS Debate designed to help humans oversee complex AI outputs by breaking them into simpler, verifiable claims, according to a paper on arXiv.
The protocol addresses "scalable oversight," the challenge of providing accurate training supervision as AI systems surpass human expert abilities in cognitively demanding tasks such as autonomous software engineering or mathematical research, the paper states. AI debate works by having two AI systems argue over an output's correctness while a human judge evaluates the interaction. Previous theoretical work struggled with the "obfuscated arguments problem," where a debater produces a dishonest argument the opponent cannot efficiently refute arXiv.
FBS Debate targets problems that allow stable decompositions into subproblems. The researchers said the new protocol improves upon the previous best protocol, the prover-estimator debate, in three areas: correctness holds in a worst-case sense rather than an average-case sense; being honest is a dominant-strategy equilibrium for both debaters; and the protocol is instance-wise optimal arXiv. The authors related stable problem decompositions to "fractional block sensitivity," a concept from query complexity, to determine if a decomposition is too sensitive to small errors in sub-claims arXiv.
In the protocol, a first debater, Alice, decomposes a claim into sub-claims. The second debater, Bob, chooses between demonstrating that the decomposition is too sensitive to small errors or selecting a sub-claim to continue the debate recursively arXiv. The researchers proved that no other protocol for this class of problems can outperform FBS Debate if it treats human judgment as a "black-box" oracle. The paper states this result shows the necessity of moving beyond the black-box model to further expand the types of problems solvable via AI debate arXiv.