An AI algorithm will not sit in a witness box. A person will.
This matters most when decisions are later challenged. If an outcome is questioned, the explanation cannot rest solely on what the system produced. The algorithm will not sit in a witness box. A person will.
That person may point to validated models, risk thresholds, and compliance processes, yet beneath all of that lies a human judgement: the judgement to rely on the system, to accept its output, to refrain from overriding it.