Finite failure modes
Sam: if failure modes for a given task aren't infinite, maybe you can keep optimising until the code resists every edge case you'll realistically meet, which cuts against his usual argument that the physical world is infinitely diverse. He was surprised that both direct control and code worked so well out of the box.
Why a failure might not be understood: the cause isn't observed (sim gives perfect state; reality doesn't show you friction or slip); there's no single cause (it emerges from many small interactions across many joints); or the fix has no compact form (it's a pile of numbers, which is what a policy is). On finiteness: bounded cells shorten the long tail, which is why industrial code already works. The risks are the tail itself (self-driving went from rules to learned models for this reason) and patches piling up and interfering as the code grows.
Connections
Parent: Engineer in the loop
