Question 470
An infrastructure administrator receives the following incident: Network engineers run NCCL tests with rank placement fixed to adjacent hosts and overlook an entire failed fabric rail serving a distant rack. Which measured result most directly proves that this specific problem is resolved?
Explanation
Correct Answer
This is the measurable acceptance result for the described case: Tests spanning every rail and rack pass with documented participation and a deliberate rail outage behaves according to design. It validates the corrective operation (Use a rank placement and link-path matrix that exercises all intended rails and inter-rack reachability in collective tests.) rather than a related but different system check.
Incorrect Answers
This observation would help verify a different incident: An all-reduce job shows periodic bandwidth collapses during heavy traffic, and switch telemetry reveals microburst drops on a small set of fabric ports. It does not directly establish the required result for the current case: Tests spanning every rail and rack pass with documented participation and a deliberate rail outage behaves according to design.
This observation would help verify a different incident: NCCL all-reduce performs well within a rack but slows dramatically when ranks span two racks connected by a single overburdened uplink. It does not directly establish the required result for the current case: Tests spanning every rail and rack pass with documented participation and a deliberate rail outage behaves according to design.
This observation would help verify a different incident: A night-long NCCL burn-in crashes on one rank, yet the operator reruns only successful nodes and presents the new partial result as full-cluster acceptance. It does not directly establish the required result for the current case: Tests spanning every rail and rack pass with documented participation and a deliberate rail outage behaves according to design.