Despite the clinical significance of event-related potential (ERP) latency abnormalities, little attention has focused on the anatomic substrate of latency variability. Volume conduction models do not identify the anatomy responsible for delayed neural transmission between neural sources. To explore the anatomic substrate of ERP latency variability in normal adults using automated measures derived from magnetic resonance imaging (MRI), ERPs were recorded in the visual three-stimulus oddball task in 59 healthy participants. Latencies of the P3a and P3b components were measured at the vertex. Measures of local anatomic size in the brain were estimated from structural MRI, using tissue segmentation and deformation morphometry. A general linear model was fitted relating latency to measures of local anatomic size, covarying for intracranial vault volume. Longer P3b latencies were related to contractions in thalamus extending superiorly into the corpus callosum, white matter (WM) anterior to the central sulcus on the left and right, left temporal WM, the right anterior limb of the internal capsule extending into the lenticular nucleus, and larger cerebrospinal fluid volumes. There was no evidence for a relationship between gray matter (GM) volumes and P3b latency. Longer P3a latencies were related to contractions in left temporal WM, and left parietal GM and WM near the interhemispheric fissure. P3b latency variability is related chiefly to WM, thalamus, and lenticular nucleus, whereas P3a latency variability is not related as strongly to anatomy. These results imply that the WM connectivity between generators influences P3b latency more than the generators themselves do.