Amnesia is a dramatic clinical syndrome caused by diverse pathologies and lesion localizations. Although amnesia is typically screened for by clinicians using verbal stimuli, amnestic syndromes have been described that do not impair verbal memory and may be confined to a single sensory system or a dominant or highly lateralized sensory function. Thus, the functional-anatomic basis for various types of amnestic disorders is complex and, in most instances, better understood as a disconnection syndrome rather than a primary processing deficit. Using the clinical disorder of sensory-specific visual amnesia in humans as a springboard, a hodological model for understanding the various types of amnestic syndromes encountered in the clinic and those produced by discrete experimental lesions in monkeys is offered. The model is then expanded to encompass memory functions, in general, including agnostic deficits and the role of prefrontal cortex in learning and remembering.