Noninvasive diffusion-weighted magnetic resonance imaging (dMRI) can be used to map the neural connectivity between distinct areas in the intact brain, but the standard resolution achieved fundamentally limits the sensitivity of such maps. We investigated the sensitivity and specificity of high-resolution postmortem dMRI and probabilistic tractography in rhesus macaque brains to produce retinotopic maps of the lateral geniculate nucleus (LGN) and extrastriate cortical visual area V5/MT based on their topographic connections with the previously established functional retinotopic map of primary visual cortex (V1). We also replicated the differential connectivity of magnocellular and parvocellular LGN compartments with V1 across visual field positions. Predicted topographic maps based on dMRI data largely matched the established retinotopy of both LGN and V5/MT. Furthermore, tractography based on in vivo dMRI data from the same macaque brains acquired at standard field strength (3T) yielded comparable topographic maps in many cases. We conclude that tractography based on dMRI is sensitive enough to reveal the intrinsic organization of ordered connections between topographically organized neural structures and their resultant functional organization.