There are distinct regions of the brain, reproducible from one person to the next, specialized for processing the most universal forms of human expertise. What is the relationship between behavioral expertise and dedicated brain structures? Do reproducible brain structures mean only certain abilities are innate, or easily learned, or does intensive early experience influence the emergence of expertise and/or dedicated brain circuits? We found that intensive early, but not late, experience influences the formation of category-selective modules in macaque temporal lobe, both for natural stimuli and for stimuli never naturally encountered by monkeys. This suggests that, as in early sensory areas, experience can drive functional segregation and that this segregation may determine how that information is processed. The pattern of novel domain formation in symbol-trained monkeys indicates the existence of a proto-architecture that governs where experience can exert its effects on brain organization. Our most recent work addresses the questions of what that proto-architecture is and what happens if monkeys never see what they would normally develop domains for.