Recent adaptive radiations are models for investigating mechanisms contributing to the evolution of biodiversity. An unresolved question is the relative importance of new mutations, ancestral variants, and introgressive hybridization for phenotypic evolution and speciation. Here, we address this issue using Darwin's finches and investigate the genomic architecture underlying their phenotypic diversity. Admixture mapping for beak and body size in the small, medium, and large ground finches revealed 28 loci showing strong genetic differentiation. These loci represent ancestral haplotype blocks with origins predating speciation events during the Darwin's finch radiation. Genes expressed in the developing beak are overrepresented in these genomic regions. Ancestral haplotypes constitute genetic modules for selection and act as key determinants of the unusual phenotypic diversity of Darwin's finches. Such ancestral haplotype blocks can be critical for how species adapt to environmental variability and change.