Hybrid inviability

Hybrid inviability is a post-zygotic barrier, which reduces a hybrid's capacity to mature into a healthy, fit adult. The relatively low health of these hybrids relative to pure-breed individuals prevents gene flow between species. Thus, hybrid inviability acts as an isolating mechanism, limiting hybridization and allowing for the differentiation of species.

The barrier of hybrid inviability occurs after mating species overcome pre-zygotic barriers (behavioral, mechanical, etc.) to produce a zygote. The barrier emerges from the cumulative effect of parental genes; these conflicting genes interfere with the embryo's development and prevents its maturation. Most often, the hybrid embryo dies before birth. However, sometimes, the offspring develops fully with mixed traits, forming a frail, often infertile adult. This hybrid displays reduced fitness, marked by decreased rates of survival and reproduction relative to the parent species. The offspring fails to compete with purebred individuals, limiting genes flow between species.

Evolution of Hybrid Inviability in Tetrapods

In the 1970s, Allan C. Wilson and his colleagues first investigated the evolution of hybrid inviability in tetrapods, specifically mammals, birds, and frogs.

Recognizing that hybrid viability decreases with time, the researchers used molecular clocks to quantify divergence time. They identified how long ago the common ancestor of hybridizing species diverged into two lines, and found that bird and frog species can produce viable hybrids up to twenty million years after speciation. In addition, the researchers showed that mammal species can only produce viable hybrids up to two or three million years after speciation.

Wilson et al. (1974) proposes two hypotheses to explain the relatively faster evolution of hybrid inviability in mammals: the Regulatory and the Immunological Hypotheses. Subsequent research finds support for these hypotheses.

The Regulatory Hypothesis accounts for two characteristics of mammals, and explains the general formation of hybrid inviability in mammals, birds, and frogs.

First, mammals display relatively lower protein diversity than frogs. As Wilson et al. (1974) suggests, “mammals that can hybridize with each other differ only slightly at the protein level, whereas frogs that differ substantially in protein sequence hybridize readily.” This analysis suggests that gene divergence is not the only determinate of hybridization in mammals, birds, or frogs.

Second, the evolution of anatomical diversity occurred far faster in mammals than in either birds or frogs. As Fitzpatrick (2004) indicates, “the morphological disparities among bats, mole-rats, and whales are more dramatic than any disparities in birds and frogs.” This anatomical diversity is evidence for the diversification of regulatory systems. This mammalian characteristic suggests that, although mammals are genetically similar, dramatic changes in regulatory genes caused distinct developmental differences.

The Regulatory Hypotheses specifically attributes hybrid inviability in mammals, birds, and frogs to differences in gene regulation. It proposes that hybrid inviability evolved faster in mammalian taxa because mammals have accumulated significantly more changes in regulatory systems than birds or frogs, and it suggests that organisms with distinctly different systems of gene regulation may not produce viable hybrids.

Wilson et al. (1974) recognizes that the development of embryos in the mammalian placenta requires regulatory compatibility. Both the regulatory genes of the sperm and egg contribute to the expression of other protein-coding genes in the zygote; if certain regulatory genes are not expressed or are expressed at the wrong time, the inter-specific zygote will abort or develop unhealthy traits. Moreover, because the development of the zygote depends on maternal characteristics, such as cytoplasmic determinants, the regulatory traits of the mother may not support the hybrid's developmental needs.

The Immunological Hypothesis proposes that the divergence of certain protein structures associated with mother and child causes hybrid inviability. The hypothesis applies only to mammals, where fertilization and development is internal. In birds and in frogs, fertilization is primarily external, and the mother’s immune system does not interfere with fetal development.

This hypothesis stems from the immunological characteristics of the placenta, where the growing fetus is in constant contact with the fluids and tissues of the mother. Variation within species and variation between species may contribute to fetal-maternal incompatibility, and according to the hypothesis, if the proteins of the fetus varies significantly from the proteins of the placenta, the mother may produce anti-bodies that will attack and abort the fetus. Therefore, if the fetal proteins of the father species are incompatible the mother's placental proteins, the mother's immune system may abort the embryo.

Evidence for the Immunological Hypothesis varies considerably. Wilson et al. (1974) recognizes studies that provide no support to the Immunological Hypotheses. In these experiments, the use of immunological suppressants provided no additional viability to inter-specific hybrids. In contrast, Elliot and Crespi (2006) documents the effects of placental immunology on hybrid inviability, showing that mammals with hemochorial placentas more readily hybridize than mammals with epitheliochorial or endotheliochorial placentas. These different placenta types possess divergent immunological systems, and consequently, they cause varying degrees of hybrid inviability.