In the ovules of most sexual flowering plants female gametogenesis is initiated from a single surviving gametic cell, the functional megaspore, formed after meiosis of the somatically derived megaspore mother cell (MMC). Because some mutants and certain sexual species exhibit more than one MMC, and many others are able to form gametes without meiosis (by apomixis), it has been suggested that somatic cells in the ovule are competent to respond to a local signal likely to have an important function in determination. Here we show that the Arabidopsis protein ARGONAUTE 9 (AGO9) controls female gamete formation by restricting the specification of gametophyte precursors in a dosage-dependent, non-cell-autonomous manner. Mutations in AGO9 lead to the differentiation of multiple gametic cells that are able to initiate gametogenesis. The AGO9 protein is not expressed in the gamete lineage; instead, it is expressed in cytoplasmic foci of somatic companion cells. Mutations in SUPPRESSOR OF GENE SILENCING 3 and RNA-DEPENDENT RNA POLYMERASE 6 exhibit an identical defect to ago9 mutants, indicating that the movement of small RNA (sRNAs) silencing out of somatic companion cells is necessary for controlling the specification of gametic cells. AGO9 preferentially interacts with 24-nucleotide sRNAs derived from transposable elements (TEs), and its activity is necessary to silence TEs in female gametes and their accessory cells. Our results show that AGO9-dependent sRNA silencing is crucial to specify cell fate in the Arabidopsis ovule, and that epigenetic reprogramming in companion cells is necessary for sRNA-dependent silencing in plant gametes.