<i>Acanthamoeba polyphaga mimivirus</i>, Marseillevirus, and <i>Sputnik</i>, a virophage, are intra-amoebal viruses that have been isolated from water collected in cooling towers. They have provided fascinating data and have raised exciting questions about viruses definition and evolution. Mimivirus and Marseillevirus have been classified in the nucleo-cytoplasmic large DNA viruses (NCLDVs) class. Their genomes are the largest and fifth largest viral genomes sequenced so far. The gene repertoire of these amoeba-associated viruses can be divided into four groups: the core genome, genes acquired by lateral gene transfer, duplicated genes, and ORFans. Open reading frames (ORFs) that have homologs in the NCLDVs core gene set represent 2.9 and 6.1% of the Mimivirus and Marseillevirus gene contents, respectively. A substantial proportion of the Mimivirus, Marseillevirus and Sputnik ORFs exhibit sequence similarities to homologs found in bacteria, archaea, eukaryotes or viruses. The large amount of chimeric genes in these viral genomes might have resulted from acquisitions by lateral gene transfers, implicating sympatric bacteria and viruses with an intra-amoebal lifestyle. In addition, lineage-specific gene expansion may have played a major role in the genome shaping. Altogether, the data so far accumulated on amoeba-associated giant viruses are a powerful incentive to isolate and study additional strains to gain better understanding of their pangenome.