Currently, hundreds of herbal products with potential hepatotoxicity were available in the literature. A comprehensive summary and analysis focused on these potential hepatotoxic herbal products may assist in understanding herb-induced liver injury (HILI). In this work, we collected 335 hepatotoxic medicinal plants, 296 hepatotoxic ingredients, and 584 hepatoprotective ingredients through a systematic literature retrieval. Then we analyzed these data from the perspectives of phylogenetic relationship and structure-toxicity relationship. Phylogenetic analysis indicated that hepatotoxic medicinal plants tended to have a closer taxonomic relationship. By investigating the structures of the hepatotoxic ingredients, we found that alkaloids and terpenoids were the two major groups of hepatotoxicity. We also identified eight major skeletons of hepatotoxicity and reviewed their hepatotoxic mechanisms. Additionally, 15 structural alerts (SAs) for hepatotoxicity were identified based on SARpy software. These SAs will help to estimate the hepatotoxic risk of ingredients from herbs. Finally, a herb-ingredient network was constructed by integrating multiple datasets, which will assist to identify the hepatotoxic ingredients of herb/herb-formula quickly. In summary, a systemic analysis focused on HILI was conducted which will not only assist to identify the toxic molecular basis of hepatotoxic herbs but also contribute to decipher the mechanisms of HILI.