Ginseng has been traditionally used as an herbal nutritional supplement in Asian countries, including Korea, China, Japan, and Vietnam for several millennia. Most studies have focused on the role of ginseng on anti-oxidative stress, anti-inflammatory, and anti-cancer activities. Recently, modulator activities of ginseng on the immune responses during pathogenic bacterial and viral infections and beneficial effects of ginseng in infectious diseases have been elucidated. In vivo and in vitro studies revealed the potential of ginseng extracts and ginsenosides Rg1, Rg3, Rb1, Rb2, Rb3, compound K, Re, Rd, Rh2 for treatment of several infectious diseases. The molecular mechanisms of these effects mainly involve inflammatory cytokines (TNF-α, IL-6, IL-1β, IFN-γ, IL-10), apoptotic pathway (bcl-2, bcl-xL), PI3K/Akt pathway, MAPKs pathway, JAK2/STAT5, NF-κB pathway, and the inflammasome. In this review, we will summarize the current knowledge on the effects of ginseng in the immune responses during the infections and its bioactivities on the prevention of infectious diseases as well as its underlying mechanisms. Moreover, the therapeutic potential of ginseng as an anti-bacterial and anti-viral medication and vaccine adjuvant will be discussed as well.