Compounds with a phosphonate group, i.e., –P(O)(OH)2 group attached directly to the molecule via a P-C bond serve as suitable non-hydrolyzable phosphate mimics in various biomedical applications. In principle, they often inhibit enzymes utilizing various phosphates as substrates. In this review we focus mainly on biologically active phosphonates that originated from our institute (Institute of Organic Chemistry and Biochemistry in Prague); i.e., acyclic nucleoside phosphonates (ANPs, e.g., adefovir, tenofovir, and cidofovir) and derivatives of non-nucleoside phosphonates such as 2-(phosphonomethyl) pentanedioic acid (2-PMPA). Principal strategies of their syntheses and modifications to prodrugs is reported. Besides clinically used ANP antivirals, a special attention is paid to new biologically active molecules with respect to emerging infections and arising resistance of many pathogens against standard treatments. These new structures include 2,4-diamino-6-[2-(phosphonomethoxy)ethoxy]pyrimidines or so-called “open-ring” derivatives, acyclic nucleoside phosphonates with 5-azacytosine as a base moiety, side-chain fluorinated ANPs, aza/deazapurine ANPs. When transformed into an appropriate prodrug by derivatizing their charged functionalities, all these compounds show promising potential to become drug candidates for the treatment of viral infections. ANP prodrugs with suitable pharmacokinetics include amino acid phosphoramidates, pivaloyloxymethyl (POM) and isopropoxycarbonyloxymethyl (POC) esters, alkyl and alkoxyalkyl esters, salicylic esters, (methyl-2-oxo-1,3-dioxol-4-yl) methyl (ODOL) esters and peptidomimetic prodrugs. We also focus on the story of cytostatics related to 9-[2-(phosphonomethoxy)ethyl]guanine and its prodrugs which eventually led to development of the veterinary drug rabacfosadine. Various new ANP structures are also currently investigated as antiparasitics, especially antimalarial agents e.g., guanine and hypoxanthine derivatives with 2-(phosphonoethoxy)ethyl moiety, their thia-analogues and N-branched derivatives. In addition to ANPs and their analogs, we also describe prodrugs of 2-(phosphonomethyl)pentanedioic acid (2-PMPA), a potent inhibitor of the enzyme glutamate carboxypeptidase II (GCPII), also known as prostate-specific membrane antigen (PSMA). Glutamate carboxypeptidase II inhibitors, including 2-PMPA have been found efficacious in various preclinical models of neurological disorders which are caused by glutamatergic excitotoxicity. Unfortunately its highly polar character and hence low bioavailability severely limits its potential for clinical use. To overcome this problem, various prodrug strategies have been used to mask carboxylates and/or phosphonate functionalities with pivaloyloxymethyl, POC, ODOL and alkyl esters. Chemistry and biological characterization led to identification of prodrugs with 44–80 fold greater oral bioavailability (tetra-ODOL-2-PMPA).