The physicochemical properties of nanobiomaterials, such as their small size and high surface area ratio, make them attractive, novel drug-carriers, with increased cellular interaction and increased permeation through several biological barriers. However, these same properties hinder any extrapolation of knowledge from the toxicity of their raw material. Though, as suggested by the Safe-by-Design (SbD) concept, the hazard assessment should be the starting point for the formulation development. This may enable us to select the most promising candidates of polymeric nanobiomaterials for safe drug-delivery in an early phase of innovation. Nowadays the majority of reports on polymeric nanomaterials are focused in optimizing the nanocarrier features, such as size, physical stability and drug loading efficacy, and in performing preliminary cytocompatibility testing and proving effectiveness of the drug loaded formulation, using the most diverse cell lines. Toxicological studies exploring the biological effects of the polymeric nanomaterials, particularly regarding immune system interaction are often disregarded. The objective of this review is to illustrate what is known about the biological effects of polymeric nanomaterials and to see if trends in toxicity and general links between physicochemical properties of nanobiomaterials and their effects may be derived. For that, data on chitosan, polylactic acid (PLA), polyhydroxyalkanoate (PHA), poly(lactic-co-glycolic acid) (PLGA) and policaprolactone (PCL) nanomaterials will be evaluated regarding acute and repeated dose toxicity, inflammation, oxidative stress, genotoxicity, toxicity on reproduction and hemocompatibility. We further intend to identify the analytical and biological tests described in the literature used to assess polymeric nanomaterials toxicity, to evaluate and interpret the available results and to expose the obstacles and challenges related to the nanomaterial testing. At the present time, considering all the information collected, the hazard assessment and thus also the SbD of polymeric nanomaterials is still dependent on a case-by-case evaluation. The identified obstacles prevent the identification of toxicity trends and the generation of an assertive toxicity database. In the future, in vitro and in vivo harmonized toxicity studies using unloaded polymeric nanomaterials, extensively characterized regarding their intrinsic and extrinsic properties should allow to generate such database. Such a database would enable us to apply the SbD approach more efficiently.