Often, Structural Health Monitoring (SHM) campaigns draw damage-identifiers from vibration-based monitoring data. One of the downstream tasks of vibration-based SHM is that of system identification, i.e., inference of a model which is able to describe the system. For vibration-based monitoring, such models often rely on partial-differential equation formulations, where the identification task may be tied to inference of the parameters of such models. The probabilistic nature of SHM incentivises the extraction of stochastic estimates of these parameters, allowing for propagation of uncertainties associated to measurement noise or model imprecision. Common approaches to generating such stochastic estimates require an assumption on the shape and independence of the posterior distributions. To circumvent these assumptions, the Hamiltonian Monte-Carlo (HMC) sampling approach simulates the posterior distribution, allowing for freedom in the posterior. However, this requires intensive computational effort, especially when the governing equations have increased complexities, such as nonlinearities. Physics-informed neural networks (PINNs) utilise physics-based objective functions in the form of known PDEs, exploiting the automatic differentiation of NN architectures. This implies that additional terms in the PDE, such as nonlinearities, or boundary conditions can be accounted for. When the domain of observations encapsulates the collocation domain of the PINN, they can be exploited for parameter identification, by optimising over these parameters in addition to the network weights. However, classic objective functions in the form of mean-squared-norms underperform on noisy data, as a consequence of their deterministic nature. In this work, PINNs are combined with a HMC sampler to estimate the posterior distribution of the parameters of dynamic systems with noisy measurements. Multiple types of dynamic systems are simulated, varying the system complexity in terms of type and strength, and with different levels of noise. The results are assessed against the known values, and the codependent posterior distributions of each system are discussed.