Confined motions in complex environments are ubiquitous in microbiology. These situations invariably involve the intricate coupling between fluid flow, soft boundaries, surface forces, and fluctuations. In the present study, such a coupling is investigated using a method combining holographic microscopy and advanced statistical inference. Specifically, the Brownian motion of soft micrometric oil droplets near rigid walls is quantitatively analyzed. All the key statistical observables are reconstructed with high precision, allowing for nanoscale resolution of local mobilities and femtonewton inference of conservative or nonconservative forces. Strikingly, the analysis reveals the existence of a novel, transient, but large, soft Brownian force. The latter might be of crucial importance for microbiological and nanophysical transport, target finding, or chemical reactions in crowded environments, and hence the whole life machinery.