Ferrofluids containing magnetic nanoparticles represent a special class of magnetic materials due to the added freedom of particle tumbling in the fluids. We studied this process, known as Brownian relaxation, and its effect on the magnetic properties of ferrofluids with controlled magnetite nanoparticle sizes. For small nanoparticles (below 10 nm diameter), the Néel process is expected to dominate the magnetic response, whereas for larger particles, Brownian relaxation becomes important. Temperature- and magnetic-field-dependent magnetization studies, differential scanning calorimetry, and AC susceptibility measurements were carried out for 6 and 13.5 nm diameter magnetite nanoparticles suspended in water. We identify clear fingerprints of Brownian relaxation for the sample of large-diameter nanoparticles as both magnetic and thermal hysteresis develop at the water freezing temperature, whereas the samples of small-diameter nanoparticles remain hysteresis-free down to the magnetic blocking temperature. This is supported by the temperature-dependent AC susceptibility measurements: above 273 K, the data show a low-frequency Debye peak, which is characteristic of Brownian relaxation. This peak vanishes below 273 K.