Observations of proto-planetary disks, as well as theoretical modeling, suggest that in the late stages of accretion leading up to the formation of planetesimals, particles grew to pebbles the size of 1-mm to tens of cm, depending on the location and ambient conditions in the disk. That is the same size range that dominates the present-day comet and primitive asteroid mass loss. Meteoroids that size cause visible meteors on Earth. Here, we hypothesize that the size distribution and the physical and chemical properties of young meteoroid streams still contain information about the conditions in the solar nebula during these late stages of accretion towards planetesimal formation. If so, they constrain where long-period (Oort Cloud) comets, Jupiter-family (Scattered Disk Kuiper Belt) comets, and primitive asteroids (Asteroid Belt) formed. From video and visual observations of 47 young meteor showers, we find that freshly ejected meteoroids from long-period comets tend to have low bulk density and are distributed with equal surface area per log-mass interval (magnitude distribution index χ ∼ 1.85), suggesting gentle accretion conditions. Jupiter-family comets, on the other hand, mostly produce meteoroids twice as dense and distributed with a steeper χ ∼ 2.15 or even χ ∼ 2.5, which implies that those pebbles grew from particles fragmenting in a collisional cascade or by catastrophic collisions, respectively. Some primitive asteroids show χ > 2.5, with most mass in small particles, indicating an even more aggressive fragmentation by processes other than mutual collisions. Both comet populations contain an admixture of compact materials that are sometimes sodium-poor, but Jupiter-family comets show a higher percentage (∼8% on average) than long-period comet showers (∼4%) and a wider range of percentages among comets. While there are exceptions in both groups, the implication is that most long-period comets formed under gentle particle growth conditions, possibly near the 30 AU edge of the Trans Neptunian Disk, while most Jupiter family comets formed closer to the Sun where pebbles reached or passed the fragmentation barrier, and primitive asteroids formed in the region where the cores of the giant planets formed. This is possible if the Scattered Disk represents all objects scattered by Neptune during its migration, while the present-day outer Oort cloud formed only during and after the time of the planet instability, well after the Sun had moved away from sibling stars.