We present a measurement of faint-source confusion in deep, wide-field 4 μm images. The 18 × 18 images with 17'' resolution are centered about the nearby edge-on spiral galaxies NGC 4565 and NGC 5907. After removing statistical noise and gain fluctuations in the focal plane array, we measure spatial fluctuations in the sky brightness to be δνIν = 2.74 ± 0.14 nW m-2 sr-1, approximately 1% of the diffuse background level observed in a single pixel. The brightness fluctuations are confirmed to be associated with the sky by subtracting sequential images of the same region. An autocorrelation analysis shows the fluctuations are well described by unresolved point sources. We see no evidence for surface brightness fluctuations on larger angular scales (2' S) = 1.04 nW m-2 sr-1 to the cosmic infrared background, evaluated at S = 4.0 × 10-8 nW m-2. From the fluctuation data we can determine the integrated source counts N(>S) = 1.79 × 107 sr-1, evaluated at S = 4.0 × 10-8 nW m-2. The observed fluctuations are consistent with reddened K-band galaxy number counts. The number counts of extracted point sources with flux νFν > 6.3 × 10-7 nW m-2 are dominated by stars and agree well with the Galactic stellar model of Wright & Reese. Removing the stellar contribution from DIRBE maps with zodiacal subtraction results in a residual brightness of 14.0 ± 2.6 (22.2 ± 5.9) nW m-2 sr-1 at 3.5 (4.9) μm for the NGC 5907 field and 24.0 ± 2.7 (36.8 ± 6.0) nW m-2 sr-1 at 3.5 (4.9) μm for the NGC 4565 field. The NGC 5907 residuals are consistent with tentative detections of the infrared background reported by Dwek & Arendt, Wright & Reese, and Gorjian, Wright, & Chary.