In the last several years, activity to develop water-based barrier coatings (WBBCs) that meet challenging packaging performance requirements has increased dramatically. Cellulose-based packaging solutions can provide a more sustainable packaging option for replacing single-use plastic-based options like extrusion-based and laminated materials. An advantage of WBBCs is the opportunity to reduce the coating thickness applied, as long as the barrier requirements can be met. A challenge that must be overcome is the ability to maintain a defect and pin-hole-free coating layer after coating and drying to retain the barrier performance. Many formulation and coating parameters can affect the barrier coating layer quality; however, methods for detecting more subtle differences in these types of studies are not widely available. Work was carried out to develop a quantitative technique for detecting and measuring the quantity and size of defects in the barrier coating layer. A test method has been developed using a combination of dyed oil and image analysis to be able to characterize the imperfections in the coating surface. The use of dyed oil serves two purposes. First, it better simulates the types of materials, in this case, oils and grease, for which the barrier coating is expected to hold out. Second, it also provides contrast between the coating and failure points for testing. An image analysis technique is employed to characterize the number and size of the imperfections. For the former, it reduces the testing time required if a quality control or laboratory technician counts the dots. For the latter, it assists with judgment on the source of the root cause of the imperfection, such as base sheet defects, coating dispersion issues, or perhaps micro-blisters in the coating, as some examples. To show the benefit of this technique, several pilot coating studies were designed to see if the new technique could be utilized to detect differences in WBBC performance. Both process and chemical variables were evaluated. With refinement, it is believed this technique can be utilized in development work, as well as for a potential quality control technique for manufacturing of coated paper and paperboard products