Human carbonic anhydrases (hCAs) play a central role in various physiological processes in the human body. HCAs catalyze the reversible hydration of CO2 into HCO3−, and hence maintains the fluid and pH balance. Overexpression of CA II is associated with diseases, such as glaucoma, and epilepsy. Therefore, CAs are important clinical targets and inhibition of different isoforms, especially hCA II is used in treatment of glaucoma, altitude sickness, and epilepsy. Therapeutically used CA inhibitors (CAI) are sulfonamide-based, such as acetazolamide, dichlorphenamide, methazolamide, ethoxzolamide, etc. However, they exhibit several undesirable effects such as numbness, tingling of extremities, malaise, metallic taste, fatigue, renal calculi, and metabolic acidosis. Therefore, there is an urgent need to identify safe and effective inhibitors of the hCAs. In this study, different phenyl boronic acids 1–5 were evaluated against bovine (bCA II) and hCA II. Among all, compound 1 (4-acetylphenyl boronic acid) was found to be active against bCAII and hCA II with IC50 values of 246 ± 0.48 and 281.40 ± 2.8 μM, respectively, while the remaining compounds were found in-active. Compound 1 was identified as competitive inhibitor of hCA II enzyme (Ki = 283.7 ± 0.002 μM). Additionally, compound 1 was found to be non-toxic against BJ Human fibroblast cell line. The X-ray crystal structure for hCA II in-complex with compound 1 was evaluated to a resolution of 2.6 Å. In fact, this the first structural analysis of a phenyl boron-based inhibitor bound to hCA II, allowing an additional structure-activity analysis of the compounds. Compound 1 was found to be directly bound in the active site of hCA II by interacting with His94, His119, and Thr199 residues. In addition, a bond of 3.11 Å between the zinc ion and coordinated boron atom of the boronic acid moiety of compound 1 was also observed, contributing to binding affinity of compound 1 for hCA II. PDB ID8IGF.