Quantum Neural Networks (QNNs) are one of the most promising applications that can be implemented on NISQ-era quantum computers. In this study, we observe that QNNs often suffer from gate redundancy, which hugely declines the performance and accuracy of the network. Even state-of-the-art architecture search techniques like QuantumNAS do not completely alleviate this problem. Especially, We find that CNOT gates are major contributors to the execution delay and noise in quantum circuits, and there are many redundant CNOT gates in the QNN post-training. This motivates us to propose a novel distribution-based greedy-search circuit optimization technique, that can be employed after the completion of the training process. Our technique significantly reduces the number of CNOT gates in QNNs without affecting the accuracy of the network. With this technique, we have achieved an average of 3 × improvement in execution time while reaching a maximum of 12.4 × improvement.