Structural brain networks constructed based on diffusion-weighted MRI (dMRI) have provided a systems perspective to explore the organization of the human brain. Some redundant and nonexistent fibers, however, are inevitably generated in whole brain tractography. We propose to add one critical step while constructing the networks to remove these fibers using the linear fascicle evaluation (LiFE) method, and study the differences between the networks with and without LiFE optimization. For a cohort of nine healthy adults and for 9 out of the 35 subjects from Human Connectome Project, the T1-weighted images and dMRI data are analyzed. Each brain is parcellated into 90 regions-of-interest, whilst a probabilistic tractography algorithm is applied to generate the original connectome. The elimination of redundant and nonexistent fibers from the original connectome by LiFE creates the optimized connectome, and the random selection of the same number of fibers as the optimized connectome creates the non-optimized connectome. The combination of parcellations and these connectomes leads to the optimized and non-optimized networks, respectively. The optimized networks are constructed with six weighting schemes, and the correlations of different weighting methods are analyzed. The fiber length distributions of the non-optimized and optimized connectomes are compared. The optimized and non-optimized networks are compared with regard to edges, nodes and networks, within a sparsity range of 0.75–0.95. It has been found that relatively more short fibers exist in the optimized connectome. About 24.0% edges of the optimized network are significantly different from those in the non-optimized network at a sparsity of 0.75. About 13.2% of edges are classified as false positives or the possible missing edges. The strength and betweenness centrality of some nodes are significantly different for the non-optimized and optimized networks, but not the node efficiency. The normalized clustering coefficient, the normalized characteristic path length and the small-worldness are higher in the optimized network weighted by the fiber number than in the non-optimized network. These observed differences suggest that LiFE optimization can be a crucial step for the construction of more reasonable and more accurate structural brain networks.