Regional ocean models usually utilize orthogonal curvilinear grids that are fit to the coastline of the modeled regions. While the orthogonality of the grid is required from the perspective of the numerical algorithms, the alignment to the irregular coastlines improves the characterization of the land-sea distribution and the ocean simulation. In this article, we carry out fractal analysis of two representative coastal regions and discuss the trade-offs between the orthogonality and coastline alignment during the grid generation of these regions. A new grid generation method based on Schwarz-Christoffel conformal mappings is proposed, with automatic coastal boundary retrieval algorithm that generates resolution dependent boundary for grid generation and alleviates the human efforts involved in traditional methods. We show that for the southeastern Pacific region, the coastline is smooth with low fractal dimension and there exists effective trade-off with a coastline boundary that adjusts to the desired grid resolution. On the contrary, there is no effective trade-off for southeast China seas where the coastline is of higher fractal dimension, and a coarser coastline boundary is recommended for better orthogonality with little loss in coastline alignment. Further numerical study of coastal trapped Kelvin waves for the typical regions demonstrate that the new coastline-fitting grids achieve smaller error in numerical dispersion and higher accuracy. Through analysis, we conclude that for grid generation for regional ocean modeling, modelers should bring into consideration of the multi-scale fractal characteristics of the coastline.