Quantization in the inversion layer and phase coherent transport are anticipated to have significant impact on device performance in “ballistic” nanoscale transistors. While the role of some quantum effects have been analyzed qualitatively using simple one-dimensional ballistic models, two-dimensional (2D) quantum mechanical simulation is important for quantitative results. In this paper, we present a framework for 2D quantum mechanical simulation of a nanotransistor/metal oxide field effect transistor. This framework consists of the nonequilibrium Green’s function equations solved self-consistently with Poisson’s equation. Solution of this set of equations is computationally intensive. An efficient algorithm to calculate the quantum mechanical 2D electron density has been developed. The method presented is comprehensive in that treatment includes the three open boundary conditions, where the narrow channel region opens into physically broad source, drain and gate regions. Results are presented for (i) drain current vs drain and gate voltages, (ii) comparison to results from Medici, and (iii) gate tunneling current, using 2D potential profiles. Methods to reduce the gate leakage current are also discussed based on simulation results.