Gene regulatory network (GRN) inference, a longstanding challenge in computational biology, aims to construct GRNs from genomic data. Graph Neural Networks (GNNs) are well-suited for this task due to their ability to leverage both node features and topological relationships. This research systematically evaluated various GNN variants, gradually narrowing the focus through a filtering process. The study considered multiple design aspects, including layers, epochs, decoders, activation functions, graph structures, aggregation methods, skip connections, dropout, and hidden dimensions. Ultimately, two promising models emerged, one based on the Chebyshev spectral graph convolutional operator and the other on the Hypergraph convolutional operator, demonstrating state-of-the-art performance. Notably, hypergraphs demonstrated superior performance on real datasets with higher-order dependencies, while the Chebyshev model showed greater generalization across both simulated and real datasets. The code for this research is available online at https://github.com/EmmaDPaul/GRN-inference-using-GNN.