Uncovering slow collective variables (CVs) of self-assembly dynamics is important to elucidate its numerous kinetic assembly pathways and drive the design of novel structures for advanced materials through the bottom-up approach. However, identifying the CVs for self-assembly presents several challenges. First, self-assembly systems often consist of identical monomers, and the feature representations should be invariant to permutations and rotational symmetries. Physical coordinates, such as aggregate size, lack high-resolution detail, while common geometric coordinates like pairwise distances are hindered by the permutation and rotational symmetry challenges. Second, self-assembly is usually a downhill process, and the trajectories often suffer from insufficient sampling of backward transitions that correspond to the dissociation of self-assembled structures. Popular dimensionality reduction methods, such as time-structure independent component analysis, impose detailed balance constraints, potentially obscuring the true dynamics of self-assembly. In this work, we employ GraphVAMPnets, which combines graph neural networks with a variational approach for Markovian process (VAMP) theory to identify the slow CVs of the self-assembly processes. First, GraphVAMPnets bears the advantages of graph neural networks, in which the graph embeddings can represent self-assembly structures in high-resolution while being invariant to permutations and rotational symmetries. Second, it is built upon VAMP theory, which studies Markov processes without forcing detailed balance constraints, which addresses the out-of-equilibrium challenge in the self-assembly process. We demonstrate GraphVAMPnets for identifying slow CVs of self-assembly kinetics in two systems: the aggregation of two hydrophobic molecules and the self-assembly of patchy particles. We expect that our GraphVAMPnets can be widely applied to molecular self-assembly.