Alzheimer's Disease, a progressive neurodegenerative disorder with no cure, presents a formidable challenge to global healthcare. The simulation of brain states, encompassing both rejuvenation and aging, aids clinicians in understanding disease progression and early detection. Recent advances in deep adversarial neural networks, effective in generating age-varied facial images, are now applied to brain MRI simulations across different time states. However, existing methods for brain MRI synthesis predominantly focus on aging simulation, lacking rejuvenation capabilities, and relying heavily on longitudinal data. Access to rejuvenated brain scans is essential as it can help researchers track the structural changes and biomarkers in the early stage of AD, facilitating AD research and personalized treatments. Additionally, previous methods are often 2D-based, failing to capture complex 3D spatial information obtained by 3D scans. To overcome these limitations, I propose BrainSim, a novel longitudinal brain MRI synthesis framework. Specifically, BrainSim leverages a bidirectional cycle-consistent generative neural network to generate rejuvenated and aged MRI scans simultaneously using cross-sectional scans only. BrainSim incorporates a 3D conditional U-Net as its fundamental model, facilitating the direct generation of 3D MRI scans conditioned on the subject's health state and sex. BrainSim was evaluated against benchmark models using longitudinal data and a pre-trained age predictor to assess the quality of generated images. Experiments using the ADNI dataset show BrainSim's state-of-the-art performance in brain aging and rejuvenation tasks, marking significant advancements in MRI synthesis for Alzheimer's research and drug development.