Handoff management is the method in which the mobile node maintains its connection active when it shifts from location to other. The devastating success of mobile devices as well as wireless communications is emphasizing the requirement for the expansion of mobility-aware facilities. Moreover, the mobility of devices requires services adapting their behavior to abrupt context variations and being conscious of handoffs, which make an intermittent discontinuities and unpredictable delays. Thus, the heterogeneity of wireless network devices confuses the situation, since a dissimilar treatment of handoffs and context-awareness is essential for every solution. Hence, this paper introduced the Deep Q network-based Firefly Aquila Optimizer (DQN-FAO) for performing the handoff management. In order to establish the handoff management, the process of selecting network is very important. Here, the network is selected based on the devised FAO algorithm, which is the consolidation of Aquila Optimizer (AO) and Firefly algorithm (FA) that considers the metrics, such as Jitter, Handoff latency, and Received Signal Strength Indicator (RSSI) as fitness function. Moreover, the handover decision is taken by the DQN, where the hyper-parameters are tuned by the devised FAO algorithm. According to the hand over decision taken, the context aware video streaming is happened by adjusting the bit rate of the videos using network bandwidth. Besides, the devised scheme attained the superior performance based on the call drop, energy consumption, handover delay, throughput, handoff latency, and PSNR of 0.5122, 7.086 J, 10.54 ms, 13.17 Mbps, 93.80 ms and 46.89 dB.