As a competitive new energy storage technology, the thermally integrated pumped thermal electricity storage (TI-PTES) system has received much attention. In this work, two TI-PTES systems are constructed: the RHP-BORC system, which consists of a heat pump cycle (HP) with a regenerator and a basic organic Rankine cycle (BORC), and the RHP-EOFC system, which consists of a HP with a regenerator and an organic flash cycle with ejectors. Meanwhile, R1233zd(E) and Cis-butene are selected to form R1233zd(E)/R1233zd(E) (denoting the working fluid used in the charge and discharge cycles, respectively), R1233zd(E)/Cis-butene, Cis-butene/Cis-butene, and Cis-butene/R1233zd(E) groups. Thermodynamic and economic analyses as well as multi-objective optimization are performed for both systems. The results indicate that for the two systems with different working fluid groups, different parameters have distinct effects on the system performance, and the thermodynamic and economic performance of the RHP-BORC system is better than those of the RHP-EOFC system. For the RHP-BORC system, with the variations of thermal storage temperature, evaporation temperature of heat pump and pinch point temperature difference, the values of different performance metrics of the system with different working fluid groups are close respectively, such as power-to-power efficiency, energy storage density and annual emission reduction. For the RHP-EOFC system under different parameter conditions, when the system uses R1233zd(E)/Cis-butene and Cis-butene/Cis-butene, the performance is better. From the results of the multi-objective optimization, the optimal working fluid group of the RHP-BORC and RHP-EOFC systems are Cis-butene/Cis-butene and R1233zd(E)/Cis-butene, and the values of the power-to-power efficiency and the levelized cost of storage under the optimal solution conditions are 75.76 %, 0.234 $·kWh−1, and 84.31 %, 0.227 $·kWh−1, respectively. The results of the exergy destruction analysis under the optimal solution conditions show that the largest exergy destruction in the RHP-BORC and RHP-EOFC systems are in the condenser (480 kW) and compressor (1275 kW) of the HP cycle. This work expands the configuration of the TI-PTES system, showing promising development prospects. It also provides theoretical support for the further advancement of the TI-PTES system and offers valuable guidance for the design of such systems.