In the competitive electricity environment, a large number of transactions are expected to be negotiated, which can compromise the system security and reliability. According to the Federal Energy Regulatory Commission (FERC) order 889, only those transactions that should be settled are within the permissible limits of available transfer capability (ATC). Therefore, it has become essential to determine secure transactions occurring in the new environment for better planning and management of open access environment. This article proposes a new method of secure bilateral transaction determination using AC distribution factors. The bilateral transaction matrix pattern has been determined in the presence of thyristor-controlled phase angle regulator (TCPAR) and thyristor-controlled series compensator (TCSC). The optimal location of TCPAR and TCSC has been determined using the mixed integer non-linear programming approach. The bilateral transaction matrix has been determined in a hybrid market model. The proposed technique has been applied on IEEE 24-bus reliability test system (RTS).