Surface electromyography (sEMG) is a non-invasive technique to characterize muscle electrical activity. The analysis of sEMG signals under muscle fatigue play a crucial part in the branch of neurorehabilitation, sports medicine, biomechanics, and monitoring neuromuscular pathologies. In this work, a method to transform sEMG signals to complex networks under muscle fatigue conditions using Markov transition field (MTF) is proposed. The importance of normalization to a constant Maximum voluntary contraction (MVC) is also considered. For this, dynamic signals are recorded using two different experimental protocols one under constant load and another referenced to 50% MVC from Biceps brachii of 50 and 45 healthy subjects respectively. MTF is generated and network graph is constructed from preprocesses signals. Features such as average self-transition probability, average clustering coefficient and modularity are extracted. All the extracted features showed statistical significance for the recorded signals. It is found that during the transition from non-fatigue to fatigue, average clustering coefficient decreases while average self-transition probability and modularity increases. The results indicate higher degree of signal complexity during non-fatigue condition. Thus, the MTF approach may be used to indicate the complexity of sEMG signals. Although both datasets showed same trend in results, sEMG signals under 50% MVC exhibited higher separability for the features. The inter individual variations of the MTF features is found to be more for the signals recorded using constant load. The proposed study can be adopted to study the complex nature of muscles under various neuromuscular conditions.