Backpropagation Step Research Articles

Improved Monte Carlo tree search formulation with multiple root nodes for discrete sizing optimization of truss structures

This article proposes a novel reinforcement learning algorithm using an improved Monte Carlo tree search (IMCTS) formulation for the discrete optimum design of truss structures. IMCTS with multiple root nodes includes the update process, best reward, accelerating technique and terminal condition. The update process means that once a final solution is found, it is used as the initial solution for the next search tree. The best reward is used in the backpropagation step. The accelerating technique is introduced by decreasing the width of the search tree and reducing the maximum number of iterations. The agent is trained to minimize the total structural weight under various constraints until the terminal condition is satisfied. The optimal solution is the minimum value of all solutions found by the search trees. Numerical examples show that the agent can find the optimal solution with low computational cost, stably produces an optimal design, and is suitable for multi-objective structural optimization and large-scale structures.

Enhanced domain transfer deep fuzzy echo state network for rotating machinery fault diagnosis based on current signal

Although vibration-based fault diagnosis methods have achieved remarkable results in rotating machinery, they are still limited by various factors, such as environment noise and additional sensor installation. Meanwhile, due to inevitable distribution differences in practical engineering, deep transfer learning models are commonly used in fault diagnostic fields. Although deep architectures can extract representative domain-invariant features, they typically result in substantial computational burdens and training time. Therefore, to address the above problems, an enhanced domain transfer deep fuzzy echo state network (EDFESN) is proposed for rotating machinery fault diagnosis based on current signals. Firstly, current signals derived from the driving motor are directly collected without additional sensors. Secondly, an enhanced domain mapping method is raised to solve the problem of data distribution discrepancies between cross domain. Furthermore, a deep fuzzy echo state network (DFESN) integrated with fuzzy clustering is constructed, which utilizes layer-wise fuzzy-tuning learning paradigm instead of backpropagation step to reduce training time. Two datasets from gearbox and bearing both verify the effectiveness of the proposed method. Compared with other popular transfer methods, EDFESN not only achieves highest accuracy, but also has fastest training speed.

Perturbation Augmentation for Adversarial Training with Diverse Attacks

Adversarial Training (AT) aims to alleviate the vulnerability of deep neural networks to adversarial perturbations. However, the AT techniques struggle to maintain the performance on natural samples while improving the deep model’s robustness. The absence of perturbation diversity in generated during the adversarial training degrades the generalizability of the robust models, causing overfitting to particular perturbations and a decrease in natural performance. This study proposes an adversarial training framework that augments adversarial directions from a single-step attack to address the trade-off between robustness and generalization. Inspired by feature scattering adversarial training, the proposed framework computes a principal adversarial direction with a single-step attack that finds a perturbation disrupting the inter-sample relationships in the mini-batch during adversarial training. The principal direction obtained at each iteration is augmented by sampling new adversarial directions within a region spanning 45 degrees from the principal adversarial direction. The proposed adversarial training approach does not require extra backpropagation steps in adversarial direction augmentation. Therefore, generalization of the robust model is improved without posing an additional burden on the feature scattering adversarial training. Experiments on CIFAR-10, CIFAR-100, SVHN, Tiny-ImageNet, and The German Traffic Sign Recognition Benchmark consistently improve the accuracy on adversarial with an almost pristine natural performance.

DaCapo: An On-Device Learning Scheme for Memory-Constrained Embedded Systems

The use of deep neural network (DNN) applications in microcontroller unit (MCU) embedded systems is getting popular. However, the DNN models in such systems frequently suffer from accuracy loss due to the dataset shift problem. On-device learning resolves this problem by updating the model parameters on-site with the real-world data, thus localizing the model to its surroundings. However, the backpropagation step during on-device learning requires the output of every layer computed during the forward pass to be stored in memory. This is usually infeasible in MCU devices as they are equipped only with a few KBs of SRAM. Given their energy limitation and the timeliness requirements, using flash memory to store the output of every layer is not practical either. Although there have been proposed a few research results to enable on-device learning under stringent memory conditions, they require the modification of the target models or the use of non-conventional gradient computation strategies. This paper proposes DaCapo, a backpropagation scheme that enables on-device learning in memory-constrained embedded systems. DaCapo stores only the output of certain layers, known as checkpoints, in SRAM, and discards the others. The discarded outputs are recomputed during backpropagation from the nearest checkpoint in front of them. In order to minimize the recomputation occurrences, DaCapo optimally plans the checkpoints to be stored in the SRAM area at a particular phase of the backpropagation and thus replaces the checkpoints stored in memory as the backpropagation progresses. We implemented the proposed scheme in an STM32F429ZI board and evaluated it with five representative DNN models. Our evaluation showed that DaCapo improved backpropagation time by up to 22% and saved energy consumption by up to 28% in comparison to AIfES, a machine learning platform optimized for MCU devices. In addition, our proposed approach enabled the training of MobileNet, which the MCU device had been previously unable to train.

Differentiable Hierarchical Optimal Transport for Robust Multi-View Learning.

Traditional multi-view learning methods often rely on two assumptions: ( i) the samples in different views are well-aligned, and ( ii) their representations obey the same distribution in a latent space. Unfortunately, these two assumptions may be questionable in practice, which limits the application of multi-view learning. In this work, we propose a differentiable hierarchical optimal transport (DHOT) method to mitigate the dependency of multi-view learning on these two assumptions. Given arbitrary two views of unaligned multi-view data, the DHOT method calculates the sliced Wasserstein distance between their latent distributions. Based on these sliced Wasserstein distances, the DHOT method further calculates the entropic optimal transport across different views and explicitly indicates the clustering structure of the views. Accordingly, the entropic optimal transport, together with the underlying sliced Wasserstein distances, leads to a hierarchical optimal transport distance defined for unaligned multi-view data, which works as the objective function of multi-view learning and leads to a bi-level optimization task. Moreover, our DHOT method treats the entropic optimal transport as a differentiable operator of model parameters. It considers the gradient of the entropic optimal transport in the backpropagation step and thus helps improve the descent direction for the model in the training phase. We demonstrate the superiority of our bi-level optimization strategy by comparing it to the traditional alternating optimization strategy. The DHOT method is applicable for both unsupervised and semi-supervised learning. Experimental results show that our DHOT method is at least comparable to state-of-the-art multi-view learning methods on both synthetic and real-world tasks, especially for challenging scenarios with unaligned multi-view data.

Surrogate modeling for high dimensional uncertainty propagation via deep kernel polynomial chaos expansion

In this paper, deep kernel polynomial chaos expansion (DKPCE) is proposed as a surrogate model for high dimensional uncertainty propagation. Firstly, deep neural network (DNN) and polynomial chaos expansion (PCE) are connected to create a novel network model, the input dimensionality of PCE layer can thus be controlled by restricting the number of neurons in the feature layer. Then, the back-propagation algorithm is employed for computing all the parameters of DKPCE, the dimension reduction and modeling process of DKPCE are thus executed simultaneously. During the modeling process, a data driven method is first implemented for computing the orthogonal polynomial bases within the PCE layer in the forward propagation step, and the partial derivatives for the coefficients of orthogonal polynomial bases are computed first in the back-propagation step. After constructing DKPCE, the coefficients of PCE layer can be utilized to compute the statistical characteristics of system response. Finally, several numerical examples are utilized for validating the effectiveness of DKPCE.

Robust loss function for class imbalanced semantic segmentation and image classification

Real world datasets are often unbalanced. Such an imbalance in data can limit the performance of modern deep learning based solutions by introducing a bias in the trained model. In particular, the trained model show weak performance on sub tasks or classes where the data availability is sparse. In this work, we address such a data imbalance problem and propose a novel modification to the existing cross-entropy loss function to mitigate the issue. Our proposed loss function can amplify the loss gradients generated during the back-propagation step. In particular, we penalize the predictions of the model which can result in higher loss values and gradients. We compare our proposed loss function with several recently proposed approaches and show superior performance. Our experiments show that proposed approach achieves state of the art performance on log-tailed image classification on CIFAR100/10-LT and Imagenet-LT datasets and on semantic segmentation task on Citiscapes dataset.

FastMVAE2: On Improving and Accelerating the Fast Variational Autoencoder-Based Source Separation Algorithm for Determined Mixtures

This article proposes a new source model and training scheme to improve the accuracy and speed of the multichannel variational autoencoder (MVAE) method. The MVAE method is a recently proposed powerful multichannel source separation method. It consists of pretraining a source model represented by a conditional VAE (CVAE) and then estimating separation matrices along with other unknown parameters so that the log-likelihood is non-decreasing given an observed mixture signal. Although the MVAE method has been shown to provide high source separation performance, one drawback is the computational cost of the backpropagation steps in the separation-matrix estimation algorithm. To overcome this drawback, a method called “FastMVAE” was subsequently proposed, which uses an auxiliary classifier VAE (ACVAE) to train the source model. By using the classifier and encoder trained in this way, the optimal parameters of the source model can be inferred efficiently, albeit approximately, in each step of the algorithm. However, the generalization capability of the trained ACVAE source model was not satisfactory, which led to poor performance in situations with unseen data. To improve the generalization capability, this article proposes a new model architecture (called the “ChimeraACVAE” model) and a training scheme based on knowledge distillation. The experimental results revealed that the proposed source model trained with the proposed loss function achieved better source separation performance with less computation time than FastMVAE. We also confirmed that our methods were able to separate 18 sources with a reasonably good accuracy.

A Hybrid Competitive Evolutionary Neural Network Optimization Algorithm for a Regression Problem in Chemical Engineering

Neural networks have demonstrated their usefulness for solving complex regression problems in circumstances where alternative methods do not provide satisfactory results. Finding a good neural network model is a time-consuming task that involves searching through a complex multidimensional hyperparameter and weight space in order to find the values that provide optimal convergence. We propose a novel neural network optimizer that leverages the advantages of both an improved evolutionary competitive algorithm and gradient-based backpropagation. The method consists of a modified, hybrid variant of the Imperialist Competitive Algorithm (ICA). We analyze multiple strategies for initialization, assimilation, revolution, and competition, in order to find the combination of ICA steps that provides optimal convergence and enhance the algorithm by incorporating a backpropagation step in the ICA loop, which, together with a self-adaptive hyperparameter adjustment strategy, significantly improves on the original algorithm. The resulting hybrid method is used to optimize a neural network to solve a complex problem in the field of chemical engineering: the synthesis and swelling behavior of the semi- and interpenetrated multicomponent crosslinked structures of hydrogels, with the goal of predicting the yield in a crosslinked polymer and the swelling degree based on several reaction-related input parameters. We show that our approach has better performance than other biologically inspired optimization algorithms and generates regression models capable of making predictions that are better correlated with the desired outputs.

AN EXPLAINABLE AI MODEL FOR DIABETIC RETINOPATHY DETECTION

Diabetes is the most common long-term condition that affects people of all ages due to inadequate insulin production. The appearance of black spots for millennia, interpreting eye pictures, and detecting diabetic retinopathy in its early stages has been a big challenge. The Explainable AI method is to explain the deep learning model that is understandable by humans and trusts the result. This is especially important in safety critical domains like healthcare or security, which replaces manual processes and understanding of the models function by non-technical domain skilled person. Explainable AI is used to describe an AI model, its expected impact and potential biases. It helps characterize model accuracy, fairness, transparency and outcomes in AI-powered decision making. Explainable AI is crucial for an organization in building trust and confidence when putting AI models into production. AI explains ability also helps an organization adopt a responsible approach to AI development. Specifically, the back propagation step is responsible for updating the weights based on its error function. SHAP or SHAPley Additive explanations are a visualization tool that can be used for making a machine learning model more explainable by visualizing its output. It can be used for explaining the prediction of any model by computing the contribution of each feature to the prediction.

Straggler Mitigation Through Unequal Error Protection for Distributed Approximate Matrix Multiplication

Large-scale machine learning and data mining methods routinely distribute computations across multiple agents to parallelize processing. The time required for the computations at the agents is affected by the availability of local resources and/or poor channel conditions giving rise to the straggler problem. As a remedy to this problem, we employ Unequal Error Protection (UEP) codes to obtain an approximation of the matrix product in the distributed computation setting to provide higher protection for the blocks with higher effect on the final result. We characterize the performance of the proposed approach from a theoretical perspective by bounding the expected reconstruction error for matrices with uncorrelated entries. We also apply the proposed coding strategy to the computation of the back-propagation step in the training of a Deep Neural Network (DNN) for an image classification task in the evaluation of the gradients. Our numerical experiments show that it is indeed possible to obtain significant improvements in the overall time required to achieve the DNN training convergence by producing approximation of matrix products using UEP codes in the presence of stragglers.

Deep Contrastive Principal Component Analysis Adaptive to Nonlinear Data

Principal component analysis (PCA) is one of the most fundamental techniques for big data analytics in e.g, smart manufacturing and biostatistics, which is capable of extracting the most essential information from a dataset. However, when it encounters multiple datasets, PCA cannot reveal the specific inherent data structure hidden in one dataset relative to the other(s), which we term contrastive data analytics in this paper. Although a number of proposals such as contrastive or discriminative PCA have been suggested, they require fine-tuning of hyper-parameters or cannot effectively deal with nonlinear data. In this context, we advocate deep contrastive (Dc) PCA for nonlinear contrastive data analytics, which leverages the power of deep neural networks to explore the hidden nonlinear relationships in the datasets and extract the desired contrastive features. An alternating minimization algorithm is developed for simultaneously seeking the best nonlinear transformations for the data as well as the associated contrastive projections, tantamount to performing an eigenvalue decomposition and a back-propagation step. Substantial numerical tests using both synthetic and real datasets from protein expression, electroencephalography, wearable human motion, and smart manufacturing, are conducted, which corroborate the superior adaptivity of DcPCA in dealing with nonlinear data relative to state-of-the-art alternatives.

Accelerating Convolutional Neural Network-Based Hyperspectral Image Classification by Step Activation Quantization

Convolutional neural networks (CNNs) have achieved excellent feature extraction capabilities in remotely sensed hyperspectral image (HSI) classification. This is due to their ability to learn representative spatial and spectral features. However, it is difficult for conventional computers to classify HSIs quickly enough for practical use in many applications, mainly because of the large number of calculations and parameters needed by deep learning-based methods. Although several weight quantization methods achieved remarkable results in network compression, the network acceleration effect is still not significant because a full exploration of the potential of network acceleration brought by network weight quantization is still absent from the literature. In this article, a new step activation quantization method is proposed to constrain the input of the network layer of the CNN so that the data can be represented by low-bit integers. As a result, floating-point operations can be replaced with integer operations to greatly accelerate the forward (inference) step of the network. Specifically, nonlinear uniform quantization is adopted in this work to restrain the input of the CNN in the forward inference of the step activation quantization layer, and two functions (constant and tanh-like) are used in the backpropagation step to avoid gradient vanishing and noise. Our newly proposed step activation quantization acceleration method is applied to a CNN for HSI with two well-known benchmark data sets and the experimental results demonstrate that the proposed method is very effective in terms of both memory savings and computation acceleration, with only a slight decrease in classification accuracy. Specifically, our method reduces memory requirements in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$13.6\times $ </tex-math></inline-formula> and obtains around <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$10\times {}$ </tex-math></inline-formula> speedup with regard to the original real-valued network version.

Accelerating On-Device Learning with Layer-Wise Processor Selection Method on Unified Memory.

Recent studies have applied the superior performance of deep learning to mobile devices, and these studies have enabled the running of the deep learning model on a mobile device with limited computing power. However, there is performance degradation of the deep learning model when it is deployed in mobile devices, due to the different sensors of each device. To solve this issue, it is necessary to train a network model specific to each mobile device. Therefore, herein, we propose an acceleration method for on-device learning to mitigate the device heterogeneity. The proposed method efficiently utilizes unified memory for reducing the latency of data transfer during network model training. In addition, we propose the layer-wise processor selection method to consider the latency generated by the difference in the processor performing the forward propagation step and the backpropagation step in the same layer. The experiments were performed on an ODROID-XU4 with the ResNet-18 model, and the experimental results indicate that the proposed method reduces the latency by at most 28.4% compared to the central processing unit (CPU) and at most 21.8% compared to the graphics processing unit (GPU). Through experiments using various batch sizes to measure the average power consumption, we confirmed that device heterogeneity is alleviated by performing on-device learning using the proposed method.

Studies on the GAN-Based Anomaly Detection Methods for the Time Series Data

Anomaly detection (AD) for times series data using the generative adversarial network (GAN) has been proposed in recent years. According to the previous study, the GAN-based AD outperformed the cumulative sum (CUSUM) chart. However, no framework for comparison is provided in their works. So, we conduct new studies crucial for the GAN-based AD methods (the MAD-GAN and the TAnoGAN). First, we propose a new framework for fair and systematic comparisons for the prediction performance of the GAN-based AD methods as well as the cumulative sum (CUSUM) chart. So, we evaluate the three methods with four simulation data and secure water treatment system data. Under the proposed comparison framework, the CUSUM chart generally shows prediction performances better than the GAN-based AD methods. Our results imply that more follow-up studies are required before deploying the GAN-based AD methods. Second, we find that adjusting the number of backpropagation steps of the inverse mapping technique can improve the prediction performance of the GAN-based AD methods. Furthermore, we find that monitoring the residuals of the fitted model significantly improves the prediction performance of the GAN-based AD methods as well as the CUSUM chart.

Hyperspectral Classification Using Deep Belief Networks Based on Conjugate Gradient Update and Pixel-Centric Spectral Block Features

This article describes the use of deep belief networks (DBNs) based on the conjugate gradient (CG) update algorithm for hyperspectral classification. DBNs perform two processes: unsupervised pretraining and supervised fine-tuning. The parameter update method in the fine-tuning stage plays a key role in optimizing the classification model. The proposed method employs CG-based fine-tuning to avoid the “zig-zagging” problem with the gradient descent algorithm and to accelerate the DBN convergence. First, the spectral features and pixel-centric spectral block features are extracted from hyperspectral images for use as the input vectors. The update variables are then calculated based on a CG algorithm and the 2-norm, and the parameters are updated during the backpropagation step of the proposed CGDBN. Two models with different CG methods are applied to a public hyperspectral image benchmark for classification experiments and analysis, and the results are compared with those from several classification methods that are currently in use. The experimental results show that the proposed classification models have advantages in terms of model convergence and low sensitivity to certain parameters. In addition, application to a hyperspectral image of coastal wetlands in the Yellow River Delta produces a satisfactory classification. The results of this study demonstrate that the proposed CG-update-based DBN provides a new approach for hyperspectral dataset classification.

Rapid Mapping for German Federal Authorities

Abstract. The Federal Agency for Cartography and Geodesy (BKG) in Germany is a technical agency under the Federal Ministry of the Interior, Building and Community (BMI). The organization consists of three departments namely Geoinformation, Geodesy and a Business Operations/Support department. Each department has multiple units which carry out their own respective duty in their field of subject. Tasks related to providing and homogenizing spatial data are located inside the geoinformation department. The main focus lies here on uniting the data collected by the federal states to form different types of nationwide datasets. The geodesy department focusses on real time satellite navigation systems, very long baseline interferometry and geodetic reference systems.A recent development in the federal authorities has been the demand for quick maps and spatial data to support fast decision making processes. The BKG has established a complete workflow which allows for rapid data processing and the distribution to the respective customers. It is split into six sub processes with several branching options, processing and backpropagation steps. This allows for a systematic approach to covering the customer’s demand for spatial data and for the internal production chain. The workflow is backed up by a project management system, a quality management system and a production management system each covered in a handbook specially prepared to ensure consistency along the workflow.The workflow gets started by the customer posing their request either via the service center (SC) unit, via mail or phone or through the controlling institution as a decree or an executive order. The first contact is crucial as important information is being exchanged and a rough concept of the demand is shaped. Once the actual task has reached the BKG a ticket number and an internal processing identifier will be assigned by the SC and the resource manager (RM) respectively. These unique keys will be used for the communication with the customer and additionally utilized as a tool for the administrative actions being carried out in the background. This action involves querying the license status of the customer which in addition determines the maximum cost of the task and the authorization of the data being used. In parallel the task, now referred to as a ticket, is being implemented in the project management system. Certain additional information will be assigned to the ticket like the project manager (PM) and the project processor. Once this step has been completed the working directory for the ticket will be created by using a predefined folder structure. This structure is composed of a data, export, communications and documentation folder. Depending on the nature of the ticket various preconfigured templates are at hand and will be used to ensure uniformity of the final products. If needed further communication with the customer is carried out. Data research is the last step in this part of the workflow (see figure 1).The processing of the actual data makes up most of the workflow regarding time. Aspects like a uniform styleguide and quality controls by colleagues are performed alongside the work progress. If any non comforming aspect is found, the feedback is given directly to the person in charge of the ticket while also being mirrored in the project management system. Once the product is ready for deployment, final communication with the customer is executed. Additional changes stated by the customer can be implemented whereby an extra iteration of the quality process will be necessary. The data deployment is either carried out as a print product, a DVD, an OGC service or via a download option (see figure 2).The final steps of the workflow involve the report of delivery to the service center, the creation of the cost report and the closure of the ticket in the project management system. The cost report is created in a semi-automatic process from data stored in the project management system and combined with the material cost report. It is then send via e-mail to the service center which will carry out any necessary invoice collections. Furthermore the internal data folder structure is cleaned up and a screenshot is taken of the product to use it as a subsidiary product in the dedicated online gallery of created products. Finally the closing information is relayed to the service center and the ticket gets closed (see figure 3). This whole workflow can be executed within several hours, days or weeks, depending on the deadline imposed by the customer or the event. Actual rapid mapping activations are done by multiple people simultaneously working on different steps of the previously explained workflow. This allows for fast response times and the accomplishment of the activation in time.

Analysis of Diffractive Optical Neural Networks and Their Integration with Electronic Neural Networks.

Optical machine learning offers advantages in terms of power efficiency, scalability and computation speed. Recently, an optical machine learning method based on Diffractive Deep Neural Networks (D2NNs) has been introduced to execute a function as the input light diffracts through passive layers, designed by deep learning using a computer. Here we introduce improvements to D2NNs by changing the training loss function and reducing the impact of vanishing gradients in the error back-propagation step. Using five phase-only diffractive layers, we numerically achieved a classification accuracy of 97.18% and 89.13% for optical recognition of handwritten digits and fashion products, respectively; using both phase and amplitude modulation (complex-valued) at each layer, our inference performance improved to 97.81% and 89.32%, respectively. Furthermore, we report the integration of D2NNs with electronic neural networks to create hybrid-classifiers that significantly reduce the number of input pixels into an electronic network using an ultra-compact front-end D2NN with a layer-to-layer distance of a few wavelengths, also reducing the complexity of the successive electronic network. Using a 5-layer phase-only D2NN jointly-optimized with a single fully-connected electronic layer, we achieved a classification accuracy of 98.71% and 90.04% for the recognition of handwritten digits and fashion products, respectively. Moreover, the input to the electronic network was compressed by >7.8 times down to 10×10 pixels. Beyond creating low-power and high-frame rate machine learning platforms, D2NN-based hybrid neural networks will find applications in smart optical imager and sensor design.

Hardware Efficient Massive MIMO Detector Based on the Monte Carlo Tree Search Method

A Monte Carlo tree search (MCTS)-based large-scale multiple-input, multiple-output (MIMO) detector is proposed. We describe how the MCTS algorithm, which has been successfully used in decision-making and game-playing problems, can be applied to MIMO detection. In particular, we discuss how the tree policy, default policy, simulation, and backpropagation steps of MCTS can be adapted to MIMO detection. We also describe some optimizations that reduce both the bit error rate and the computational complexity. The proposed MCTS MIMO detector exhibits performance that is comparable to existing methods while having a lower computational load. The design has been implemented in a 65-nm CMOS technology. For a $64 \times 8$ MIMO system, it achieves a throughput of 665 Mbps with a core area of 1.43 mm2, and it exhibits higher hardware efficiency than previous MIMO detector designs in the literature.

MIMO Nonlinear Ultrasonic Tomography by Propagation and Backpropagation Method

This paper develops a fast ultrasonic tomographic imaging method in a multiple-input multiple-output (MIMO) configuration using the propagation and backpropagation (PBP) method. By this method, ultrasonic excitation signals from multiple sources are transmitted simultaneously to probe the objects immersed in the medium. The scattering signals are recorded by multiple receivers. Utilizing the nonlinear ultrasonic wave propagation equation and the received time domain scattered signals, the objects are to be reconstructed iteratively in three steps. First, the propagation step calculates the predicted acoustic potential data at the receivers using an initial guess. Second, the difference signal between the predicted value and the measured data is calculated. Third, the backpropagation step computes updated acoustical potential data by backpropagating the difference signal to the same medium computationally. Unlike the conventional PBP method for tomographic imaging where each source takes turns to excite the acoustical field until all the sources are used, the developed MIMO-PBP method achieves faster image reconstruction by utilizing multiple source simultaneous excitation. Furthermore, we develop an orthogonal waveform signaling method using a waveform delay scheme to reduce the impact of speckle patterns in the reconstructed images. By numerical experiments we demonstrate that the proposed MIMO-PBP tomographic imaging method results in faster convergence and achieves superior imaging quality.