Tunable Network Research Articles

Hyperspectral and multispectral remote sensing image fusion using SwinGAN with joint adaptive spatial-spectral gradient loss function

ABSTRACT Hyperspectral remote sensing image (HSI) fusion with multispectral remote sensing images (MSI) improves data resolution. However, current fusion algorithms focus on local information and overlook long-range dependencies. The parameter of network tuning prioritizes global optimization, neglecting spatial and spectral constraints, and limiting spatial and spectral reconstruction capabilities. This study introduces SwinGAN, a fusion network combining Swin Transformer, CNN, and GAN architectures. SwinGAN's generator employs a detail injection framework to separately extract HSI and MSI features, fusing them to generate spatial residuals. These residuals are injected into the supersampled HSI to produce the final image, while a pure CNN architecture acts as the discriminator, enhancing the fusion quality. Additionally, we introduce a new adaptive loss function that improves image fusion accuracy. The loss function uses L1 loss as the content loss, and spatial and spectral gradient loss functions are introduced to improve the spatial representation and spectral fidelity of the fused images. Our experimental results on several datasets demonstrate that SwinGAN outperforms current popular algorithms in both spatial and spectral reconstruction capabilities. The ablation experiments also demonstrate the rationality of the various components of the proposed loss function.

Differential evolution-based neural architecture search for brain vessel segmentation

Brain vasculature analysis is critical in developing novel treatment targets for neurodegenerative diseases. Such an accurate analysis cannot be performed manually but requires a semi-automated or fully-automated approach. Deep learning methods have recently proven indispensable for the automated segmentation and analysis of medical images. However, optimizing a deep learning network architecture is another challenge. Manually selecting deep learning network architectures and tuning their hyper-parameters requires a lot of expertise and effort. To solve this problem, neural architecture search (NAS) approaches that explore more efficient network architectures with high segmentation performance have been proposed in the literature. This study introduces differential evolution-based NAS approaches in which a novel search space is proposed for brain vessel segmentation. We select two architectures that are frequently used for medical image segmentation, i.e. U-Net and Attention U-Net, as baselines for NAS optimizations. The conventional differential evolution and the opposition-based differential evolution with novel search space are employed as search methods in NAS. Furthermore, we perform ablation studies and evaluate the effects of specific loss functions, model pruning, threshold selection and generalization performance on the proposed models. The experiments are conducted on two datasets providing 335 single-channel 8-bit gray-scale images. These datasets are a public volumetric cerebrovascular system dataset (vesseINN) and our own dataset called KUVESG. The proposed NAS approaches, namely UNAS-Net and Attention UNAS-Net architectures, yield better segmentation performance in terms of different segmentation metrics. More specifically, UNAS-Net with differential evolution reveals high dice score/sensitivity values of 79.57/81.48, respectively. Moreover, they provide shorter inference times by a factor of 9.15 than the baseline methods.

Silk-protein-based gradient hydrogels with multimode reprogrammable shape changes for biointegrated devices

Biocompatible and morphable hydrogels capable of multimode reprogrammable, and adaptive shape changes are potentially useful for diverse biomedical applications. However, existing morphable systems often rely on complicated structural designs involving cumbersome and energy-intensive fabrication processes. Here, we report a simple electric-field-activated protein network migration strategy to reversibly program silk-protein hydrogels with controllable and reprogrammable complex shape transformations. The application of a low electric field enables the convergence of net negatively charged protein cross-linking networks toward the anode (isoelectric point plane) due to the pH gradient generated in the process, facilitating the formation of a gradient network structure and systems suitable for three-dimensional shape change. These tunable protein networks can be reprogrammed or permanently fixed by control of the polymorphic transitions. We show that these morphing hydrogels are capable of conformally interfacing with biological tissues by programming the shape changes and a bimorph structure consisting of aligned carbon nanotube multilayers and the silk hydrogels was assembled to illustrate utility as an implantable bioelectronic device for localized low-voltage electrical stimulation of the sciatic nerve in a rabbit.

Data-driven modelling with coarse-grid network models

We propose to use a conventional simulator, formulated on the topology of a coarse volumetric 3D grid, as a data-driven network model that seeks to reproduce observed and predict future well responses. The conceptual difference from standard history matching is that the tunable network parameters are calibrated freely without regard to the physical interpretation of their calibrated values. The simplest version uses a minimal rectilinear mesh covering the assumed map outline and base/top surface of the reservoir. The resulting CGNet models fit immediately in any standard simulator and are very fast to evaluate because of the low cell count. We show that surprisingly accurate network models can be developed using grids with a few tens or hundreds of cells. Compared with similar interwell network models (e.g., Ren et al., 2019, 10.2118/193855-MS), a typical CGNet model has fewer computational cells but a richer connection graph and more tunable parameters. In our experience, CGNet models therefore calibrate better and are simpler to set up to reflect known fluid contacts, etc. For cases with poor vertical connection or internal fluid contacts, it is advantageous if the model has several horizontal layers in the network topology. We also show that starting with a good ballpark estimate of the reservoir volume is a precursor to a good calibration.

Advances in Metal-Organic Frameworks for the Removal of Chemical Warfare Agents: Insights into Hydrolysis and Oxidation Reaction Mechanisms.

The destruction of chemical warfare agents (CWAs) is a crucial area of research due to the ongoing evolution of toxic chemicals. Metal-organic frameworks (MOFs), a class of porous crystalline solids, have emerged as promising materials for this purpose. Their remarkable porosity and large surface areas enable superior adsorption, reactivity, and catalytic abilities, making them ideal for capturing and decomposing target species. Moreover, the tunable networks of MOFs allow customization of their chemical functionalities, making them practicable in personal protective equipment and adjustable to dynamic environments. This review paper focuses on experimental and computational studies investigating the removal of CWAs by MOFs, specifically emphasizing the removal of nerve agents (GB, GD, and VX) via hydrolysis and sulfur mustard (HD) via selective photooxidation. Among the different MOFs, zirconium-based MOFs exhibit extraordinary structural stability and reusability, rendering them the most promising materials for the hydrolytic and photooxidative degradation of CWAs. Accordingly, this work primarily concentrates on exploring the intrinsic catalytic reaction mechanisms in Zr-MOFs through first-principles approximations, as well as the design of efficient degradation strategies in the aqueous and solid phases through the establishment of Zr-MOF structure-property relationships. Recent progress in the tuning and functionalization of MOFs is also examined, aiming to enhance practical CWA removal under realistic battlefield conditions. By providing a comprehensive overview of experimental findings and computational insights, this review paper contributes to the advancement of MOF-based strategies for the destruction of CWAs and highlights the potential of these materials to address the challenges associated with chemical warfare.

The structural tuning of the convolutional neural network forspeaker identification in mel frequency cepstrumcoefficients space

As a result of the literature analysis, the main methods for speaker identification from speech signals were defined. These are statistical methods based on Gaussian mixture model and a universal background model, as well as neural network methods, in particular, using convolutional or Siamese neural networks. The main characteristics of these methods are the recognition performance, a number of parameters, and the training time. High recognition performance is achieved by using convolutional neural networks, but a number of parameters of these networks are much higher than for statistical methods, although lower than for Siamese neural networks. A large number of parameters require a large training set, which is not always available for the researcher. In addition, despite the effectiveness of convolutional neural networks, model size and inference efficiency remain important fordevices with a limited source of computing power, such as peripheral or mobile devices. Therefore, the aspects of tuning of the structure of existing convolutional neural networks are relevant for research. In this work, we have performed a structural tuning of an existing convolutional neural network based on the VGGNet architecture for speaker identification in the space of mel frequency cepstrum coefficients. The aim of the work was to reduce the number of neural network parameters and, as a result, to reduce the network training time, provided that the recognition performance is sufficient (the correct recognition is above 95 %). The neural network proposed as a result of structural tuning has fewer layers than the architecture of the basic neural network. Insteadof the ReLU activation function, the related Leaky ReLU function with a parameter of 0.1 was used. The number of filters and the size of kernels in convolutional layers are changed. The size of kernels for the max pooling layer has been increased. It is proposedto use the averaging of the results of each convolution to input a two-dimensional convolution results to a fully connected layer with the Softmax activation function. The performed experiment showed that the number of parameters of the proposed neural network is 29 % less than the number of parameters of the basic neural network, provided that the speaker recognition performance is almost the same. In addition, the training time of the proposed and basic neural networks was evaluated on five datasets of audio recordings corresponding to different numbers of speakers. The training time of the proposed network was reduced by 10-39 % compared to the basic neural network. The results of the research show the advisability of the structural tuning of the convolutional neural network for devices with a limited source of computing, namely, peripheral or mobile devices.

The Design Methods and Experiments for a 220-GHz Quasi-Optical Cryogenic Schottky Subharmonic Mixer of High Performance

In this article, we present systematic design methods and experimental verifications of a 220-GHz quasi-optical cryogenic Schottky subharmonic mixer. To maximize device performance over a wide bandwidth and overcome the difficulties in conjugate matching designs, an impedance optimization-based cooperative design method is proposed for such quasi-optical mixer that includes the on-chip antenna and heterodyne mixing circuit. A coplanar waveguide (CPW) tuning network is utilized and optimized to offset the parasitic reactance effect of Schottky diodes. Besides, a dual-band log-periodic-slot integrated lens antenna is presented for coupling the radio frequency and local oscillator radiations with high efficiency. A reversed field tracing method is proposed to acquire the position and size of the equivalent Gaussian waists at both frequencies, thus offering the guidance for optimizing the substrate thickness and lens dimensions for improved antenna gain and radiation pattern. A 220-GHz quasi-optical mixer prototype was fabricated and experimentally verified at both room and cryogenic temperatures. At 134 K, the single-sideband noise factor and the conversion loss are around 4.2–8.5 and 6–11 dB over a wide intermediate-frequency bandwidth of 0.5–20 GHz, showing the superior device performance at the comparable bands.

IMPLEMENTATION OF TECHNOLOGY FOR IMPROVING THE QUALITY OF SEGMENTATION OF MEDICAL IMAGES BY SOFTWARE ADJUSTMENT OF CONVOLUTIONAL NEURAL NETWORK HYPERPARAMETERS

Background. The scientists have built effective convolutional neural networks in their research, but the issue of optimal setting of the hyperparameters of these neural networks remains insufficiently researched. Hyperparameters affect model selection. They have the greatest impact on the number and size of hidden layers. Effective selection of hyperparameters improves the speed and quality of the learning algorithm. It is also necessary to pay attention to the fact that the hyperparameters of the convolutional neural network are interconnected. That is why it is very difficult to manually select the effective values of hyperparameters, which will ensure the maximum efficiency of the convolutional neural network. It is necessary to automate the process of selecting hyperparameters, to implement a software mechanism for setting hyperparameters of a convolutional neural network. The author has successfully implemented the specified task.
 Objective. The purpose of the paper is to develop a technology for selecting hyperparameters of a convolutional neural network to improve the quality of segmentation of medical images..
 Methods. Selection of a convolutional neural network model that will enable effective segmentation of medical images, modification of the Keras Tuner library by developing an additional function, use of convolutional neural network optimization methods and hyperparameters, compilation of the constructed model and its settings, selection of the model with the best hyperparameters.
 Results. A comparative analysis of U-Net and FCN-32 convolutional neural networks was carried out. U-Net was selected as the tuning network due to its higher quality and accuracy of image segmentation. Modified the Keras Tuner library by developing an additional function for tuning hyperparameters. To optimize hyperparameters, the use of the Hyperband method is justified. The optimal number of epochs was selected - 20. In the process of setting hyperparameters, the best model with an accuracy index of 0.9665 was selected. The hyperparameter start_neurons is set to 80, the hyperparameter net_depth is 5, the activation function is Mish, the hyperparameter dropout is set to False, and the hyperparameter bn_after_act is set to True.
 Conclusions. The convolutional neural network U-Net, which is configured with the specified parameters, has a significant potential in solving the problems of segmentation of medical images. The prospect of further research is the use of a modified network for the diagnosis of symptoms of the coronavirus disease COVID-19, pneumonia, cancer and other complex medical diseases.

On the Tuning of the Parallel-T RC Network

This paper deals with two methods of tuning the parallel-T RC network. The first one is the simplest possible, in the form of a grounded resistance. The other one is a passive, rather than active, dual input form of a different kind. Neither of them have so far appeared in the literature.

Anti-swelling P(DAC-co-NaSS-co-VDT) polyampholyte hydrogel with solvent displacement and salinity enhanced wet strength and adhesion

Polyampholyte (PA) hydrogels in general possess excellent mechanical properties, but their underwater applications are largely limited due to swelling induced dramatically reduced strength and poor wet adhesion, especially in high salinity environments. Here, we report our approach to fabricating an anti-swelling poly(acryloyloxyethyltrimethyl ammonium chloride (DAC)-co-sodium p-styrenesulfonate (NaSS)-co-2-Vinyl-4,6-diamino-1,3,5-triazine (VDT), or P(DAC-co-NaSS-co-VDT), PA hydrogel by introducing multi-hydrogen bonding capable functional VDT moieties into the DAC/NaSS coexisting and cation/anion ratio easily tunable PA network. The mechanical properties of the P(DAC-co-NaSS-co-VDT) hydrogel in bulk and at interface were controllable with dimethyl sulfoxide (DMSO)/H2O solvent displacement and Hofmeister effect in a wide salinity range, exhibiting maximum tensile strength, Young's modulus, and wet adhesion toughness of 7.6 MPa, 200 MPa, and 350 J m−2, respectively. The anti-swelling and strong P(DAC-co-NaSS-co-VDT) PA hydrogel developed in this work may find great potential in underwater sensing and sewage treatment as well as for many biomedical applications.

An Improved One-to-Three WPT System With Tunable Compensation Network and Enhanced Pulse Density Voltage Control

In order to realize selective power transfer and output voltage regulation of LCC-S compensated one-to-three wireless power transfer (WPT) system, a novel control strategy based on dual switch-controlled capacitor (DSCC) and enhanced pulse density voltage control (EPDVC) is proposed in this paper. The improved series compensation network increases the selectivity of the receivers. Then, by introducing the DSCC into the LCC topology, both the load-independent current in the transmitting coil and the zero voltage switching (ZVS) of the inverter are maintained under different operating frequency. Besides, the common phase shift modulation (PSM) used to regulate the output voltage is substituted by a novel pulse density control strategy to overcome the hard switching issue when the system operates at light load. To verify the theoretical analysis, a 200-W prototype with 50-V input voltage is built and tested. The experimental results validate the selectivity of the one-to-three WPT system and the feasibility and stability of the proposed control strategy based on dual SCC and EPDM. The inverter achieves ZVS within the whole output power range under different operating frequencies and the peak efficiency reaches 90.57% with resistive load.

Surrogate Model Based on an MLP Neural Network and Bayesian Hyperparameter Tuning for Ship Hull Form Optimization

Surrogate Model Based on an MLP Neural Network and Bayesian Hyperparameter Tuning for Ship Hull Form Optimization

An Interfacial Gel Electrode Patch with Tunable Hydrogen Bond Network for Electromyographic Sensing and Discrimination

An Interfacial Gel Electrode Patch with Tunable Hydrogen Bond Network for Electromyographic Sensing and Discrimination

Tunable neural networks for CT image formation.

Optimization of CT image quality typically involves balancing variance and bias. In traditional filtered back-projection, this trade-off is controlled by the filter cutoff frequency. In model-based iterative reconstruction, the regularization strength parameter often serves the same function. Deep neural networks (DNNs) typically do not provide this tunable control over output image properties. Models are often trained to minimize the expected mean squared error, which penalizes both variance and bias in image outputs but does not offer any control over the trade-off between the two. We propose a method for controlling the output image properties of neural networks with a new loss function called weighted covariance and bias (WCB). Our proposed method uses multiple noise realizations of the input images during training to allow for separate weighting matrices for the variance and bias penalty terms. Moreover, we show that tuning these weights enables targeted penalization of specific image features with spatial frequency domain penalties. To evaluate our method, we present a simulation study using digital anthropomorphic phantoms, physical simulation of CT measurements, and image formation with various algorithms. We show that the WCB loss function offers a greater degree of control over trade-offs between variance and bias, whereas mean-squared error provides only one specific image quality configuration. We also show that WCB can be used to control specific image properties including variance, bias, spatial resolution, and the noise correlation of neural network outputs. Finally, we present a method to optimize the proposed weights for a spiculated lung nodule shape discrimination task. Our results demonstrate this new image quality can control the image properties of DNN outputs and optimize image quality for task-specific performance.

A systematic review and meta-analysis of artificial intelligence diagnostic accuracy in prostate cancer histology identification and grading.

Artificial intelligence (AI) is a promising tool in pathology, including cancer diagnosis, subtyping, grading, and prognostic prediction. The aim of the study is to assess AI application in prostate cancer (PCa) histology. We carried out a systematic literature search in 3 databases. Primary outcome was AI accuracy in differentiating between PCa and benign hyperplasia. Secondary outcomes were AI accuracy in determining Gleason grade and agreement among AI and pathologists. Our final sample consists of 24 studies conducted from 2007 to 2021. They aggregate data from roughly 8000 cases of prostate biopsy and 458 cases of radical prostatectomy (RP). Sensitivity for PCa diagnostic exceeded 90% and ranged from 87% to 100%, and specificity varied from 68% to 99%. Overall accuracy ranged from 83.7% to 98.3% with AUC reaching 0.99. The meta-analysis using the Mantel-Haenszel method showed pooled sensitivity of 0.96 with I2 = 80.7% and pooled specificity of 0.95 with I2 = 86.1%. Pooled positive likehood ratio was 15.3 with I2 = 87.3% and negative - was 0.04 with I2 = 78.6%. SROC (symmetric receiver operating characteristics) curve represents AUC = 0.99. For grading the accuracy of AI was lower: sensitivity for Gleason grading ranged from 77% to 87%, and specificity from 82% to 90%. The accuracy of AI for PCa identification and grading is comparable to expert pathologists. This is a promising approach which has several possible clinical applications resulting in expedite and optimize pathology reports. AI introduction into common practice may be limited by difficult and time-consuming convolutional neural network training and tuning.

Heterodimensional structure porous nanofibers embedded confining magnetic nanocrystals for electromagnetic functional material and device

The development of electronic devices and communication technology has put forward higher requirements for electromagnetic (EM) materials, and the exploration of multifunctional EM materials has become a new trend. Herein, heterodimensional structured porous nanofibers are constructed by zero-dimensional Fe nanocrystals and one-dimensional carbon nanofibers, which show excellent microwave absorption and electromagnetic interference (EMI) shielding performance. The balance between impedance matching and EM energy attenuation ability can be tailored through the tunable conductive network. The maximum reflection loss (RL) value of −56.8 dB can be achieved at a lower loading ratio, and the efficient EMI shielding effectiveness of 29.3 dB can be obtained at a higher loading ratio. In addition, the green EMI shielding with absorption as the main microwave attenuation mechanism can be observed in the higher frequency range. An EM energy conversion device is designed to effectively recover wasted EM energy. The EM energy is converted into valuable electrical energy through multiple loss mechanisms caused by heterodimensional structures. This work provides new guidance for the realization of multifunctional EM materials and devices.

Algorithms for Hyperparameter Tuning of LSTMs for Time Series Forecasting

The rapid growth in the use of Solar Energy for sustaining energy demand around the world requires accurate forecasts of Solar Irradiance to estimate the contribution of solar power to the power grid. Accurate forecasts for higher time horizons help to balance the power grid effectively and efficiently. Traditional forecasting techniques rely on physical weather parameters and complex mathematical models. However, these techniques are time-consuming and produce accurate results only for short forecast horizons. Deep Learning Techniques like Long Short Term Memory (LSTM) networks are employed to learn and predict complex varying time series data. However, LSTM networks are susceptible to poor performance due to improper configuration of hyperparameters. This work introduces two new algorithms for hyperparameter tuning of LSTM networks and a Fast Fourier Transform (FFT) based data decomposition technique. This work also proposes an optimised workflow for training LSTM networks based on the above techniques. The results show a significant fitness increase from 81.20% to 95.23% and a 53.42% reduction in RMSE for 90 min ahead forecast after using the optimised training workflow. The results were compared to several other techniques for forecasting solar energy for multiple forecast horizons.

Task-Driven CT Image Quality Optimization for Low-Contrast Lesion Detectability with Tunable Neural Networks.

Low-contrast lesions are difficult to detect in noisy low-dose CT images. Improving CT image quality for this detection task has the potential to improve diagnostic accuracy and patient outcomes. In this work, we use tunable neural networks for CT image restoration with a hyperparameter to control the variance/bias tradeoff. We use clinical images from a super-high-resolution normal-dose CT scan to synthesize low-contrast low-dose CT images for supervised training of deep learning CT reconstruction models. Those models are trained using with multiple noise realizations so that variance and bias can be penalized separately. We use a training loss function with one hyperparameter called the denoising level, which controls the variance/bias tradeoff. Finally, we evaluate the CT image quality to find the optimal denoising level for low-contrast lesion detectability. We evaluate performance using a shallow neural network model classification model to represent a suboptimal image observer. Our results indicate that the optimal networks for low-contrast lesion detectability are those that prioritize bias reduction rather than mean-squared error, which demonstrates the potential clinical benefit of our proposed tunable neural networks.

A 2.2–4.2-GHz Wideband Analog Phase Shifter MMIC With Low RMS Phase/Amplitude Error

This letter presents a 2.2–4.2-GHz 360° continuous-tuned phase shifter (PS) using the Gallium arsenide (GaAs) heterojunction bipolar transistor (HBT) technology. A new method for broadband analog PS design based on a tunable low-pass network (LPN), a tunable high-pass network (HPN), and a magnetically coupled all-pass network (MCAPN) configuration is proposed to achieve uniform phase shifting and flat amplitude response at the same time for each phase-shifting state. The fabricated PS demonstrates a low average insertion loss (IL) of 4.9 dBC a low root-mean-square (rms) phase error of less than 5.5°, and a low rms amplitude error of less than 0.47 dB between 2.2 and 4.2 GHz. The proposed analog PS has the most flattened phase performance compared with previously reported pass PSs over a wide bandwidth.

Application of generative adversarial neural networks for the formation of databases in scanning tunneling microscopy

We discuss the development of a technique for automatic generation of databases of images obtained with a scanning tunneling microscope. An analysis of state-of-the-art methods and means of automatic processing of images obtained from probe and electron microscopes is carried out. We proposed using generative-adversarial networks for generating images taken with a scanning tunneling microscope to form training databases of images. A process of training and comparison of deep convolutional generative adversarial network (DCGAN) architectures using the OpenCV and Keras libraries together with TensorFlow is described, with the best of them identified by computing the metrics IS, FID, and KID. The scaling of images obtained from DCGAN is performed using a method of fine tuning of a super-resolution generative adversarial neural network (SRGAN) and bilinear interpolation based on the Python programming language. An analysis of calculated quantitative metrics values shows that the best results of image generation are obtained using DCGAN96 and SRGAN. It is found that FID and KID metric values for SRGAN method are better than values for bilinear interpolation in all cases except for DCGAN32. All calculations are performed on a GTX GeForce 1070 video card. A method for automatic generation of a scanning tunneling microscope image database based on the stepwise application of DCGAN and SRGAN is developed. Results of generation and comparison of the original image, the one obtained with DCGAN96 and the enlarged image with SRGAN are presented.