Wide Network Research Articles

Asymptotics of representation learning in finite Bayesian neural networks* *This article is an updated version of: Zavatone-Veth J, Canatar A, Ruben B and Pehlevan C 2021 Asymptotics of representation learning in finite Bayesian neural networks Advances in Neural Information Processing Systems vol 34 ed M Ranzato, A Beygelzimer, Y Dauphin, P S Liang and J Wortman Vaughan (New York: Curran

Recent works have suggested that finite Bayesian neural networks may sometimes outperform their infinite cousins because finite networks can flexibly adapt their internal representations. However, our theoretical understanding of how the learned hidden layer representations of finite networks differ from the fixed representations of infinite networks remains incomplete. Perturbative finite-width corrections to the network prior and posterior have been studied, but the asymptotics of learned features have not been fully characterized. Here, we argue that the leading finite-width corrections to the average feature kernels for any Bayesian network with linear readout and Gaussian likelihood have a largely universal form. We illustrate this explicitly for three tractable network architectures: deep linear fully-connected and convolutional networks, and networks with a single nonlinear hidden layer. Our results begin to elucidate how task-relevant learning signals shape the hidden layer representations of wide Bayesian neural networks.

Improved Residual Network based on norm-preservation for visual recognition

Residual Network (ResNet) achieves deeper and wider networks with high-performance gains, representing a powerful convolutional neural network architecture. In this paper, we propose architectural refinements to ResNet that address the information flow through several layers of the network, including the input stem, downsampling block, projection shortcut, and identity blocks. We will show that our collective refinements facilitate stable backpropagation by preserving the norm of the error gradient within the residual blocks, which can reduce the optimization difficulties of training very deep networks. Our proposed modifications enhance the learning dynamics, resulting in high accuracy and inference performance by enforcing norm-preservation throughout the network training. The effectiveness of our method is verified by extensive experimental results on five computer vision tasks, including image classification (ImageNet and CIFAR-100), video classification (Kinetics-400), multi-label image recognition (MS-COCO), object detection and semantic segmentation (PASCAL VOC). We also empirically show consistent improvements in generalization performance when applying our modifications over different networks to provide new insights and inspire new architectures. The source code is publicly available at: https://github.com/bharatmahaur/LeNo.

Refund of State Financial Losses in Realizing the Welfare State of Law

The concept of punishment is not only emphasized to the subject of the perpetrator, but also the consequences that can be caused need to be accounted. What is the ideal form of punishment for perpetrators of corruption. Does it really need extraordinary efforts to deal with this crime? This article was a type of normative research. The approach used in normative research was the statutory approach, which is an approach taken by examining all laws and regulations related to the legal issues being handled. Refund of state financial losses, especially corruption in Indonesia, is not easy and can be done. The perpetrators of corruption have very strong access to a wide network of power that is difficult for law enforcement to reach. This article describes the regulation of the return of state financial losses due to criminal acts that can be carried out through two legal instruments, namely criminal law instruments and civil law instruments.

New partitioned domino circuit for power-efficient wide gates

In this article, a new domino circuit is presented to decrease the power consumption of wide gates without considerable performance and robustness degradation. The suggested circuit technique modifies the output inverter to decrease the switching power by decreasing the voltage swing. Moreover, the wide pull-down networks can be partitioned into smaller networks. To merge the partitioned pull-down networks, a dynamic NAND gate is employed to increase the circuit performance. Using the proposed technique, the leakage current is decreased in the wide gates because of the stacking effect. Also, the noise immunity is improved due to the body effect. The wide OR gates are simulated in 90-nm CMOS technology. Simulation results show 54% power reduction and 2.55 times improvement in the noise immunity at the same speed in comparison with the conventional circuit technique for 64-input OR gates. In addition, a 128-input AND-OR gate is designed using the proposed circuit. Results demonstrate that power and delay are reduced by 39% and 11%, respectively, compared to the conventional AND-OR gate at the same robustness.

Multi-Angle Fast Neural Tangent Kernel Classifier

Multi-kernel learning methods are essential kernel learning methods. Still, the base kernel functions in most multi-kernel learning methods only with select kernel functions with shallow structures, which are weak for large-scale uneven data. We propose two types of acceleration models from a multidimensional perspective of the data: the neural tangent kernel (NTK)-based multi-kernel learning method is proposed, where the NTK kernel regressor is shown to be equivalent to an infinitely wide neural network predictor, and the NTK with deep structure is used as the base kernel function to enhance the learning ability of multi-kernel models; and a parallel computing kernel model based on data partitioning techniques. An RBF, POLY-based multi-kernel model is also proposed. All models use historical memory-based PSO (HMPSO) for efficient search of parameters within the model. Since NTK has a multi-layer structure and thus has a significant computational complexity, the use of a Monotone Disjunctive Kernel (MDK) to store and train Boolean features in binary achieves a 15–60% training time compression of NTK models in different datasets while obtaining a 1–25% accuracy improvement.

Global Emergency System Based on WPAN and LPWAN Hybrid Networks.

There are multiple methods of communication for the transmission of every type of alarm. In the case of emergency systems, they are usually controlled by private companies that work as bridges between the source and the receiver of an emergency. Furthermore, it is necessary to use an independent communication system for each building, requiring human vigilance, leading to an increase in infrastructure and service costs as well as response time. This paper proposes a hybrid network by combining both wireless personal access network (WPAN) and low power wide access network (LPWAN) communication for the complete development of a communication architecture oriented to an emergency system. The main aim of this work is to provide a global emergency system that is focused on fire detection but is also suitable for other critical events, with low energy consumption, a wide communication range covering up to 50 km, and a low cost of service and infrastructure. This proposal reduces the total energy consumption of the system with respect to typical fire detection systems.

Human activity recognition based on multienvironment sensor data

With the development of artificial intelligence and the broad application of sensors, human activity recognition (HAR) technologies based on noninvasive environmental sensors have received extensive attention and have shown great application value. Owing to the initiative of human activities and machine learning-based methods relying on domain knowledge, obtaining a uniform model to understand the daily behaviors of different residents is difficult. From the perspective of data feature constraints to recognition methods, we constructed a methodology for single user's daily behavior recognition that can adaptively constrain the sensor noise during human activities in multitenant smart home scenarios. We propose a sensor data contribution significance analysis (CSA) method based on the sensor status frequency-inverse type frequency for HAR. This method is employed to measure the contribution of a particular type of sensor to a certain type of behavior recognition. We then build a spatial distance matrix based on the layout of environmental sensors for context-awareness and reducing data noise. Finally, we propose a HAR algorithm based on wide time-domain convolutional neural network and multienvironment sensor data (HAR_WCNN) for daily behavior recognition. Comparative experiment results on the CASAS dataset show that the proposed HAR_WCNN outperforms the compared state-of-the-art methods in terms of HAR accuracy and time consumption.

Gender prediction using ensemble based wide residual network from surveillance video

Gender prediction using ensemble based wide residual network from surveillance video

Multi-isotopic study of the earliest mediaeval inhabitants of Santiago de Compostela (Galicia, Spain)

Santiago de Compostela is, together with Rome and Jerusalem, one of the three main pilgrimage and religious centres for Catholicism. The belief that the remains of St James the Great, one of the twelve apostles of Jesus Christ, is buried there has stimulated, since their reported discovery in the 9th century AD, a significant flow of people from across the European continent and beyond. Little is known about the practical experiences of people living within the city during its rise to prominence, however. Here, for the first time, we combine multi-isotope analysis (δ13C, δ15N, δ18Oap, δ13Cap and 87Sr/86Sr) and radiocarbon dating (14C) of human remains discovered at the crypt of the Cathedral of Santiago to directly study changes in diet and mobility during the first three centuries of Santiago’s emergence as an urban centre (9th–12th centuries AD). Together with assessment of the existing archaeological data, our radiocarbon chronology broadly confirms historical tradition regarding the first occupation of the site. Isotopic analyses reveal that the foundation of the religious site attracted migrants from the wider region of the northwest corner of the Iberian Peninsula, and possibly from further afield. Stable isotope analysis of collagen, together with information on tomb typology and location, indicates that the inhabitants of the city experienced increasing socioeconomic diversity as it became wealthier as the hub of a wide network of pilgrimage. Our research represents the potential of multidisciplinary analyses to reveal insights into the origins and impacts of the emergence of early pilgrimage centres on the diets and status of communities within Christian mediaeval Europe and beyond.

An Optimal Control Approach for Enhancing Transients Stability and Resilience in Super Smart Grids

Super smart grids (SSGs) are a wide area transmission network that mainly uses renewable energy resources (RERs), contributing to the reduction of greenhouse gas (GHGs) emissions and supporting the power infrastructure of multiple countries. The SSGs comprise two-way communication between the loads and sources of different countries, and these loads can be mostly served with numerous types of RERs tied with the grids. The RERs will play a pivotal role in the development of future grids and the generation of electricity. However, the main challenge to tackle in these RERs is that they are intermittent in nature. Due to intermittency in these RERs, transient stability issues have become one of the critical research challenges in SSGs. These stability issues are escalated and become more difficult to handle if a network is vulnerable to an arising of different kinds of faults. To address these problems, multiple approaches to enhance transient stability already exist in the current literature. After reviewing the literature, flexible alternating current transmission systems (FACTS) devices proved more promising in improving transient stability. Among FACTSdevices, UPFC is a versatile FACTS device, which provides complete stability to power system networks in the form of series and shunt compensations. Considering this scenario, a hypothetical network for SSGs is designed in this research work based on the interconnection between two countries, i.e., Denmark and Norway, to address the transient stability issues in SSGs. The complete probabilistic model of the system is also designed to enhance the stability of the system. The results clearly showed that the insertion of UPFC is an effective technique to enhance the transient stability and resilience of the power system networks as compared to other purposed techniques in the literature. The main contribution of this paper is that extensive simulation studies employing accurate RERs models are used to analyze and investigate various problems arising due to the integration of many clusters of RERs in SSGs.

Діяльність політосвітніх організацій на Поділлі у 1920-ті роки

The purpose of the study is to highlight and analyze the activities of Soviet political and educational organizations (clubs, reading rooms) in Podillia to follow the tasks and evolution of the content of their work. Th e research methodology is based on the principles of a concrete-historical approach or historicism, comprehensiveness, integrity, systematicity, as well as the use of the methods of analysis and synthesis, historical-genetic, historical-comparative, statistical, and problem-chronological. Th e scientifi c novelty lies in the fact that for the first time in historiography, the content and essence of the Soviet political and educational work in Podillia during the period of the policy of “Ukrainization” has been comprehensively ana- lyzed. Conclusions. At the beginning of the 1920s, a wide network of Soviet political and world organizations (clubs, farm buildings, and others) was created in Podillia. Th eir creation was preceded by the closure or reorganization of those cultural associations that operated during the time of the Ukrainian People’s Republic. Th e main goal of Soviet political and educational work was to attract the population to “proletarian culture” to Bolshevize it and to turn it away from the Ukrainian national process. However, during the “Ukrainization” policy, clubs and villages also played a positive role in the elimination of illiteracy among the adult population contributing to the formation of a new rural Ukrainian intelligentsia and spreading the achieve- ments of Ukrainian art among the peasantry. In the course of the 1920s, numerous “cleanings” of the funds of village houses, reading rooms, clubs, and libraries took place periodically. Th ey especially intensifi ed aft er the beginning of “complete collectivization” at the end of 1929. Many employees of political organizations began to be accused of political mistakes, carrying out anti- Soviet nationalist work, but this is a topic for a separate study

Antenna contactless partial discharges detection in covered conductors using ensemble stacking neural networks

High impedance faults caused by vegetation are difficult to detect when covered conductors in medium voltage overhead power lines are used. Long-term contact of XLPE insulation with vegetation causes partial discharges (PDs) which damage the insulation. Although a cheap and easy to install, contactless detection method was developed using an antenna, there is a lack of classification algorithms for this method. Only two custom machine learning algorithms have been tested so far, and both rendered unsatisfactory results for the real application. This work investigates the use of neural network algorithms for this problem and the application of heterogeneous stacking ensembles using neural networks. We used real data collected from a number of detection stations in the Czech Republic. Also, we limited ourselves to supporting edge computing using devices such as Edge TPU. We propose the application of a heterogeneous stacking ensemble neural network to classify PDs obtained by the contactless method. The algorithm we propose is based on a stacking ensemble with a novel combination of base learners, and the Wide and Deep neural network is used as a meta-learner. We compared the results of our algorithm with other algorithms designated for time series classification. Also, an ablation study of the ensemble was conducted, and satisfactory results were obtained using the proposed algorithm. The ensemble outperformed all algorithms tested and is usable on the edge using AI HW accelerator as the ensemble is only feedforward and contains only well-used and known layers. This research improves our understanding of the classification of PDs using the contactless PD detection method and also introduces a stacking ensemble of convolutional neural network and autoencoders for a time series classification for the first time.

Apocrypha Hiberniae II: Apocalyptica 2

Apocrypha Hiberniae II: Apocalyptica 2

Review: Art for the Future: Artists Call and Central American Solidarities, edited by Erina Duganne and Abigail Satinsky

Review: <i>Art for the Future: Artists Call and Central American Solidarities</i>, edited by Erina Duganne and Abigail Satinsky

An update on the knee osteoarthritis severity grading using wide residual learning.

Knee Osteoarthritis (KOA) is the most common type of Osteoarthritis (OA) and it is diagnosed by physicians using a standard 0 -4 Kellgren Lawrence (KL) grading system which sets the KOA on a spectrum of 5 grades; starting from normal (0) to Severe OA (4). In this paper, we propose a transfer learning approach of a very deep wide residual learning-based network (WRN-50-2) which is fine-tuned using X-ray plain radiographs from the Osteoarthritis Initiative (OAI) dataset to learn the KL severity grading of KOA. We propose a data augmentation approach of OAI data to avoid data imbalance and reduce overfitting by applying it only to certain KL grades depending on their number of plain radiographs. Then we conduct experiments to test the model based on an independent testing data of original plain radiographs acquired from the OAI dataset. Experimental results showed good generalization power in predicting the KL grade of knee X-rays with an accuracy of 72% and Precision 74%. Moreover, using Grad-Cam, we also observed that network selected some distinctive features that describe the prediction of a KL grade of a knee radiograph. This study demonstrates that our proposed new model outperforms several other related works, and it can be further improved to be used to help radiologists make more accurate and precise diagnosis of KOA in future clinical practice.

A New Bolt Defect Identification Method Incorporating Attention Mechanism and Wide Residual Networks.

Bolts are important components on transmission lines, and the timely detection and exclusion of their abnormal conditions are imperative to ensure the stable operation of transmission lines. To accurately identify bolt defects, we propose a bolt defect identification method incorporating an attention mechanism and wide residual networks. Firstly, the spatial dimension of the feature map is compressed by the spatial compression network to obtain the global features of the channel dimension and enhance the attention of the network to the vital information in a weighted way. After that, the enhanced feature map is decomposed into two one-dimensional feature vectors by embedding a cooperative attention mechanism to establish long-term dependencies in one spatial direction and preserve precise location information in the other direction. During this process, the prior knowledge of the bolts is utilized to help the network extract critical feature information more accurately, thus improving the accuracy of recognition. The test results show that the bolt recognition accuracy of this method is improved to 94.57% compared with that before embedding the attention mechanism, which verifies the validity of the proposed method.

Application of Data Warehouse Technology Based on Neural Network in Physical Education Quality Management

Data warehouse technology has been created because of China’s technological advancement and the increasing requirements of the educational sector. Physical assessments are treated as tests by many students. Institutions spend plenty of time every year through physical tests, yet the results are rarely shared with students. Teachers are impeded by the size and complexity of physical test data, finding it challenging to support experiments or judge individual students’ development. Students have trouble following up and delivering test-based feedback after instruction. In recent years, various researchers have offered insightful advice on how to build multidimensional database structures for such trouble. However, quality requirements alone are not adequate to guarantee quality in reality. So, this paper presents a novel Hypertuned wide polynet convolutional neural network (HWPCNN) framework in the data warehouse technology to attain the greatest performance in physical education quality management. In this paper, we first apply HWPCNNs for physical education quality management to analyze the accuracy and recall of the model. It is no secret that HWPCNN is now one of the most widely used deep learning techniques. When it comes to managing the quality of physical education, the HWPCNN’s local perception feature in the data warehouse technology allows it to achieve the best possible results. To validate the model’s performance, it is compared to other models and then improved further to increase its accuracy. The physical education resources are gathered as a raw dataset for this inquiry. The raw dataset is cleaned using the Z-score approach to get it ready for further data processing. Then, a sparse matrix approach is employed to build a data cube, while the proposed method is used to index multidimensional databases. To demonstrate that our work is of the best quality in managing physical education, performance metrics of the suggested method are also evaluated and compared with other traditional methods.

Common Neuroanatomical Substrate of Cholinergic Pathways and Language-Related Brain Regions as an Explanatory Framework for Evaluating the Efficacy of Cholinergic Pharmacotherapy in Post-Stroke Aphasia: A Review.

Despite the relative scarcity of studies focusing on pharmacotherapy in aphasia, there is evidence in the literature indicating that remediation of language disorders via pharmaceutical agents could be a promising aphasia treatment option. Among the various agents used to treat chronic aphasic deficits, cholinergic drugs have provided meaningful results. In the current review, we focused on published reports investigating the impact of acetylcholine on language and other cognitive disturbances. It has been suggested that acetylcholine plays an important role in neuroplasticity and is related to several aspects of cognition, such as memory and attention. Moreover, cholinergic input is diffused to a wide network of cortical areas, which have been associated with language sub-processes. This could be a possible explanation for the positive reported outcomes of cholinergic drugs in aphasia recovery, and specifically in distinct language processes, such as naming and comprehension, as well as overall communication competence. However, evidence with regard to functional alterations in specific brain areas after pharmacotherapy is rather limited. Finally, despite the positive results derived from the relevant studies, cholinergic pharmacotherapy treatment in post-stroke aphasia has not been widely implemented. The present review aims to provide an overview of the existing literature in the common neuroanatomical substrate of cholinergic pathways and language related brain areas as a framework for interpreting the efficacy of cholinergic pharmacotherapy interventions in post-stroke aphasia, following an integrated approach by converging evidence from neuroanatomy, neurophysiology, and neuropsychology.

IFogLearn++: A new platform for fog layer's IoT attack detection in critical infrastructure using machine learning and big data processing

Nowadays, with an increasing expansion of the internet of things (IoT) that has created massive data streams, online attack detection via stream processing has become a matter of extensive attention. Current encryption and authentication methods cannot satisfy the security requirements of IoT critical infrastructures because of many heterogeneous connected devices, a wide network geographical scope, rapid software development, the possibility of security bugs, and the emergence of new attacks. Therefore, an extended method based on machine learning, which can process data stream to detect the fog layer's IoT attacks and prevent the spread of intrusion to other network segments, was proposed in this paper. The IFogLearn++ uses a fog layer to facilitate fast data stream processing in the fog layer. Using a fog layer helps the network tolerate more attacks due to the annexation of an extra security layer before the cloud layer. Based on the results, the IFogLearn++ has similar accuracy to competitors and 18 and 8 times faster performance compared to SVM and Learn++. This characteristic helps secure massive data streams in IoT.

Chief Executive Officer (CEO) Characteristics and Firm Value: Evidence from Malaysia

Nowadays, the capabilities and competency of top managers such as CEOs are extremely crucial for each firm in order to emerge the business into the market. It is the determinants to a success or failure of a particular firm. The main purpose of this study is to examine the effects of CEO characteristics on firm value. The samples are annual reports from non-financial firms listed on the Bursa Malaysia during the period of 2013 to 2015. CEO characteristics of gender, age, education, ownership and network were developed based on the upper-echelon (UP), agency and resource dependence theories. The results imply that these CEO gender and networks are negatively significant to Malaysia firms and play an important role in determining firm value. The involvement of women is still rare in listed companies harm firm performance. When a CEO has a wider network, they tend to lost focus because of having too much commitment which resulting to decrease the firm value. In addition, the firm characteristics consisting firm age, firm size and firm leverage also found significant to firm value. This study would contribute to assist practitioners and policy maker about the various impacts of CEO characteristics to firm value. For future research, this study emphasizes other measurements CEO characteristics that could affect firm value.