Large-scale Sets Research Articles

Effectiveness Analysis of Stock KDJ Indicator Method based on K-means Clustering

The application of data mining technology expands various techniques in stock investment. Among them, cluster analysis is one of the common means to study stock technical indicators. There is a problem in the current cluster analysis of stock technical indicators -- the lack of validation of large-scale stock technical indicators data sets. Most of them are suitable for the comprehensive analysis of technical indicators of a single stock or multiple stocks. Aiming at this problem, this paper takes "the validity analysis of the large-scale stock KDJ index set method based on K-means clustering" as the theme. Firstly, the k-means clustering algorithm was used to construct a deep analysis model (KDJ-k-means) for the KDJ index set of the Shenzhen Index component data group. Secondly, the K, D and J index sets of 2697 constituent stocks of Shenzhen Composite Index are analyzed experimentally. Finally, the results of integrated data mining are obtained. The KDJ-k-means model is an optimization scheme based on the KDJ index set using clustering technology, which provides an intuitive and efficient visual application for deep analysis of large-scale stock data groups.

Scalable NMF via linearly optimized data compression.

Orthonormal projective non-negative matrix factorization (opNMF) has been widely used in neuroimaging and clinical neuroscience applications to derive representations of the brain in health and disease. The non-negativity and orthonormality constraints of opNMF result in intuitive and well-localized factors. However, the advantages of opNMF come at a steep computational cost that prohibits its use in large-scale data. In this work, we propose novel and scalable optimization schemes for orthonormal projective non-negative matrix factorization that enable the use of the method in large-scale neuroimaging settings. We replace the high-dimensional data matrix with its corresponding singular value decomposition (SVD) and QR decompositions and combine the decompositions with opNMF multiplicative update algorithm. Empirical validation of the proposed methods demonstrated significant speed-up in computation time while keeping memory consumption low without compromising the accuracy of the solution.

A data-driven investigation of human action representations

Understanding actions performed by others requires us to integrate different types of information about people, scenes, objects, and their interactions. What organizing dimensions does the mind use to make sense of this complex action space? To address this question, we collected intuitive similarity judgments across two large-scale sets of naturalistic videos depicting everyday actions. We used cross-validated sparse non-negative matrix factorization to identify the structure underlying action similarity judgments. A low-dimensional representation, consisting of nine to ten dimensions, was sufficient to accurately reconstruct human similarity judgments. The dimensions were robust to stimulus set perturbations and reproducible in a separate odd-one-out experiment. Human labels mapped these dimensions onto semantic axes relating to food, work, and home life; social axes relating to people and emotions; and one visual axis related to scene setting. While highly interpretable, these dimensions did not share a clear one-to-one correspondence with prior hypotheses of action-relevant dimensions. Together, our results reveal a low-dimensional set of robust and interpretable dimensions that organize intuitive action similarity judgments and highlight the importance of data-driven investigations of behavioral representations.

Meta-Learning for Zero-Shot Remote Sensing Image Super-Resolution

Zero-shot super-resolution (ZSSR) has generated a lot of interest due to its flexibility in various applications. However, the computational demands of ZSSR make it ineffective when dealing with large-scale low-resolution image sets. To address this issue, we propose a novel meta-learning model. We treat the set of low-resolution images as a collection of ZSSR tasks and learn meta-knowledge about ZSSR by leveraging these tasks. This approach reduces the computational burden of super-resolution for large-scale low-resolution images. Additionally, through multiple ZSSR task learning, we uncover a general super-resolution model that enhances the generalization capacity of ZSSR. Finally, using the learned meta-knowledge, our model achieves impressive results with just a few gradient updates when given a novel task. We evaluate our method using two remote sensing datasets with varying spatial resolutions. Our experimental results demonstrate that using multiple ZSSR tasks yields better outcomes than a single task, and our method outperforms other state-of-the-art super-resolution methods.

Confidence Screening Detector: A New Method for Detecting Test Collusion.

Test collusion (TC) is a form of cheating in which, examinees operate in groups to alter normal item responses. TC is becoming increasingly common, especially within high-stakes, large-scale examinations. However, research on TC detection methods remains scarce. The present article proposes a new algorithm for TC detection, inspired by variable selection within high-dimensional statistical analysis. The algorithm relies only on item responses and supports different response similarity indices. Simulation and practical studies were conducted to (1) compare the performance of the new algorithm against the recently developed clique detector approach, and (2) verify the performance of the new algorithm in a large-scale test setting.

A Parallel and Updatable Architecture for FPGA-Based Packet Classification With Large-Scale Rule Sets

As a programmable hardware, field-programmable gate array (FPGA) provides more opportunities for algorithmic network packet classification. Despite more than 10 years of research, the most actively investigated pipeline architectures still struggle to support fast rule search and efficient rule update for large-scale rule sets. In this article, we design and implement a novel architecture for multitree-based packet classification on FPGA, where the search and update processes are decoupled. A strategy of multi-processing elements (PEs), parallel search, and serial update is adopted. The parsing of multiple tree search results adopts a modular and hierarchical design, supporting architecture with various tree numbers. In addition, incremental rule updates can be achieved simply by traversing all PEs in one pass, with little and bounded impact on rule searching. Compared with TcbTree, the state-of-the-art updatable classifier, the experimental results on FPGA show that the classification performance of our design improves 3.4× on average for various 100k-scale rule sets.

DomBpred: Protein Domain Boundary Prediction Based on Domain-Residue Clustering Using Inter-Residue Distance.

Domain boundary prediction is one of the most important problems in the study of protein structure and function, especially for large proteins. At present, most domain boundary prediction methods have low accuracy and limitations in dealing with multi-domain proteins. In this study, we develop a sequence-based protein domain boundary prediction, named DomBpred. In DomBpred, the input sequence is first classified as either a single-domain protein or a multi-domain protein through a designed effective sequence metric based on a constructed single-domain sequence library. For the multi-domain protein, a domain-residue clustering algorithm inspired by Ising model is proposed to cluster the spatially close residues according inter-residue distance. The unclassified residues and the residues at the edge of the cluster are then tuned by the secondary structure to form potential cut points. Finally, a domain boundary scoring function is proposed to recursively evaluate the potential cut points to generate the domain boundary. DomBpred is tested on a large-scale test set of FUpred comprising 2549 proteins. Experimental results show that DomBpred better performs than the state-of-the-art methods in classifying whether protein sequences are composed by single or multiple domains, and the Matthew's correlation coefficient is 0.882. Moreover, on 849 multi-domain proteins, the domain boundary distance and normalised domain overlap scores of DomBpred are 0.523 and 0.824, respectively, which are 5.0% and 4.2% higher than those of the best comparison method, respectively. Comparison with other methods on the given test set shows that DomBpred outperforms most state-of-the-art sequence-based methods and even achieves better results than the top-level template-based method. The executable program is freely available at https://github.com/iobio-zjut/DomBpred and the online server at http://zhanglab-bioinf.com/DomBpred/.

Multi-Agent Deep Reinforcement Learning for Coordinated Energy Trading and Flexibility Services Provision in Local Electricity Markets

Local electricity markets (LEM) have recently attracted great interest as an effective solution to the challenging problem of distributed energy resources’ (DER) management. However, LEM designs combining the market functions of local energy trading and flexibility services (FS) provision to wider system operators have not attracted sufficient attention. In the context of addressing this research gap, this paper firstly provides a new model-based system-centric formulation for the coordination of such a LEM, which provides a theoretical optimality benchmark. Compared to previous formulations, it considers the time-coupling operating characteristics of flexible DERs, and optimizes the two market functions simultaneously. Furthermore, this paper explores for the very first time a model-free prosumer-centric coordination approach for such a LEM, in order to address the practical limitations of model-based system-centric approaches. This is achieved through a new multi-agent deep reinforcement learning method which combines the beneficial properties of the multi-actor-attention-critic and the prioritized experience replay approaches. Case studies on a real-world, large-scale setting validate that the proposed LEM design successfully encapsulates the economic benefits of both local energy trading and FS provision functions, and demonstrate that the proposed learning method outperforms previous methods.

Red light-driven electron sacrificial agents-free photoreduction of inert aryl halides via triplet-triplet annihilation

Selective photoactivation of inert aryl halides is a fundamental challenge in organic synthesis. Specially, the long-wavelength red light is more desirable than the widely-applied blue light as the excitation source for photoredox catalysis, due to its superior penetration depth. However, the long-wavelength red light-driven photoactivation of inert aryl halides remains a challenge, mainly because of the low energy of the single long-wavelength red photon. Herein, we report the photoreduction of aryl bromides/chlorides with 656 nm LED via triplet-triplet annihilation (TTA) strategy. This method is based on our discovery that the commonly used chromophore of perylene can serve as an efficient and metal-free photocatalyst to enable the photoreduction of inert aryl halides without the conventional need for electronic sacrificial agents. By introducing a red light-absorbing photosensitizer to this perylene system, we accomplish the long-wavelength red light-driven photoreduction of aryl halides via sensitized TTA mechanism. Moreover, the performance of such a TTA-mediated photoreduction can be significantly enhanced when restricting the rotation freedom of phenyl moiety for perylene derivatives to suppress their triplet nonradiative transition, in both small and large-scale reaction settings.

Computation Offloading Method for Large-Scale Factory Access in Edge-Edge Collaboration Mode

Large-scale manufacturing enterprises have complex business processes in their production workshops, and the edge-edge collaborative business model cannot adapt to the traditional computation offloading methods, which leads to the problem of load imbalance. For this problem, a computation offloading algorithm based on edge-edge collaboration mode for large-scale factory access is proposed, called the edge and edge collaborative computation offloading (EECCO) algorithm. First, the method partitions the directed acyclic graphs (DAGs) on edge server and terminal industrial equipment, then updates the tasks using a synchronization policy based on set theory to improve the accuracy effectively, and finally achieves load balancing through processor allocation. The experimental results show that the method shortens the processing time by improving computational resource utilization and employs a heterogeneous distributed system to achieve high computing performance when processing large-scale task sets.

A computationally effective time-restricted stability preserving [formula omitted]-optimal model order reduction approach

Several approaches for reducing model order on the definite time segments have become the topic of investigation in a series of papers that bring challenges during application in a large-scale setting. The subject of discussion of this paper is the computationally efficient time-restricted H2-optimal model order reduction method of higher dimensional sparse systems that requires the solutions of time-restricted Lyapunov and Sylvester equations. Our discussion is on developing the algorithms to solve these matrix equations that face difficulty when calculating the matrix exponential of the large-scale matrices. As a result, an efficient remedy is also proposed to compute the matrix exponential. Our ideas are also evaluated for index-1 descriptor systems apart from the generalized structure. Numerical analyses are conducted on several benchmark examples to illustrate how accurate and efficient our suggested approaches are by comparing them with the existing methods.

UnifiedFace: A Uniform Margin Loss Function for Face Recognition

Face recognition has achieved great success due to the development of deep convolutional neural networks (DCNNs) and loss functions based on margin. However, complex DCNNs bring a large number of parameters as well as computational effort, which pose a significant challenge to resource-constrained embedded devices. Meanwhile, the popular margin-based loss functions all introduce only one type of margin and cannot further introduce a larger margin to achieve tighter classification boundary. In contrast to the common approach, we believe that additive and multiplicative margins should be used jointly to introduce larger margins from the margin perspective. Therefore, we propose a new margin-based loss function called UnifiedFace. First, we introduce an additive margin in the target angle activation function. Second, we add a multiplicative margin in the non-target angle. UnifiedFace introduces both additive and multiplicative margins, allowing for the introduction of large margins to achieve more compact intra-class variance and closer separated inter-class variance. In addition, we specifically design efficient face recognition models called GhostFaceNet for resource-constrained embedded devices. Experimental results demonstrate that UnifiedFace achieves state-of-the-art performance or performance competed with popular methods in training datasets of different sizes. UnifiedFace achieves optimal performance in models of varying complexity. Moreover, competitive results are achieved in the large-scale test set IJBB/C, especially the state-of-the-art performance achieved in TAR (FAR=1e−6). GhostFaceNet can significantly improve operational efficiency without significantly degrading recognition performance, making it ideal for embedded devices with limited resources.

Group threat, political extremity, and collective dynamics in online discussions

Collectives adapt their network structure to the challenges they face. It has been hypothesized that collectives experiencing a real or imagined threat from an outgroup tend to consolidate behind a few group members, and that network structures in which a few members attract most of the attention are more likely in politically extreme groups. These hypotheses have not been tested in large-scale real-world settings. We reconstruct networks of tens of thousands of commenters participating in comment sections of high-profile U.S. political news websites spanning the political spectrum from left to right, including Mother Jones, The Atlantic, The Hill, and Breitbart. We investigate the relationship between different indices of inequality of attention in commenters’ networks and perceived group threat associated with significant societal events, from elections and political rallies to mass shootings. Our findings support the hypotheses that groups facing a real or imagined outgroup threat and groups that are more politically extreme are more likely to attend to a few high-profile members. These results provide an extensive real-world test of theoretical accounts of collective adaptation to outgroup threats.

Investigation on the use of ensemble learning and big data in crop identification

The agriculture sector in Egypt faces several problems, such as climate change, water storage, and yield variability. The comprehensive capabilities of Big Data (BD) can help in tackling the uncertainty of food supply occurs due to several factors such as soil erosion, water pollution, climate change, socio-cultural growth, governmental regulations, and market fluctuations. Crop identification and monitoring plays a vital role in modern agriculture. Although several machine learning models have been utilized in identifying crops, the performance of ensemble learning has not been investigated extensively. The massive volume of satellite imageries has been established as a big data problem forcing to deploy the proposed solution using big data technologies to manage, store, analyze, and visualize satellite data. In this paper, we have developed a weighted voting mechanism for improving crop classification performance in a large scale, based on ensemble learning and big data schema. Built upon Apache Spark, the popular DB Framework, the proposed approach was tested on El Salheya, Ismaili governate. The proposed ensemble approach boosted accuracy by 6.5%, 1.9%, 4.4%, 4.9%, 4.7% in precision, recall, F-score, Overall Accuracy (OA), and Matthews correlation coefficient (MCC) metrics respectively. Our findings confirm the generalization of the proposed crop identification approach at a large-scale setting.

Automated measurement of inter-arytenoid distance on 4D laryngeal CT: A validation study.

Changes to the voice are prevalent and occur early in Parkinson's disease. Correlates of these voice changes on four-dimensional laryngeal computed-tomography imaging, such as the inter-arytenoid distance, are promising biomarkers of the disease's presence and severity. However, manual measurement of the inter-arytenoid distance is a laborious process, limiting its feasibility in large-scale research and clinical settings. Automated methods of measurement provide a solution. Here, we present a machine-learning module which determines the inter-arytenoid distance in an automated manner. We obtained automated inter-arytenoid distance readings on imaging from participants with Parkinson's disease as well as healthy controls, and then validated these against manually derived estimates. On a modified Bland-Altman analysis, we found a mean bias of 1.52 mm (95% limits of agreement -1.7 to 4.7 mm) between the automated and manual techniques, which improves to a mean bias of 0.52 mm (95% limits of agreement -1.9 to 2.9 mm) when variability due to differences in slice selection between the automated and manual methods are removed. Our results demonstrate that estimates of the inter-arytenoid distance with our automated machine-learning module are accurate, and represents a promising tool to be utilized in future work studying the laryngeal changes in Parkinson's disease.

SHEAR saliva collection device augments sample properties for improved analytical performance.

Despite being a convenient clinical substrate for biomonitoring, saliva's widespread utilization has not yet been realized. The non-Newtonian, heterogenous, and highly viscous nature of saliva complicate the development of automated fluid handling processes that are vital for accurate diagnoses. Furthermore, conventional saliva processing methods are resource and/or time intensive precluding certain testing capabilities, with these challenges aggravated during a pandemic. The conventional approaches may also alter analyte structure, reducing application opportunities in point-of-care diagnostics. To overcome these challenges, we introduce the SHEAR saliva collection device that mechanically processes saliva, in a rapid and resource-efficient way. We demonstrate the device's impact on reducing saliva's viscosity, improving sample's uniformity, and increasing diagnostic performance of a COVID-19 rapid antigen test. Additionally, a formal user experience study revealed generally positive comments. SHEAR saliva collection device may support realization of the saliva's potential, particularly in large-scale and/or resource-limited settings for global and community diagnostics.

Federated multi-objective reinforcement learning

Multi-objective reinforcement Learning (MORL) has significant potential for solving complex decision problems with conflicting objectives. Desiring sufficient training samples, it is promising to achieve federated MORL in large-scale distributed settings. However, itstill suffers from poor efficiency and high privacy risks. To mitigate the inefficiency issue, we first propose a novel probablistic algorithm PMORL that can seek an optimal policy via the expectation maximization (EM) algorithm with high efficiency. To extend PMORL to distributed settings with privacy protection, we then present the first federated MORL algorithm Fed-PMORL with client-level differential privacy (DP). In Fed-PMORL, personalized actors are trained and maintained at local clients whereas critics are aggregated and sanitized at the central server. Extensive experimental results in benchmark MORL environments demonstrate that Fed-PMORL under DP guarantees can achieve superior performance with high efficiency. In particular, compared with the state-of-the-art methods, PMORL and Fed-PMORL can save up to 50% training episodes for achieving the same model utility. With a sufficient number of clients (e.g., 1000 clients), Fed-PMORL with a formal DP guarantee shows utility comparable to that of the non-private algorithm.

Stochastic Compositional Gradient Descent Under Compositional Constraints

This work studies constrained stochastic optimization problems where the objective and constraint functions are convex and expressed as compositions of stochastic functions. The problem arises in the context of fair classification, fair regression, and the design of queuing systems. Of particular interest is the large-scale setting where an oracle provides the stochastic gradients of the constituent functions, and the goal is to solve the problem with a minimal number of calls to the oracle. Owing to the compositional form, the stochastic gradients provided by the oracle do not yield unbiased estimates of the objective or constraint gradients. Instead, we construct approximate gradients by tracking the inner function evaluations, resulting in a quasi-gradient saddle point algorithm. We prove that the proposed algorithm is guaranteed to find the optimal and feasible solution almost surely. We further establish that the proposed algorithm requires <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(1/\epsilon ^{4})$</tex-math></inline-formula> data samples in order to obtain an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula> -approximate optimal point while also ensuring zero constraint violation. The result matches the sample complexity of the stochastic compositional gradient descent method for unconstrained problems and improves upon the best-known sample complexity results for the constrained settings. The efficacy of the proposed algorithm is tested on both fair classification and fair regression problems. The numerical results show that the proposed algorithm outperforms the state-of-the-art algorithms in terms of the convergence rate.

Multi-Objective Dynamic Software Project Scheduling: A Novel Approach to Handle Employee’s Addition

The software project scheduling (SPS) problem deals with optimal allocation of employees to tasks such that the project is completed within budget and schedule. This is well addressed in the absence of dynamic events and has been demonstrated in SPS research with the static scenarios only. The software development methodologies have evolved where customer requirements are not initially locked but are flexible to take into account ongoing additional requirements or changes. In this regard, ‘new employee addition’ has emerged as one of the critical dynamic events that could significantly disrupt the project budget and schedule. In this paper, the SPS is modelled as multi-objective optimization problem and a novel approach is presented to deal with ‘new employee addition’ dynamic event; that uses domain knowledge to generate a robust schedule. The proposed heuristic is further evaluated on 18 dynamic benchmark scenarios and 3 real-world instances. The results show that the proposed heuristic besides its capability to handle dynamic events also demonstrates good convergence while maintaining good distribution of solutions in a dynamic environment. The principal target audience for this research are project managers in software development organizations who will get optimal project reschedule under ‘new employee addition’ dynamic event particularly in complex, large scale and multi-project settings.

Play to get Instant Feedback – Using Game-Based Apps to Overcome the Feedback Gap in Lectures and Online Teaching

COVID-19 accelerated the digital transformation in education worldwide. Within a short time frame of several weeks, a full digital transformation took place. In the meantime, however, the trend of this rapid digital transformation seems to be reverting and rolling back to face-to-face teaching and traditional teaching methods. In this article, we aim to show that digital teaching approaches cannot be exclusive to online teaching but should be considered important even after returning to face-to-face teaching. In particular, we focus on improving feedback in large-scale educational settings. We observe that providing and receiving appropriate feedback on learning and teaching activities is challenging independent of the specific teaching setting (in-class or online). This is particularly problematic because feedback is essential in the learning processes. To address this, we combine the concepts of learning analytics and gamification to design a lecture game app that can be used (1) to motivate students to participate in formative assessments, (2) to collect learning-related data, and (3) to analyze this data to provide instant feedback. After integrating the lecture game in different settings, we conclude that digital tools can be useful independent of the specific learning setting.