General Process Research Articles

Learning obstacles in the generalization process: In case number pattern topic

Generalization plays an important role in mathematics because it is considered inherent in mathematical thinking in general. Where number pattern is a topic that is closely related to generalization. Problems are still found in generalizing number patterns so this can cause learning obstacles. This research aims to identify learning obstacles for Junior high school students in the generalization process with a focus on number pattern topics. This research is qualitative research with a case study method. The research subjects consisted of 30 grade 8 students in Jakarta who had studied number pattern. Subjects were given three test tasks, the results of which were used to figure out learning obstacles in the generalization process experienced by students and then continued with interviews with ten subjects. The findings show that learning obstacles in the pattern generalization process occur primarily at the expression and symbolic stages. Most students bypassed the generalization method, relying instead on formulas or manual calculations. Based on these results, it is recommended that educators implement didactic designs that offer targeted interventions during the expression and symbolic stages, such as using more interactive and symbolic reasoning activities to strengthen students' understanding and generalization skills in number patterns.

Spectroscopy-Guided Discovery of Three-Dimensional Structures of Disordered Materials with Diffusion Models

Abstract Spectroscopy techniques such as X-ray absorption near edge structure (XANES) provide valuable insights into the atomic structures of materials, yet the inverse prediction of precise structures from spectroscopic data remains a formidable challenge. In this study, we introduce a framework that combines generative artificial intelligence (AI) models with XANES spectroscopy to predict three-dimensional atomic structures of disordered systems, using amorphous carbon ($a$-C) as a model system. In this work, we introduce a new framework based on the diffusion model, a recent generative machine learning method, to predict 3D structures of disordered materials from a target property.For demonstration, we apply the model to identify the atomic structures of $a$-C as a representative material system from the target XANES spectra. We show that conditional generation guided by XANES spectra reproduces key features of the target structures. Furthermore, we show that our model can steer the generative process to tailor atomic arrangements for a specific XANES spectrum. Finally, our generative model exhibits a remarkable scale-agnostic property, thereby enabling generation of realistic, large-scale structures through learning from a small-scale dataset (i.e., with small unit cells). Our work represents a significant stride in bridging the gap between materials characterization and atomic structure determination; in addition, it can be leveraged for materials discovery in exploring various material properties as targeted.

Controllable Anime Image Editing via Probability of Attribute Tags

AbstractEditing anime images via probabilities of attribute tags allows controlling the degree of the manipulation in an intuitive and convenient manner. Existing methods fall short in the progressive modification and preservation of unintended regions in the input image. We propose a controllable anime image editing framework based on adjusting the tag probabilities, in which a probability encoding network (PEN) is developed to encode the probabilities into features that capture continuous characteristic of the probabilities. Thus, the encoded features are able to direct the generative process of a pre‐trained diffusion model and facilitate the linear manipulation. We also introduce a local editing module that automatically identifies the intended regions and constrains the edits to be applied to those regions only, which preserves the others unchanged. Comprehensive comparisons with existing methods indicate the effectiveness of our framework in both one‐shot and linear editing modes. Results in additional applications further demonstrate the generalization ability of our approach.

Stochastic collapse: how gradient noise attracts SGD dynamics towards simpler subnetworks*

Abstract In this work, we reveal a strong implicit bias of stochastic gradient descent (SGD) that drives overly expressive networks to much simpler subnetworks, thereby dramatically reducing the number of independent parameters, and improving generalization. To reveal this bias, we identify invariant sets, or subsets of parameter space that remain unmodified by SGD. We focus on two classes of invariant sets that correspond to simpler (sparse or low-rank) subnetworks and commonly appear in modern architectures. Our analysis uncovers that SGD exhibits a property of stochastic attractivity towards these simpler invariant sets. We establish a sufficient condition for stochastic attractivity based on a competition between the loss landscape’s curvature around the invariant set and the noise introduced by stochastic gradients. Remarkably, we find that an increased level of noise strengthens attractivity, leading to the emergence of attractive invariant sets associated with saddle-points or local maxima of the train loss. We observe empirically the existence of attractive invariant sets in trained deep neural networks, implying that SGD dynamics often collapses to simple subnetworks with either vanishing or redundant neurons. We further demonstrate how this simplifying process of stochastic collapse benefits generalization in a linear teacher-student framework. Finally, through this analysis, we mechanistically explain why early training with large learning rates for extended periods benefits subsequent generalization.

Temporal error monitoring: Does agency matter?

Error monitoring is the ability to report one's errors without relying on feedback. Although error monitoring is investigated mostly with choice tasks, recent studies have discovered that participants parametrically also keep track of the magnitude and direction of their temporal, spatial, and numerical judgment errors. We investigated whether temporal error monitoring relies on internal generative processes that lead to the to-be-judged first-order timing performance. We hypothesized that if the endogenous processes underlie temporal error monitoring, one can monitor timing errors in emitted but not observed timing behaviors. We conducted six experiments to test this hypothesis. The first two experiments showed that confidence ratings were negatively related to error magnitude only in emitted behaviors, but error directionality judgments of observed behaviors were more precise. Experiment 3 replicated these effects even after controlling for the motor aspects of first-order timing performance. The last three experiments demonstrated that belief of agency (i.e., believing that the error belongs to the self or someone else) was critical in accounting for the confidence rating effects observed in the first two experiments. The precision of error directionality judgments was higherin thenon-agency condition. These results show that confidence is sensitive to belief, and short-long judgment is sensitive to the actual agency of timing behavior (i.e., whether the behavior was emitted by the self or someone else).

Overgeneralization of autonomic defensive reactions in obesity

Generalizing defensive responses to new stimuli resembling learned threats is an adaptive process within an ever-changing environment. However, evaluation mechanisms excessively biased toward generalization (i.e., overgeneralization) may underlie anxiety-related symptoms. In the context of obesity, fear memory and fear generalization processes have never been investigated. In this study, participants with obesity and healthy participants as controls underwent a single-cue auditory fear conditioning paradigm and recognition memory tasks. We analyzed the autonomic reactions evoked by threat-predictive and new stimuli, as well as the recognition performance towards the same cues. We found that participants with obesity displayed similar autonomic defensive responses to a learned fearful stimulus, but enhanced reactions to new stimuli, when compared with the controls. We detected no significant differences between groups in recognition abilities. Our results provided the first evidence that obesity may widen fear generalization patterns. This alteration may encourage future research in investigating the link between emotional dysregulation and clinical anxiety-related symptoms in obesity.

Life cycle assessment and generative design: development of a national LCA tool for exterior walls

PurposeThe promotion of sustainable design is demanded globally. The life cycle assessment (LCA) proved its reliability in this mission, but the difficulty and time required to apply it discouraged designers. This research aims to integrate LCA into the building design process through a software tool, taking advantage of generative design features. This will facilitate decision-making by architects and construction professionals.Design/methodology/approachThe study develops the EGY-LCA (http://egy-lca.com/). This prototype tool suggests exterior wall design alternatives for residential buildings in Egypt, using the environmental impact indicators of LCA data and other criteria related to national codes, materials, construction methods and required thermal resistance. Within a generative design process, the algorithm tests every possible wall method with materials and thickness combinations for each layer in compliance with inputs. The paper begins by explaining the tool’s working method. Afterward, different sets of inputs are examined and the values of the resultant environmental impacts of several suggested wall solutions are statistically analyzed. The application demonstrates the importance of the generative design tool. Proposing several solutions based on a set of inputs facilitates the selection of sustainable choices and allows comparisons between alternatives.FindingsThe prototype experiment confirms the research hypothesis. Unlike the available LCA tools, architects can make decisions with limited LCA experience if the data and equations are integrated into a generative design tool. The prototype proves its applicability for exterior wall alternatives.Research limitations/implicationsThe prototype is the initial step toward a whole-building LCA tool. It includes limited LCA stages and materials for the external wall. Future research is required to expand this parametric tool concept to include all the building components. The framework in Section 5 proposes a visualization.Practical implicationsThe prototype tool: EGY-LCA (http://egy-lca.com/). The value added to the design and construction sectors through the uncomplicated LCA application is fostering sustainable design, generative design tools can achieve this.Originality/valueThe novelty of this work is that it is the first initiative offering a parametric LCA tool. It promotes the application of LCA at the design stage using generative design, which contributes to sustainable development.

Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning.

Different brain systems have been hypothesized to subserve multiple "experts" that compete to generate behavior. In reinforcement learning, two general processes, one model-free (MF) and one model-based (MB), are often modeled as a mixture of agents (MoA) and hypothesized to capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by a static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from a set of agents and the temporal dynamics of underlying "hidden" states that capture shifts in agent contributions over time. Applying this model to a multi-step, reward-guided task in rats reveals a progression of within-session strategies: a shift from initial MB exploration to MB exploitation, and finally to reduced engagement. The inferred states predict changes in both response time and OFC neural encoding during the task, suggesting that these states are capturing real shifts in dynamics.

Mitigating Adversarial Attacks in Object Detection through Conditional Diffusion Models

The field of object detection has witnessed significant advancements in recent years, thanks to the remarkable progress in artificial intelligence and deep learning. These breakthroughs have significantly enhanced the accuracy and efficiency of detecting and categorizing objects in digital images. Nonetheless, contemporary object detection technologies have certain limitations, such as their inability to counter white-box attacks, insufficient denoising, suboptimal reconstruction, and gradient confusion. To overcome these hurdles, this study proposes an innovative approach that uses conditional diffusion models to perturb adversarial examples. The process begins with the application of a random chessboard mask to the adversarial example, followed by the addition of a slight noise to fill the masked area during the forward process. The adversarial image is then restored to its original form through a reverse generative process that only considers the masked pixels, not the entire image. Next, we use the complement of the initial mask as the mask for the second stage to reconstruct the image once more. This two-stage masking process allows for the complete removal of global disturbances and aids in image reconstruction. In particular, we employ a conditional diffusion model based on a class-conditional U-Net architecture, with the source image further conditioned through concatenation. Our method outperforms the recently introduced HARP method by 5% and 6.5% in mAP on the COCO2017 and PASCAL VOC datasets, respectively, under non-APT PGD attacks. Comprehensive experimental results confirm that our method can effectively restore adversarial examples, demonstrating its practical utility.

Structural optimization of hydraulic valve block based on generative design and selective laser melting process simulation

Structural optimization of hydraulic valve block based on generative design and selective laser melting process simulation

SplineGen: Approximating unorganized points through generative AI

This paper presents a learning-based method to solve the traditional parameterization and knot placement problems in B-spline approximation. Different from conventional heuristic methods or recent AI-based methods, the proposed method does not assume ordered or fixed-size data points as input. There is also no need for manually setting the number of knots. Parameters and knots are generated in an associative way to attain better parameter-knot alignment, and therefore a higher approximation accuracy. These features are attained by using a new generative model SplineGen, which casts the parameterization and knot placement problems as a sequence-to-sequence translation problem. It first adopts a shared autoencoder model to learn a 512-D embedding for each input point, which has the local neighborhood information implicitly captured. Then these embeddings are autoregressively decoded into parameters and knots by two associative decoders, a generative process automatically determining the number of knots, their placement, parameter values, and their ordering. The two decoders are made to work in a coordinated manner by a new network module called internal cross-attention. Once trained, SplineGen demonstrates a notable improvement over existing methods, with one to two orders of magnitude increase in approximation accuracy on test data.

Computational Design with Kurtboğaz: The Generation of Timber Structures with an Aggregative Design Algorithm

This paper investigates the form-finding capacity of the traditional timber-joint construction method, Kurtboğaz, aiming to explore new architectural forms and possibilities through computational design techniques to preserve vernacular construction methods and integrate them into contemporary architecture. It presents an Aggregative Design Algorithm (ADA) that creates different structures based on designer rules and simple assembly rules of Kurtboğaz, leading to unique emergent forms through random rule application. The paper also explores how reinforcement learning, a type of machine learning, can improve this design process through a theoretical framework. The study tries to use a rule-based generative algorithm to explore the modular and reconfigurable characteristics of the Kurtboğaz. The ADA enables random rule application, leading to diverse forms. However, several challenges may be encountered during the application of ADA because of its random aggregation, such as collision avoidance, structural integrity, boundary detection, and the optimization of structural parameters. The study suggests using Reinforcement Learning (RL) in the ADA framework to address these problems. Incorporating RL is anticipated to enable the algorithm to adaptively learn and optimize the form-finding process, enhancing the performance and applicability of the Kurtboğaz method in contemporary architectural practice. In the future, with this generative process described by the study, designs that create spatial differences with the help of walls, floors, and rooms on a human scale can be realized. The study also plans to explore the synergy between craftsmanship and digital fabrication in the future

Introducing bidirectional attention for autoregressive models in abstractive summarization

Abstractive summarization methods typically follow the autoregressive paradigm using the causal masks in the decoder for training and inference efficiency. However, this approach leads to a constant context throughout the generation process, which conflicting with the bidirectional characteristics of natural language. Although previous attempts have been made to incorporate bidirectional attention in the decoding process through non-autoregressive approach, the evaluation results are not comparable to the autoregressive methods. To bring bidirectional attention to the autoregressive process while maintaining superior performance, we propose the global autoregressive paradigm, which takes the outputs of the autoregressive process as additional inputs in the subsequent global iteration. Specifically, we build a bidirectional decoder alongside the original encoder and decoder to capture the bidirectional context of the outputs. This context is updated after each autoregressive decoding iteration. The decoder then integrates the updated context into subsequent autoregressive decoding steps, enhancing the generative process with a more comprehensive and authentic context. Additionally, we use contrastive learning to train the model to extract reliable features from the bidirectional context and apply reinforcement learning to improve the model's utilization of this context. We evaluate our method on CNN/DM, XSum, and NYT datasets, and the results highlight the significance of the bidirectional context. Our method achieves the best performance in terms of ROUGE-2 on CNN/DM (23.96), and performs comparably on XSum (25.45) and NYT (27.91). It also outperforms all the baselines in terms of BERTScore, with a score of 89.96 on CNN/DM, 92.70 on XSum, and 90.04 on NYT. Furthermore, our method can perform better with a larger beam size.

EXPRESS: Stratified Distributional Analysis - a Novel Perspective on RT Distributions.

Response times and their distributions serve as a powerful lens into cognitive processes. We present a novel statistical methodology called Stratified Distributional Analysis (SDA) to quantitatively assess how key determinants of response times (word frequency and length) shape their distributions. Taking advantage of the availability of millions of lexical decision response times in the English Lexicon Project and the British Lexicon Project, we made important advances into the theoretical issue of linking response times and word frequency by analysing RT distributions as a function of word frequency and word length. We tested these distributions against the lognormal, Wald, and Gamma distributions and three measures of word occurrence (word form frequencies obtained from subtitles and contextual diversity as operationalized as discourse contextual diversity and user contextual diversity). We found that the RT distributions were best described by a lnorm distribution across both megastudies when word occurrence was quantified by a contextual diversity measure. The link between the lnorm distribution and its generative process highlights the power of SDA in elucidating mechanisms that govern the generation of RTs through the fitting of probability distributions. Using a hierarchical Bayesian framework, SDA yielded posterior distributions for the distributional parameters at the single-participant level, enabling probabilistic predictions of response times as a function of word frequency and word length, which has the potential to serve as a diagnostic tool to uncover idiosyncratic features of word processing. Crucially, while we applied our parsimonious methodology to lexical decision response times, it is applicable to a variety of tasks such as word-naming and eye-tracking data.

Assembly Theory is an approximation to algorithmic complexity based on LZ compression that does not explain selection or evolution

We formally prove the equivalence between Assembly Theory (AT) and Shannon Entropy via a method based upon the principles of statistical compression that belongs to the LZ family of popular compression algorithms. Such popular lossless compression algorithms behind file formats such as ZIP and PNG have been shown to empirically reproduce the results that AT considers its cornerstone. The same results have also been reported before AT in successful application of other complexity measures in the areas covered by AT such as separating organic from non-organic molecules and in the context of the study of selection and evolution. We demonstrate that the assembly index is equivalent to the size of a minimal context-free grammar. The statistical compressibility of such a method is bounded by Shannon Entropy and other equivalent traditional LZ compression schemes, such as LZ77 and LZW. We also demonstrate that AT, and the algorithms supporting its pathway complexity, assembly index, and assembly number, define compression schemes and methods that are subsumed into algorithmic information theory. We conclude that the assembly index and the assembly number do not lead to an explanation or quantification of biases in generative (physical or biological) processes, including those brought about by (abiotic or biotic) selection and evolution, that could not have been arrived at using Shannon Entropy, or that have not been already reported before using classical information theory or algorithmic complexity.

Humans flexibly integrate social information despite interindividual differences in reward

There has been much progress in understanding human social learning, including recent studies integrating social information into the reinforcement learning framework. Yet previous studies often assume identical payoffs between observer and demonstrator, overlooking the diversity of social information in real-world interactions. We address this gap by introducing a socially correlated bandit task that accommodates payoff differences among participants, allowing for the study of social learning under more realistic conditions. Our Social Generalization (SG) model, tested through evolutionary simulations and two online experiments, outperforms existing models by incorporating social information into the generalization process, but treating it as noisier than individual observations. Our findings suggest that human social learning is more flexible than previously believed, with the SG model indicating a potential resource-rational trade-off where social learning partially replaces individual exploration. This research highlights the flexibility of humans' social learning, allowing us to integrate social information from others with different preferences, skills, or goals.

Causal representation learning through higher-level information extraction

The large gap between the generalization level of state-of-the-art machine learning and human learning systems calls for the development of artificial intelligence (AI) models that are truly inspired by human cognition. In tasks related to image analysis, searching for pixel-level regularities has reached a power of information extraction still far from what humans capture with image-based observations. This leads to poor generalization when even small shifts occur at the level of the observations. We explore a perspective on this problem that is directed to learning the generative process with causality-related foundations, using models capable of combining symbolic manipulation, probabilistic reasoning and pattern recognition abilities. We briefly review and explore connections of research from machine learning, cognitive science and related fields of human behavior to support our perspective for the direction to more robust and human-like artificial learning systems.

On the Ethics of Mediating Embodied Vulnerability to Violence

Media ethics has long been haunted by the question of representing human beings’ vulnerability to violence. While journalism and photojournalism have an obligation to report on the realities of violence and suffering in the world, the “spectacle of suffering” is fraught with ethical dilemmas. In this essay, I seek to theorize the ethics of vulnerability to violence in media representation. As a starting point, I argue for the politics of embodiment as a generative process that constitutes differential vulnerabilities. I move then to consider the way embodied vulnerabilities play out in the media, as exemplified by recent events such as the Black Lives Matter and MeToo movements as well as in times of war, from Vietnam to, more recently, Ukraine and Israel/Palestine. This leads to considerations of spectatorship: who looks and who is looked at? How are these relations of gazing related to the vectors of social and geopolitical power? Are images of embodied vulnerability simply media spectacles that reinforce power hierarchies, or are they powerful prosocial messages that might mobilize humanitarian activism? To address these epistemic questions, I propose that the feminist ethics of care encompasses an invitational rhetoric that can guide media praxis. Care ethics is aligned with various religious epistemologies, and because of that, I argue for it as an umbrella framework that has application in a variety of national and cultural contexts.

Knowledge‐Guided Automated Cartographic Generalization Process Construction: A Case Study Based on Map Analysis of Public Maps of China

ABSTRACTThe efficacy of conveying information through maps heavily depends on the quality of map generalization. However, automating map generalization poses a complex decision‐making challenge, requiring a profound understanding of the process—specifically, knowledge about the generalization procedure. Currently, there is a scarcity of research on the sequence of generalization operations, particularly for cartographic generalization involving symbolization and labeling. On the contrary, customary maps generated in practical applications consistently adhere to the specified generalization and symbolization protocol, which makes it feasible and credible to construct this overall process based on expert knowledge. To reconcile this incongruity, this paper presents a knowledge‐guided automated cartographic generalization process construction. Firstly, an exhaustive examination of the sequential procedures involved in manual generalization and a well‐applied automated generalization system are delineated, drawing upon map analysis methodologies, observations, and expert interviews. Then, elaborate guidelines governing each phase within this process, particularly concerning the symbolization and labeling of map features, are explored. Ultimately, details of the expert interview are described and a map generalized by the well‐applied system is analyzed. The results show that the automated generalization system follows the knowledge‐guided process in this paper can significantly improve production efficiency in practice, this study serves as a connection between cartographers and developers and may help achieve a higher level of automated cartographic generalization.

A New Zero Waste Design for a Manufacturing Approach for Direct-Drive Wind Turbine Electrical Generator Structural Components

An integrated structural optimization strategy was produced in this study for direct-drive electrical generator structures of offshore wind turbines, implementing a design for an additive manufacturing approach, and using generative design techniques. Direct-drive configurations are widely implemented on offshore wind energy systems due to their high efficiency, reliability, and structural simplicity. However, the greatest challenge associated with these types of machines is the structural optimization of the electrical generator due to the demanding operating conditions. An integrated structural optimization strategy was developed to assess a 100-kW permanent magnet direct-drive generator structure. Generated topologies were evaluated by performing finite element analyses and a metal additive manufacturing process simulation. This novel approach assembles a vast amount of structural information to produce a fit-for-purpose, adaptative, optimization strategy, combining data from static structural analyses, modal analyses, and manufacturing analyses to automatically generate an efficient model through a generative iterative process. The results obtained in this study demonstrate the importance of developing an integrated structural optimization strategy at an early phase of a large-scale project. By considering the typical working condition loads and the machine’s dynamic behavior through the structure’s natural frequencies during the optimization process coupled with a design for an additive manufacturing approach, the operational range of the wind turbine was maximized, the overall costs were reduced, and production times were significantly diminished. Integrating the constraints associated with the additive manufacturing process into the design stage produced high-efficiency results with over 23% in weight reduction when compared with conventional structural optimization techniques.