Extended Feature Space Research Articles

Extended Features Based Random Vector Functional Link Network for Classification Problem

Random vector functional link (RVFL) network has been successfully employed in diverse domains such as computer vision and machine learning, due to its universal approximation capability. Recently, the shallow RVFL architecture has been extended to deep architectures. In deep architectures, multiple hidden layers are stacked for extracting informative features from the original feature space. Therefore, having rich features, deep models are very successful compared to shallow models. In this article, we propose an extended feature RVFL (efRVFL) model that is trained over extended feature space generated analytically from the original feature space. The proposed efRVFL model has three types of features, i.e., original features, supervised randomized (newly generated) features, and unsupervised randomized features, in its feature matrix. The proposed efRVFL model with additional features has capability to capture nonlinear hidden relationships within the dataset. The proposed efRVFL model is an unstable classifier, and thus, its performance can be improved further via ensemble learning. Ensemble models are stable and accurate and have better generalization performance than single models. Therefore, we also propose an ensemble of extended feature RVFL (en-efRVFL) model. Each base model of en-efRVFL is trained over different feature spaces so that more accurate and diverse base models can be generated. The outcome of the base models is integrated via average voting scheme. Empirical evaluation over <inline-formula> <tex-math notation="LaTeX">$46$</tex-math> </inline-formula> UCI classification datasets demonstrates that the proposed efRVFL and en-efRVFL models have better performance than RVFL and other given deep models. Furthermore, the experimental results over <inline-formula> <tex-math notation="LaTeX">$12$</tex-math> </inline-formula> sparse datasets show that the proposed en-efRVFL model has a winning performance among several deep feedforward neural networks (FNNs).

Scalable kernel balancing weights in a nationwide observational study of hospital profit status and heart attack outcomes.

Weighting is a general and often-used method for statistical adjustment. Weighting has two objectives: first, to balance covariate distributions, and second, to ensure that the weights have minimal dispersion and thus produce a more stable estimator. A recent, increasingly common approach directly optimizes the weights toward these two objectives. However, this approach has not yet been feasible in large-scale datasets when investigators wish to flexibly balance general basis functions in an extended feature space. To address this practical problem, we describe a scalable and flexible approach to weighting that integrates a basis expansion in a reproducing kernel Hilbert space with state-of-the-art convex optimization techniques. Specifically, we use the rank-restricted Nyström method to efficiently compute a kernel basis for balancing in nearly linear time and space, and then use the specialized first-order alternating direction method of multipliers to rapidly find the optimal weights. In an extensive simulation study, we provide new insights into the performance of weighting estimators in large datasets, showing that the proposed approach substantially outperforms others in terms of accuracy and speed. Finally, we use this weighting approach to conduct a national study of the relationship between hospital profit status and heart attack outcomes in a comprehensive dataset of 1.27 million patients. We find that for-profit hospitals use interventional cardiology to treat heart attacks at similar rates as other hospitals but have higher mortality and readmission rates.

GMM-Based Expanded Feature Space as a Way to Extract Useful Information for Rare Cell Subtypes Identification in Single-Cell Mass Cytometry.

Cell subtype identification from mass cytometry data presents a persisting challenge, particularly when dealing with millions of cells. Current solutions are consistently under development, however, their accuracy and sensitivity remain limited, particularly in rare cell-type detection due to frequent downsampling. Additionally, they often lack the capability to analyze large data sets. To overcome these limitations, a new method was suggested to define an extended feature space. When combined with the robust clustering algorithm for big data, it results in more efficient cell clustering. Each marker's intensity distribution is presented as a mixture of normal distributions (Gaussian Mixture Model, GMM), and the expanded space is created by spanning over all obtained GMM components. The projection of the initial flow cytometry marker domain into the expanded space employs GMM-based membership functions. An evaluation conducted on three established cellular identification algorithms (FlowSOM, ClusterX, and PARC) utilizing the most substantial publicly available annotated dataset by Samusik et al. demonstrated the superior performance of the suggested approach in comparison to the standard. Although our approach identified 20 cell clusters instead of the expected 24, their intra-cluster homogeneity and inter-cluster differences were superior to the 24-cluster FlowSOM-based solution.

Developing a machine learning based building energy consumption prediction approach using limited data: Boruta feature selection and empirical mode decomposition

Artificial Intelligence methods (AI) have been widely applied in building energy consumption prediction. As data-intensive methods, lacking sufficient input features will significantly impede prediction performance. For some buildings where the building energy management systems (BEMs) are underperformed, limited information can be extracted. In this study, a feature engineering framework that combines feature construction and selection was developed to deal with limited feature problems. Empirical mode decomposition and Boruta feature selection were applied with the purpose of generating new informative features and selecting all relevant features, respectively. The proposed strategy was then tested using some popular machine learning algorithms for three different buildings. The results indicated that the proposed strategy was able to extend the feature dimensions and determine all relevant features from the extended feature space, which resulted to a significant improvement in the prediction performance. Unlike most other existing studies whereby observed performance enhancements may be marginal and restricted to few of the tested algorithms, the features selected here consistently improved the outcomes of all the machine learning algorithms tested for all 3 buildings.

Online Microfluidic Droplets Characterization Using Microscope Data Intelligent Analysis

Microfluidic devices have opened new opportunities for functional material chemical synthesis in a few applications. The screening of microfluidic synthesis processes is an urgent task of the experimental process in terms of automation and intellectualization. This study proposes a methodology and software for extracting the morphological and dynamic characteristics of generated monodisperse droplets from video data streams obtained from a digital microscope. For this purpose, the paper considers an approach to generating an extended feature space characterizing the process of droplet generation using a microfluidic device based on the creation of synthetic image datasets. YOLOv7 was used as an algorithm for detecting objects in the images. When training this algorithm, the values in the test dataset mAP@0.5 0.996 were obtained. The algorithms proposed for image processing and analysis implement the basic functionality to extract the morphological and dynamic characteristics of monodisperse droplets in the synthesis process. Laboratory validation and verification of the software demonstrated high results of the identification of key characteristics of the monodisperse droplets generated by the microfluidic device with the average deviation from the real values not exceeding 8%.

Melanoma Detection Based on Hybridization of Extended Feature Space

Melanoma Detection Based on Hybridization of Extended Feature Space

LSTSVM classifier with enhanced features from pre-trained functional link network

In this paper, we propose an improved model for the classification problems. We use least squares twin support vector machines (LSTSVM) and pre-trained functional link to enhance the feature space. LSTSVM algorithm is used in many real world classification problems as it has lower computational complexity and solves system of linear equations instead of solving quadratic programming problems (QPPs). Since neural network models provide implicit feature representation and is one of the reasons for the success of neural networks. Here, we propose a model wherein the input feature space is enhanced by the pre-trained functional link network. Weights are generated by LSTSVM, and a non-linear function is applied on the product between input features and the weights to get the enhanced features. These features are concatenated with the input features to get the extended feature space. Final classification is done by LSTSVM based on these extended features. Numerical experiments and statistical tests conducted show that the proposed model outperforms the baseline methods.

The extrapolation effect: an illusory experience of extended feature space beyond reality

The extrapolation effect: an illusory experience of extended feature space beyond reality

Detecting dwelling destruction in Darfur through object-based change analysis of very high-resolution imagery

ABSTRACTTimely and detailed information on the situation in conflict areas is essential to monitor the impact of conflicts on civilians and to document human rights issues, such as large scale displacement. Remote sensing provides valuable means for conflict monitoring, especially in areas where the ground-based documentation of violence is hampered, e.g. by the limited access to or persistent insecurity of conflict zones. The manual analysis of remote-sensing data is time consuming and labour intensive, but automatic methods can increase the efficiency of corresponding workflows, if the required user interference is minimized. In this study, the use of object-based change analysis for the automatic and selective detection of destructed dwellings in bi-temporal images is explored in test areas in Darfur. The presented approach automatically determines areas of interest (settlements), and detects changes in those areas by analysis of two change features (change of edge intensity and spectral change). It applies automatically defined local reference values and thresholds of these change features to reduce the required user interference. In addition, the extended feature space in the object-based approach (including, e.g. shape, size, and relational features in addition to spectral properties) is used to distinguish destructed dwellings from other, similarly changed objects. The developed method was applied to two study areas using images from three different sensors (GeoEye-1, WorldView-2, and QuickBird) without adaptation of the thresholds or rule sets. This resulted in a producer’s accuracy of 75.4% in the first and 81.2% in the second study area. The achieved user’s accuracy was 73.3% in study area 1 and 77.2% in study area 2. The evaluation of the results shows that the automatic calculation of local reference values and thresholds for the change detection can increase robustness when the proposed method is applied on study areas with different image properties. It also demonstrates the advantages as well as the specific constraints of using object- and context-specific features in this use case for the extraction of a certain structure type on a high level of detail.

Large-area settlement pattern recognition from Landsat-8 data

The study presents an image processing and analysis pipeline that combines object-based image analysis with a Support Vector Machine to derive a multi-layered settlement product from Landsat-8 data over large areas. 43 image scenes are processed over large parts of Central Asia (Southern Kazakhstan, Kyrgyzstan, Tajikistan and Eastern Uzbekistan). The main tasks tackled by this work include built-up area identification, settlement type classification and urban structure types pattern recognition. Besides commonly used accuracy assessments of the resulting map products, thorough performance evaluations are carried out under varying conditions to tune algorithm parameters and assess their applicability for the given tasks. As part of this, several research questions are being addressed. In particular the influence of the improved spatial and spectral resolution of Landsat-8 on the SVM performance to identify built-up areas and urban structure types are evaluated. Also the influence of an extended feature space including digital elevation model features is tested for mountainous regions. Moreover, the spatial distribution of classification uncertainties is analyzed and compared to the heterogeneity of the building stock within the computational unit of the segments. The study concludes that the information content of Landsat-8 images is sufficient for the tested classification tasks and even detailed urban structures could be extracted with satisfying accuracy. Freely available ancillary settlement point location data could further improve the built-up area classification. Digital elevation features and pan-sharpening could, however, not significantly improve the classification results. The study highlights the importance of dynamically tuned classifier parameters, and underlines the use of Shannon entropy computed from the soft answers of the SVM as a valid measure of the spatial distribution of classification uncertainties.

Semi-supervised classification method through oversampling and common hidden space

Semi-supervised classification methods attempt to improve classification performance based on a small amount of labeled data through full use of abundant unlabeled data. Although existing semi-supervised classification methods have exhibited promising results in many applications, they still have drawbacks, including performance degeneration, due to the introduction of unlabeled data and partially false labels in a small amount of labeled data. To circumvent such drawbacks, a new semi-supervised classification method OCHS-SSC through oversampling and a common hidden space is proposed in the paper. The primary characteristics of the proposed method include two aspects. One is that unlabeled data are only used to generate new synthetic data to extend the minimal amount of labeled data. The other is that the final classifier is learned in the extended feature space, which is composed of the original feature space and the common hidden space found between labeled data and the synthetic data instead of the original feature space. Extensive experiments on 23 datasets indicate the effectiveness of the proposed method.

Mapping megacity growth with multi-sensor data

Abstract In our increasingly urbanized world, monitoring and mapping of urban growth and thereby induced land-use and land-cover change (LULCC) is of emergent importance. Remote sensing can reveal spatio-temporal growth trajectories of cities, which again allow a thorough understanding of the impacts of urbanization on ecosystems and ecosystem services. However, the mapping of urban areas remains one of the most challenging tasks of remote sensing data analysis. This paper presents an approach to map urban growth from multi-sensoral data, exemplified for the Dhaka megacity region in Bangladesh between 1990 and 2006. The approach is globally applicable and can facilitate regional urban growth maps in arbitrary complex and dynamic environments. Dhaka's densified urban landscape, its deltaic locality and the highly dynamic monsoon-related phenology call for a sophisticated analysis approach that is able to separate intra-annual land-cover variations from actual urbanization. Imagery from the Landsat series of satellites is a great asset for such an analysis due to its synoptic coverage of large urban areas as well as its unique historical archives. In our approach, we solve problems of spectral ambiguities and seasonal phenological dynamics through incorporating multi-temporal imagery for each monitoring year (1990, 2000, and 2006) and by extending the spectral feature space with synthetic aperture radar (SAR) data. The resulting datasets are heterogeneous and comprise measurements of unequal scaling. Non-parametric classification algorithms are required to delineate multi-modal and non-Gaussian class distributions of heterogeneous as well as temporally and spectrally complex land-cover classes of interest in such an extended feature space. We therefore used a support vector machine (SVM) classifier and post classification comparison to reveal spatio-temporal patterns of urban land-use and land-cover changes. An SVM based forward feature selection procedure allowed deriving in-depth information about the individual contribution of different input bands. Our methodology delineated relevant land-cover classes and resulted in overall accuracies better than 83% for all years considered. Change analysis unveiled a profound expansion of urban areas at the expense of prime agricultural areas and wetlands. During the 1990s, change was primarily characterized by a densification of urban fabric, whereas more recent changes included vast in-filling of low lying land and an extensive industrial sprawl into Dhaka's peri-urban areas. Our multi-sensoral and multi-temporal mapping approach allowed for delineating temporally dynamic LULCC, which again allowed for an insightful characterization of land system changes in the megacity region of Dhaka.

고차원을 갖는 생체 스펙트럼 데이터의 특징추출 및 분류기법

본 논문에서는 비선형 변환에 의해 입력신호를 고차원의 확장공간으로 변환한 후, 주성분분석기법(PCA)에 의해 신호의 특징을 추출하는 기법을 제안한다. 특징추출을 위해 사용되는 기존의 주성분분석기법은 입력데이터가 비선형 특성을 갖는 경우 최적의 변환행렬을 구할 수 없다는 문제점을 가지고 있다. 이러한 문제점을 해결하기 위해, 확장공간상에서 구간별로 입력데이터를 분할한 후 주성분분석기법에 의해 구간별 특징을 추출하는 서브패턴기반 주성분분석기법(SpPCA)을 적용하였다. 다음 단계인 분류단계에서는 MLP 비선형분류기를 이용하여 구간마다 추출된 특징벡터를 이용하여 기준패턴과의 유사도를 산출한다. 최종 분류단계에서는 MLP에 의해서 산출된 유사도에 기반을 둔 융합법칙에 의하여 생체 스펙트럼 패턴을 분류한다. 제안된 방법의 유용성을 보이기 위한 실험결과에서 기존의 방법들에 비해서 향상된 인식결과를 보임을 확인하였다. In this paper, we propose the biomedical spectral pattern classification techniques by the fusion scheme based on the SpPCA and MLP in extended feature space. A conventional PCA technique for the dimension reduction has the problem that it can't find an optimal transformation matrix if the property of input data is nonlinear. To overcome this drawback, we extract features by the SpPCA technique in extended space which use the local patterns rather than whole patterns. In the classification step, individual classifier based on MLP calculates the similarity of each class for local features. Finally, biomedical spectral patterns is classified by the fusion scheme to effectively combine the individual information. As the simulation results to verify the effectiveness, the proposed method showed more improved classification results than conventional methods.

Neurolinguistic approach to natural language processing with applications to medical text analysis

Understanding written or spoken language presumably involves spreading neural activation in the brain. This process may be approximated by spreading activation in semantic networks, providing enhanced representations that involve concepts not found directly in the text. The approximation of this process is of great practical and theoretical interest. Although activations of neural circuits involved in representation of words rapidly change in time snapshots of these activations spreading through associative networks may be captured in a vector model. Concepts of similar type activate larger clusters of neurons, priming areas in the left and right hemisphere. Analysis of recent brain imaging experiments shows the importance of the right hemisphere non-verbal clusterization. Medical ontologies enable development of a large-scale practical algorithm to re-create pathways of spreading neural activations. First concepts of specific semantic type are identified in the text, and then all related concepts of the same type are added to the text, providing expanded representations. To avoid rapid growth of the extended feature space after each step only the most useful features that increase document clusterization are retained. Short hospital discharge summaries are used to illustrate how this process works on a real, very noisy data. Expanded texts show significantly improved clustering and may be classified with much higher accuracy. Although better approximations to the spreading of neural activations may be devised a practical approach presented in this paper helps to discover pathways used by the brain to process specific concepts, and may be used in large-scale applications.

A color image segmentation approach for content-based image retrieval

This paper describes a new color image segmentation method based on low-level features including color, texture and spatial information. The mean-shift algorithm with color and spatial information in color image segmentation is in general successful, however, in some cases, the color and spatial information are not sufficient for superior segmentation. The proposed method addresses this problem and employs texture descriptors as an additional feature. The method uses wavelet frames that provide translation invariant texture analysis. The method integrates additional texture feature to the color and spatial space of standard mean-shift segmentation algorithm. The new algorithm with high dimensional extended feature space provides better results than standard mean-shift segmentation algorithm as shown in experimental results.

Neural Networks in Non-Euclidean Spaces

Multilayer Perceptrons (MLPs) use scalar products to compute weighted activation of neurons providing decision borders using combinations of soft hyperplanes. The weighted fun-in activation function may be replaced by a distance function between the inputs and the weights, offering a natural generalization of the standard MLP model. Non-Euclidean distance functions may also be introduced by normalization of the input vectors into an extended feature space. Both approaches influence the shapes of decision borders dramatically. An illustrative example showing these changes is provided.