Complex Concept-Based Readability Estimation from Arabic Curriculum

Sentiment analysis in the Arabic language is challenging because of its linguistic complexity. Arabic is complex in words, paragraphs, and sentence structure. Moreover, most Arabic documents contain multiple dialects, writing alphabets, and styles (e.g., Franco-Arab). Nevertheless, fine-tuned bidirectional encoder representations from transformers (BERT) models can provide a reasonable prediction accuracy for Arabic sentiment classification tasks. This paper presents a fine-tuning approach for BERT models for classifying Arabic sentiments. It uses Arabic BERT pre-trained models and tokenizers and includes three stages. The first stage is text preprocessing and data cleaning. The second stage uses transfer-learning of the pre-trained models’ weights and trains all encoder layers. The third stage uses a fully connected layer and a drop-out layer for classification. We tested our fine-tuned models on five different datasets that contain reviews in Arabic with different dialects and compared the results to 11 state-of-the-art models. The experiment results show that our models provide better prediction accuracy than our competitors. We show that the choice of the pre-trained BERT model and the tokenizer type improves the accuracy of Arabic sentiment classification.

This study aims at exploring linguistic complexity of the English Text Books prescribed by the Federal Board of Intermediate and Secondary Education Islamabad, Pakistan, for intermediate classes. For the last two decades, the concept of linguistic complexity has appealed interest of the linguists and other researchers to investigate this phenomenon. There are several factors that influence linguistic complexity to a tex. The importance of a text in the process of teaching and learning is beyond any doubt, and this significance enhances in the context of the third world countries where teaching and learning is considered almost impossible in absence of textbooks. In this environment selection of a textbook without gauging its linguistic complexity and grammatical intricacy is vulnerable. Two textbooks, Text- A, for first year class, and Text- B, for second year class, were chosen as samples purposefully. To build the edifice of this research Halliday and Ure’s lexical density and grammatical intricacy methods were used. The findings reveal that the actual text of Text- A falls in the slab of the more linguistic complexity with an index of 5.1 and Text- B which falls in the category of simple texts with an average index of 4.6, as Halliday and Ure’s methods suggest. This paper concludes with recommendations that text book writers and the textbook designers must familiar with th phenomenon of linguistic complexity and grammatical density and before prescribing any text to the ultimate learners, the textbook must be examined and evaluated to measure linguistic complexity

It is well-known that a large amount of data is required to train deep learning systems. However, data collection is very costly if it is not impossible to do. To overcome the limited data problem, one can use models that have been trained with a large dataset and apply them in the target domain with a limited dataset. In this paper, we use pre-trained models on imageNet data and re-train them on our data to detect tea leaf diseases. Those pre-trained models use deep convolutional neural network (DCNN) architectures: VGGNet, ResNet, and Xception. To mitigate the difference tasks of ImageNet and ours, we apply fine-tuning on the pre-trained models by replacing some parts of the pre-trained models with new structures. We evaluate the performance using various re-training and fine-tuning schema. The vanilla pre-trained model is used as the baseline while other techniques such as re-training the models on the appended structures, partially re-training the pre-trained models, and fully re-training the whole networks where the pre-trained models are used in the initialization as the evaluator. Our experiments show that applying transfer learning only on our data may not be effective due to the difference in our task to ImageNet. Applying fine-tuning on pre-trained DCNN models is found to be effective. It is consistently better than that of using transfer learning only or partial fine-tuning. It is also better than training the model from scratch, i.e., without using pre-trained models.

Accurate genomic prediction of complex phenotypes is crucial for accelerating genetic progress in swine breeding. However, conventional methods like Genomic Best Linear Unbiased Prediction (GBLUP) face limitations in capturing complex non-additive effects that contribute significantly to phenotypic variation, restricting the potential accuracy of phenotype prediction. To address this challenge, we introduce a novel framework based on a self-supervised, pre-trained encoder-only Transformer model. Its core novelty lies in tokenizing SNP sequences into non-overlapping 6-mers (sequences of 6 SNPs), enabling the model to directly learn local haplotype patterns instead of treating SNPs as independent markers. The model first undergoes self-supervised pre-training on the unlabeled version of the same SNP dataset used for subsequent fine-tuning, learning intrinsic genomic representations through a masked 6-mer prediction task. Subsequently, the pre-trained model is fine-tuned on labeled data to predict phenotypic values for specific economic traits. Experimental validation demonstrates that our proposed model consistently outperforms baseline methods, including GBLUP and a Transformer of the same architecture trained from scratch (without pre-training), in prediction accuracy across key economic traits. This outperformance suggests the model's capacity to capture non-linear genetic signals missed by linear models. This research contributes not only a new, more accurate methodology for genomic phenotype prediction but also validates the potential of self-supervised learning to decipher complex genomic patterns for direct application in breeding programs. Ultimately, this approach offers a powerful new tool to enhance the rate of genetic gain in swine production by enabling more precise selection based on predicted phenotypes.

Natural language processing (NLP) has made significant progress with the introduction of Transformer-based architectures that have revolutionized tasks such as question-answering (QA). While English is a primary focus of NLP research due to its high resource datasets, low-resource languages such as Turkish present unique challenges such as linguistic complexity and limited data availability. This study evaluates the performance of Transformer-based pre-trained language models on QA tasks and provides insights into their strengths and limitations for future improvements. In the study, using the SQuAD-TR dataset, which is the machine-translated Turkish version of the SQuAD 2.0 dataset, variations of the mBERT, BERTurk, ConvBERTurk, DistilBERTurk, and ELECTRA Turkish pre-trained models were fine-tuned. The performance of these fine-tuned models was tested using the XQuAD-TR dataset. The models were evaluated using Exact Match (EM) Rate and F1 Score metrics. Among the tested models, the ConvBERTurk Base (cased) model performed the best, achieving an EM Rate of 57.81512% and an F1 Score of 71.58769%. In contrast, the DistilBERTurk Base (cased) and ELECTRA TR Small (cased) models performed poorly due to their smaller size and fewer parameters. The results indicate that case-sensitive models generally perform better than case-insensitive models. The ability of case-sensitive models to discriminate proper names and abbreviations more effectively improved their performance. Moreover, models specifically adapted for Turkish performed better on QA tasks compared to the multilingual mBERT model.

There is a trend of researchers and practitioners to directly apply pre-trained models to solve their specific tasks. For example, researchers in software engineering (SE) have successfully exploited the pre-trained language models to automatically generate the source code and comments. However, there are domain gaps in different benchmark datasets. These data-driven (or machine learning based) models trained on one benchmark dataset may not operate smoothly on other benchmarks. Thus, the reuse of pre-trained models introduces large costs and additional problems of checking whether arbitrary pre-trained models are suitable for the task-specific reuse or not. To our knowledge, software engineers can leverage code contracts to maximize the reuse of existing software components or software services. Similar to the software reuse in the SE field, reuse SE could be extended to the area of pre-trained model reuse. Therefore, according to the model card’s and FactSheet’s guidance for suppliers of pre-trained models on what information they should be published, we propose model contracts including the pre- and post-conditions of pre-trained models to enable better model reuse. Furthermore, many non-trivial yet challenging issues have not been fully investigated, although many pre-trained models are readily available on the model repositories. Based on our model contract, we conduct an exploratory study of 1908 pre-trained models on six mainstream model repositories (i.e., the TensorFlow Hub, PyTorch Hub, Model Zoo, Wolfram Neural Net Repository, Nvidia, and Hugging Face) to investigate the gap between necessary pre- and post-condition information and actual specifications. Our results clearly show that (1) the model repositories tend to provide confusing information of the pre-trained models, especially the information about the task’s type, model, training set, and (2) the model repositories cannot provide all of our proposed pre/post-condition information, especially the intended use, limitation, performance, and quantitative analysis. On the basis of our new findings, we suggest that (1) the developers of model repositories shall provide some necessary options (e.g., the training dataset, model algorithm, and performance measures) for each of pre/post-conditions of pre-trained models in each task type, (2) future researchers and practitioners provide more efficient metrics to recommend suitable pre-trained model, and (3) the suppliers of pre-trained models should report their pre-trained models in strict accordance with our proposed pre/post-condition and report their models according to the characteristics of each condition that has been reported in the model repositories.

The integration of various algorithms in the medical field to diagnose brain disorders is significant. Generally, Computed Tomography, Magnetic Resonance Imaging techniques have been used to diagnose brain images. Subsequently, segmentation and classification of brain disease remain an exigent task in medical image processing. This paper presents an extended model for brain image classification based on a Modified pre-trained convolutional neural network model with extensive data augmentation. The proposed system has been efficiently trained using the technique of substantial data augmentation in the pre-processing stage. In the first phase, the pre-trained models namely AlexNet, VGGNet-19, and ResNet-50 are employed to classify the brain disease. In the second phase, the idea of integrating the existing pre-trained model with a multiclass linear support vector machine is incorporated. Hence, the SoftMax layer of pre-trained models is replaced with a multi class linear support vector machine classifier is proposed. These proposed modified pre-trained model is employed to classify brain images as normal, inflammatory, degenerative, neoplastic and cerebrovascular diseases. The training loss, mean square error, and classification accuracy have been improved through the concept of Cyclic Learning rate. The appropriateness of transfer learning has been demonstrated by applying three convolutional neural network models, namely, AlexNet, VGGNet-19, and ResNet-50. It has been observed that the modified pre-trained models achieved a higher classification rate of accuracies of 93.45% when compared with a finetuned pre-trained model of 89.65%. The best classification accuracy of 92.11%, 92.83% and 93.45% has been attained in the proposed method of the modified pre-trained model. A comparison of the proposed model with other pre-trained models is also presented.

This paper systematically reviews the aspect-based sentiment analysis techniques that integrate data augmentation and pre-trained language models. Aspect-based sentiment analysis aims to identify the sentiment tendency of specific aspects in texts. Traditional methods face challenges such as data sparsity and insufficient model generalization. Data augmentation and pre-trained language models bring opportunities to solve these problems. Data augmentation can alleviate data sparsity, and pre-trained language models have powerful feature extraction and transfer learning capabilities. This paper elaborates on the task definition of aspect-based sentiment analysis, focusing on specific methods based on data augmentation and pre-trained language models, including data augmentation strategies and methods, as well as methods based on pre-trained language models such as BERT, RoBERTa, BART, and XLNet, and explores how to combine data augmentation and pre-trained models to improve the performance of aspect-level sentiment analysis. Finally, it is pointed out that there are still some challenges and opportunities in this field, such as the diversity of data augmentation techniques, optimization of pre-trained models, multimodal sentiment analysis, interpretability, and credibility, which need to be further explored.

Geostationary satellites are valuable tools for monitoring the entire lifetime of tropical cyclones (TCs). Although the most widely used method for TC intensity estimation is manual, several automatic methods, particularly artificial intelligence (AI)-based algorithms, have been proposed and have achieved significant performance. However, AI-based techniques often require large amounts of input data, making it challenging to adopt newly introduced data such as those from recently launched satellites. This study proposed a transfer-learning-based TC intensity estimation method to combine different source data. The pre-trained model was built using the Swin Transformer (Swin-T) model, utilizing data from the Communication Ocean and Meteorological Satellite Meteorological Imager sensor, which has been in operation for an extensive period (2011–2021) and provides a large dataset. Subsequently, a transfer learning model was developed by fine-tuning the pre-trained model using the GEO-KOMPSAT-2A Advanced Meteorological Imager, which has been operational since 2019. The transfer learning approach was tested in three different ways depending on the fine-tuning ratio, with the optimal performance achieved when all layers were fine-tuned. The pre-trained model employed TC observations from 2011 to 2017 for training and 2018 for testing, whereas the transfer learning model utilized data from 2019 and 2020 for training and 2021 for testing to evaluate the model performance. The best pre-trained and transfer learning models achieved mean absolute error of 6.46 kts and 6.48 kts, respectively. Our proposed model showed a 7–52% improvement compared to the control models without transfer learning. This implies that the transfer learning approach for TC intensity estimation using different satellite observations is significant. Moreover, by employing a deep learning model visualization approach known as Eigen-class activation map, the spatial characteristics of the developed model were validated according to the intensity levels. This analysis revealed features corresponding to the Dvorak technique, demonstrating the interpretability of the Swin-T-based TC intensity estimation algorithm. This study successfully demonstrated the effectiveness of transfer learning in developing a deep learning-based TC intensity estimation model for newly acquired data.

目的 基于计算机的胸腔X线影像疾病检测和分类目前存在误诊率高，准确率低的问题。本文在视觉Transformer（vision Transformer，ViT）预训练模型的基础上，通过迁移学习方法，实现胸腔X线影像辅助诊断，提高诊断准确率和效率。方法 选用带有卷积神经网络（convolutional neural network，CNN）的ViT模型，其在超大规模自然图像数据集中进行了预训练；通过微调模型结构，使用预训练的ViT模型参数初始化主干网络，并迁移至胸腔X线影像数据集中再次训练，实现疾病多标签分类。结果 在IU X-Ray数据集中对ViT迁移学习前、后模型平均AUC（area under ROC curve）得分进行对比分析实验。结果表明，预训练ViT模型平均AUC得分为0.774，与不使用迁移学习相比提升了0.208。并针对模型结构和数据预处理进行了消融实验，对ViT中的注意力机制进行可视化，进一步验证了模型有效性。最后使用Chest X-Ray14和CheXpert数据集训练微调后的ViT模型，平均AUC得分为0.839和0.806，与对比方法相比分别有0.014～0.031的提升。结论 与其他方法相比，ViT模型胸腔X线影像的多标签分类精确度更高，且迁移学习可以在降低训练成本的同时提升ViT模型的分类性能和泛化性。消融实验与模型可视化表明，包含CNN结构的ViT模型能重点关注有意义的区域，高效获取胸腔X线影像的视觉特征。;Objective The chest X-ray-relevant screening and diagnostic method is essential for radiology nowadays. Most of chest X-ray images interpretation is still restricted by clinical experience and challenged for misdiagnose and missed diagnoses. To detect and identify one or more potential diseases in images automatically，it is beneficial for improving diagnostic efficiency and accuracy using computer-based technique. Compared to natural images，multiple lesions are challenged to be detected and distinguished accurately in a single image because abnormal areas have a small proportion and complex representations in chest X-ray images. Current convolutional neural network（CNN）based deep learning models have been widely used in the context of medical imaging. The structure of the CNN convolution kernel has sensitive to local detail information，and it is possible to extract richer image features. However，the convolution kernel cannot be used to get global information，and the features-extracted are restricted of redundant information like its relevance of background， muscles，and bones. The model’s performance in multi-label classification tasks are affected to a certain extent. At present，the vision Transformer（ViT）model has achieved its priorities in computer vision-related tasks. The ViT can be used to capture information simultaneously and effectively for multiple regions of the entire image. However，it is required to use large-scale dataset training to achieve good performance. Due to some factors like patient privacy and manual annotate costs，the size of the chest X-ray image data set has been limited. To reduce the model's dependence on data scale and improve the performance of multi-label classification，we develop the CNN-based ViT pre-training model in terms of the transfer learning method for diagnosis-assisted of chest X-ray image and multi-label classification. Method The CNN-based ViT model is pre-trained on a huge scale ground truth dataset，and it is used to obtain the initial parameters of the model. The model structure is fine-tuned according to the features of chest X-ray dataset. A 1×1 convolution layer is used to convert the chest X-ray images channels between 1 to 3. The number of output nodes of the linear layer in the classifier is balanced from 1 000 to the number of chest X-ray classification labels，and the Sigmoid is used as an activation function. The parameters of the backbone network are initialized in terms of the pre-trained ViT model parameters，and it is trained in the chest X-ray dataset after that to complete multi-label classification. The experiment is configured of Python3. 7 and PyTorch1. 8 to construct the model and RTX3090 GPU for training. Stochastic gradient descent（SGD）optimizer，binary cross-entropy（BCE）loss function，an initial learning rate of 1E-3，the cosine annealing learning rate decay are used. For training，each image is scaled to a size of 512×512 pixels，and a 224×224 pixels area and it is then cropped in random as the model input，and data augmentation is performed randomly by some of the flipping，perspective transformation， shearing，translation，zooming，and changing brightness. For testing，the chest X-ray image is scaled to 256×256 pixels and center crop a 224×224 area to input the trained model. Result The experiment is performed on the IU X-Ray，which is a small-scale chest X-ray dataset. This model is evaluated in quantitative using the average of area under ROC curve （AUC）scores across all classification labels. The results show that the average AUC score of the pre-trained ViT model is 0. 774. The accuracy and training efficiency of the non-pre-trained ViT model is dropped significantly. The average AUC score is reached to 0. 566 only，which is 0. 208 lower. In addition，the attention mechanism heat map is generated based on the ViT model，which can strengthen the interpretability of the model. A series of ablation experiments are carried out for data augmentation，model structure，and batch size design. The fine-tuned ViT model is trained on the Chest-Ray14 and CheXpert dataset as well. The average AUC score is reached to 0. 839 and 0. 806，which is optimized by 0. 014 and 0. 031. Conclusion A pre-trained ViT model is used for the multi-label classification of chest X-ray images via transfer learning. The experimental results illustrate that the ViT has its stronger multi-label classification performance in chest Xray images，and its attention mechanism is beneficial for lesions precision-focused like the interior of the chest cavity and the heart. Transfer learning is potential to improve the classification performance and model generalization of the ViT in small-scale datasets，and the training cost is reduced greatly. Ablation experiments demonstrate that the incorporated model of CNN and Transformer has its priority beyond single-structure model. Data enhancement and the batch size cutting can improve the performance of the model，but smaller scale of batch is still interlinked to longer training span. To improve the model's ability，we predict that future research direction can be focused on the extraction for complex disease and highlevel semantic information，such as their small lesions，disease location，and severity.

In linguistics the concept of complexity has been analysed from various perspectives, among them language typology and the speech/writing distinction. Within intralinguistic studies, certain key linguistic features associated with reduced or increased complexity have been identified. These features occur in different patterns across various registers and their frequency is an indicator of the level of complexity of different kinds of texts. The concept of complexity has not, to date, been evaluated in early English medical writing, especiallyin terms of different text types. Thus, the present article analyses linguistic complexity in two Early Modern English medical texts, a surgical treatise (ff. 34r-73v) and a collection of medical recipes (ff. 74r-121v) housed as MS Hunter 135 in Glasgow University Library. Since they represent two different types of medical text, they can be productively compared in terms of linguistic complexity. The results obtained confirm that the surgical treatise is more complex than the collection of medical recipes owing to the higher presence of linguistic features denoting increased complexity in the former and of those indicating reduced linguistic complexity in the latter.

Substantivfrasens kompleksitetsutvikling i innlærerspråk: En konstruksjonsgrammatisk analyse av skriftlig produksjon fra A1- til B2-nivå

Custom object detection plays a vital role in computer vision applications. However, developing an accurate and efficient custom object detector requires a substantial amount of labeled training data and significant computational resources. In this research, we propose a custom object detection framework that leverages transfer learning with pre-trained models to improve detection tech-niques.The framework first utilizes a pre-trained deep learning model, such as ResNet or VGGNet, as a feature extractor. The pre-trained model is trained on a large-scale dataset, enabling it to learn high-level features from various objects. By reusing the pre-trained model's convolutional layers, we effectively capture generic features that can be transferred to the custom object detection task.Experimental evaluations on benchmark datasets demonstrate the effectiveness of our ap-proach. The custom object detector achieved superior detection performance compared to tradi-tional methods, especially when the target objects have limited training data. Additionally, our framework significantly reduces the amount of training time and computational resources required, as it leverages pre-trained models as a starting point.

In military operations, the proper use of force in accordance with the laws of war are non-negotiable. As the battlespace is becoming more and more volatile, uncertain, complex, and ambiguous, military leaders are becoming more dependent upon advanced technologies such as artificial intelligence to assist in their decision making. One such area that military leaders need advanced technology support is positive control of all friendly assets as well as identification of enemy and noncombatant assets in all domains of air, land, sea, and space through the identification of friend or foe (IFF). The purpose of this study is to assess the viability of deep transfer learning to assist in military decision-making and answer the research question: Can pre-trained deep learning models be used to adequately identify and classify enemy targets? This study is designed as a comparative study to assess and compare various pre-trained deep-learning models to determine if they are adequate for targeting and engaging the enemy. An ensemble model was also incorporated using three pretrained models and compared to the results of the individual models. A discussion of Human-in-the-loop concepts as well as the ethical considerations of the use of AI for IFF is incorporated in this study.

Здійснено порівняльний аналіз різних методів оброблення природної мови для виявлення симптомів ментального захворювання. Розглянуто принцип роботи та ефективність моделей оцінювання семантичної когерентності тексту (моделі тан-генційності та некогерентності) для класифікації текстів здорових і хворих осіб. У роботі зазначається залежність точності моделей некогерентності та тангенційності від моделі семантичного представлення фрагментів тексту; підкреслюється недолік використання такої моделі в зв'язку з відсутністю можливості враховувати регулярне повторення фраз. Проаналізовано переваги та недоліки застосування комбінації моделей семантичного представлення елементів тексту для врахування постійних повторів його фрагментів. Обґрунтовано доцільність застосування лінгвістичних характеристик тексту пацієнта для підвищення точності класифікаторів виявлення симптомів захворювань та розрізнення їх типу. Розглянуто можливість аналізу частоти появи неоднозначних займенників у тексті для підвищення точності класифікації даних. Проаналізовано особливості застосування різних методів виявлення симптомів ментального захворювання для текстів англійською, німецькою та російською мовами. Запропоновано здійснювати оцінювання зв'язності тексту за допомогою графу узгодженості словосполучень. Здійснено експериментальну перевірку ефективності пропонованого підходу для побудови класифікаційної моделі порівняно з іншими характеристиками тексту.