Web Scrapping Research Articles

Kiswahili Translation in Digital Media Platforms: Bridging Language Gaps in the AI Age

Purpose: As digital media platforms continue to proliferate globally, the need for multilingual content in these platforms has become increasingly apparent. Translation of online content is a necessity for netizens and other digital platform users who seek knowledge and information around a myriad of user-generated content on diverse issues and topics. Being a widely spoken language that goes beyond the borders of East Africa, Kiswahili’s significance in bridging linguistic barriers and facilitating communication and information access for Kiswahili speakers is clear. The purpose of this exploratory research paper was to examine the state of Kiswahili as a language of translation through different technologies across varied translation apps and social media platforms such as Facebook, You Tube, Tik Tok and Netflix. Methodology: The study adopted a descriptive research design. Data collection was done through online data mining tools including API access for web scrapping and manual extraction of content through random search engine searches. The data for the study included 10 sentences, 10 videos and 25 specific word forms such as proverbs, spiritual/religious words, similes and cultural words for food, beauty and family ties. Content analysis for both text and multimedia content such as written sentences, social media posts and videos was done. The study analyzed translation outputs of 14 Kiswahili translation apps in order to establish the status and accuracy of these translations. Of key consideration was the role of AI technology in the translation apps and the effect it has on the translated texts across different apps. The challenges and emerging issues facing Kiswahili translation through digital technologies were also explored. Findings: The findings of this study showed that the Apps used for Kiswahili translation have demonstrated great translation accuracy in basic semantic forms, with an average score of 60% accuracy upon random searches and between 10-30% accuracy for cultural forms and idiomatic expressions. Unique Contribution to Theory, Practice and Policy: The study was guided by the Skopos theory by Hans Vermeer (1978) which was complemented by the Relevance Theory by Dan Sperber and Deidre Wilson (1986). These Apps seem to have the feature of interoperability in that they seem to produce the same results suggesting that the AI supported feature in the apps may be interconnecting the apps during translation. The study advocates for more AI integration translation apps as this will help standardize Kiswahili translation corpus.

Contextual Conversational Intelligence : Leveraging Language Models For College Chatbot By Retrieval Augmented Generation

By harnessing the capabilities of Large Language Models (LLM), the proposed system evolves into a sophisticated conversational agent, capable of understanding nuanced user inputs, providing context-aware responses, and delivering an unparalleled level of engagement. This chatbot is also equipped to assess a student's likelihood of being accepted into a specific program at the college using a machine learning (ML) model. The model analyzes the student's rank and eligible quota (such as reserved categories) in conjunction with historical admission data to gauge their chances of acceptance. It also recommends alternative programs where the student may have a better chance of admission based on their rank. The chatbot also features a smart Applicant Tracking System (ATS) that assists students in evaluating their resumes. This system provides keyword matching suggestions based on job descriptions and generates an overview of the resume. Additionally, it highlights areas for improvement and highlights the resume's distinctiveness. The methodology involves the development and deployment of a website chat bot integrated with Lang Chain, with the potential of advanced language models in transforming website chat bot interactions using Retrieval augmented generation (RAG) technique to ground LLM to generate responses to user queries based on a custom knowledge-base. Key Words: Chatbot, Generative AI, Large Langauge Model, Website, Machine Learning, Retrieval-augmented generation , Web scrapping , Resume , ATS.

The Application of Web Scrapping Method to Evaluate the Customers’ Inclination to Share the Online Word-of-Mouth in Hotel Sector: The European Perspective

The Application of Web Scrapping Method to Evaluate the Customers’ Inclination to Share the Online Word-of-Mouth in Hotel Sector: The European Perspective

Analysis of Vina Film Sentiment on Social Media X Using The Naïve Bayes Method

The increasingly rapid development of technology and information, one of which is the internet. Where users can share opinions and discuss various topics or problems around them, namely social media One of the news items that frequently appears as a trending topic on X is the Vina film controversy. However, with the large amount of review data available, it will be difficult to process manually. Therefore, sentiment analysis is needed to see whether people's tendencies toward the Vina film case are positive or negative. The stages carried out were data collection taken via web scrapping with an initial amount of data of 833 and processed through the preprocessing stage, including cleaning, case folding, normalization, stopword removal, tokenization, and stemming, the data became 830. The application of the Naïve Bayes algorithm in this research uses the probability method to classify and predict 664 training data and 166 test data, with the help of the Python library. The accuracy calculation results show quite good performance with TF-IDF weighting producing an accuracy of 78%, precision of 80%, and recall of 90%, f1-score of 84%. Analysis from this research shows that the dominance of negative sentiment is 517 while positive sentiment is 313. The amount and quality of training data play an important role in system quality, where high data quality provides better accuracy in predicting sentiment classes.

Approving Academic Advancement: Crafting Customized Recommendations for Grant Seekers within the Centralized Research Grant Management System

Introducing a groundbreaking Centralized Framework for Research Grant Management, poised to revolutionize grant lifecycles. This innovative system offers a unified interface, enhancing efficiency, transparency, and accountability. It includes features like rigorous peer review, real-time tracking, and personalized recommendations using machine learning algorithms. Despite limited existing research papers, this platform significantly aids grant seekers in identifying opportunities aligned with their specific department or research interests, filling a crucial gap in the grant-seeking process. Robust online security ensures data integrity, while adaptability supports diverse funding models and interdisciplinary collaboration. KEYWORD’S Centralized Grant Platform, Research Funding, Grant Allocation, AI Recommendations, Machine learning, Web Scrapping

Implementasi Tunelling pada Perancangan Sistem Peringkasan dan Klasifikasi Berita Otomatis menggunakan Textrank dan KNN

News summarization is very important in the news analysis process. However, in the summarization process, there are often obstacles such as the large number of news stories and the need for news classification. This research aims to build a simple web-based system that can be used to summarize and classify news which will be very useful in the news analysis process. The proposed summarization method is Textrank, and the news classification method that will be used is KNN. This system is expected to provide an automatic summarization function to make it easier to analyze news content. The data that will be used as the basis for classification modeling is sports news in 3 months, and the classification that will be used to determine whether the news includes sports news in three branches, namely football, rackets or basketball. Testing of the summarization model using textrank was carried out by applying ROUGE-1 and ROUGE-2, with results of 0.79 and 0.67. Meanwhile, testing the classification model using KNN with k=3 and k=5 is 0.9866 and 0.9666 so k=3 will be used. This system will be built using the web scrapping library, textrank, stopword from PySastrawi, scikit-learn for the classification module using the KNN algorithm, and ngrok for publishing web-based applications. By using ngrok, we can expose the application through internet with a temporary public url without hosting required

Sentiment Analysis of Brand Ambassador Influence on Product Buyer Interest Using KNN and SVM

In the dynamic marketing, companies usually use strategies involving celebrities or influencers to promote their products or brands. The currently popular strategy is using Korean boy bands as brand ambassadors. This collaboration certainly gets a lot of opinion responses through tweets on X app social media. This research aims to analyze sentiment to determine how the product buyer's interest responds to brand suitability, brand image management, and the influence of issues that arise in this collaboration. The research stages consist of data collection, pre-processing, labeling, weighting, and classification with K-Nearest Neighbor and Support Vector Machine and performance evaluation using a confusion matrix. The dataset used was 696 tweets taken using web scrapping techniques. This research uses the Lexicon-based method to divide the dataset into positive, negative, and neutral classes. The SVM method shows superior test results by achieving an accuracy rate of 83.34% compared to the KNN method, which produces an accuracy value of 71.2% in its calculations

E-Commerce Spam Review Detection using Machine Learning

With the proliferation of e-commerce platforms, the authenticity of online reviews has become increasingly crucial. In this research, we present a method for detecting spam reviews in e-commerce platforms, employing the Random Forest algorithm. Leveraging a dataset comprising reviews from Amazon Yelp dataset, we employ a combination of Natural Language Processing (NLP) techniques and text processing methods to uncover underlying patterns distinguishing between genuine and fake reviews. Real-time data scraping from the Amazon website facilitated the acquisition of a diverse range of reviews, subsequently stored in a CSV file for analysis. These reviews stored in CSV file is fed to model for prediction. Our model effectively discerns between authentic and spam reviews, offering a valuable tool for maintaining the integrity of e-commerce platforms and ensuring informed consumer decision-making. Key Words: Web Scrapping, Text Mining, Sentiment Analysis.

Recommended Strategies Based on The Capability And Purchasing Power of Low-Income Communities (MBR) In Providing MBR Houses In Kartamantul

The housing needs that shape urban spatial structures or the relationships between the three cannot be fully understood without considering the state of housing market prices. The reason is that changes in urban housing market prices no longer depend on the city level, but over time begin to be characterized by urban agglomeration processes. Furthermore, agglomeration dynamics that change urban spatial structure and housing market prices will have an impact on socio-economic disparities and challenges in providing affordable housing for people with low incomes in densely populated urban areas. The aim of this research is to find out whether the MBR housing allocated by the regional government is appropriately located in low-income areas/zones in KARTAMANTUL. The research methods used are web scrapping, mapping and comparative evaluation of housing locations with the distribution of MBR in KARTAMANTUL. The results obtained were to simplify the simulation of affordability and household purchasing power regarding home ownership, the researchers tried to average the income of the two categories of low-income households. Therefore, with an average MBR income limit of Rp. 3,900,000 in DIY, then the ability to repay the house in installments is IDR. 1,170,000/month. Furthermore, regarding house buildings, the criteria for a habitable house is a per capita area of at least 9 m2 per person. With the average number of family members in DIY being 4 people, the house area required per household is 36 m2. The recommendations given are a) 25 year house installments are a payment option that can be used for low-income households in KARTAMANTUL, b) the choice of house locations for MBR is mostly in Bantul Regency and Sleman Regency, both of which have urban status. fringe and urban shadow.

Digital Innovation Hubs and portfolio of their services across European economies

Research background: Digital ecosystems in Europe are heterogenous organizations involving different economies, industries, and contexts. Among them, Digital Innovation Hubs (DIHs) are considered a policy-driven organization fostered by the European Commission to push companies’ digital transition through a wide portfolio of supporting services. Purpose of the article: There are DIHs existing in all European economies, but literature needs more precise indications about their status and nature. The purpose is to study a distribution of DIHs and differences in portfolios of DIHs’ services across European economies. Therefore, the paper wants to deliver more precise data on effects on national and European policies. This is required to define their final role and scope in the complex dynamics of the digital transition, depending on regional context and heterogeneity of industries. Methods: Data on 38 economies was collected from the S3 platform (on both existing and in preparation DIHs) and further verified by native speaking researchers using manual web scrapping of websites of DIHs identified from S3. To find potential similarities of digital ecosystems in different economies as emanated by the existence of DIHs, clusterization (Ward’s method and Euclidean distances) was applied according to the services offered. Economies were clustered according to the number of DIHs and the spread of DIHs intensity in different cities. The results were further analyzed according to the scope of the provided services. Findings & value added: The applied clustering classified European economies in four different sets, according to the types of services offered by the DIHs. These sets are expression of the different digitalization statuses and strategies of the selected economies and, as such, the services a company can benefit from in a specific economy. Potential development-related reasons behind the data-driven clustering are then conjectured and reported, to guide companies and policy makers in their digitalization strategies.

Developing a Framework of Equity Crowdfunding for Small and Medium Enterprises (SMEs) in Malaysia: A Conceptual Paper

The Covid-19 pandemic has affected the global economic turmoil leading to the global recession around the world. This will adversely affect Malaysia’s small and medium enterprises (SMEs) sustainability. Companies with less than three years of operations face various challenges tapping into funding facilities as banks will prioritise existing customers with good track records. An alternative avenue for funding and financing is necessary to survive in the business. To gain holistic understanding of the ECF, this study has four objectives: (1) To uncover the roles of ECF to SMEs in the time of crisis; (2) To investigate the crucial influencing factors to success the crowdfunding campaigns; (3) To explore the roles of regulation to protect investor and issuers in equity crowdfunding; (4) To develop a framework for fundraisers to ensure their successful campaign. This study will employ a mixed-method approach to achieve the objectives. The interview will be conducted with registered ECF operators, and the questionnaire will be distributed to ECF fundraisers. Since different data sources will be used, the data preparation is divided into web scrapping and text extraction. Python programming language and scrapper library, known as Beautiful Soup, will used to obtain the company’s details, while text extraction will be performed on the pdf files. Therefore, Python library known as PyPDF2 will be used to extract data from PDF files. It is hoped that the findings and proposed framework can assist the government and policymakers in enhancing successful applicant campaigns and improving the investors’ protection legal framework. Along with the national agenda, namely Sustainability Development Goal (SDG) 8 and the Strategic Thrust 1 of the Shared Prosperity Vision 2030, successful SMEs are vital as they can create jobs opportunity for the local community and foster economic growth through more and large pool of investments.

Bridging the Skills Gap: Examining the Electronics Technology Students’ Competence and Industry Demands

Educational institutions often struggle to keep up with the emerging trends in the electronics industry, leading to a gap between what students learn in their programs and what is in demand in the job market. Beyond technical skills, electronics technology students also need to develop soft skills vital for success in any job but can sometimes be overlooked in traditional academic settings. Previous studies reported a skills gap between industry demands and students’ preparedness. Thus, this study determined the skills demanded by the electronics industry in Leyte, emphasizing both technical and soft skills; it also explored the acquired skills of the electronics technology students based on their experiences. This study employed mixed-method research, specifically the convergent parallel design, wherein quantitative and qualitative data were collected simultaneously and analyzed separately. For the quantitative data collection, 21 online job postings were utilized in the web scrapping technique and were limited to Leyte province only. For the qualitative data collection, an interview was conducted with 14 purposively selected 4th-year Electronics Technology students. Descriptive statistics were used to summarize quantitative data, while qualitative data underwent thematic analysis. Results revealed a skills gap among electronics students, influenced by the dearth of electronics-related industries and job openings in the province. Most employers require technical skills in troubleshooting and maintaining electronic systems. Being keen on details, being customer-oriented, and working well as a team are the necessary soft skills. The qualitative findings from the interview complement the quantitative results. The implication is clear for educational institutions that soft skills development should be emphasized in educational programs, preparing students technically and socially for the workplace.

Review to unfold the role of Machine Learning Algorithms in Natural Language Processing

This research review has been written to explore the role of machine learning algorithms in natural language processing (NLP), a computer field that offers human-like comprehension of virtual text. The main motive of the article is to reveal the participation of machine learning (ML) in NLP and its scope in coherent fields. In this study, the researchers have used a systematic literature review approach to explore the role of ML algorithms in NLP. Highlighting the techniques of ML algorithms as supervised, unsupervised, semi-supervised, and reinforcement methods, this study discloses the connectivity with morphological, semantic, syntactic, pragmatic, and discourse analysis in NLP. The article describes that ML/NLP has immense applications in different fields where several tools of ML/NLP are utilized. The researchers have studied the use of chatbots, text summarization tools, web scrapping, sentimental analysis in the social media stock market, medical field disease detection, and fraud detection. ML’s role and contributions to the progressions are detained in current studies. A distinct comparison has been made between the past present and future of ML in NLP. ML algorithms and applications with examples are disclosed in the classification of logistic regression, SVM, Naive Bayes, K-Nearest Neighbor, and decision tree. Data distortion, interpretation of the research findings, and contextual ambiguity are visibly determined as challenges and discussed in the study. This writing opens up to the recent advancements and advantages in multiple areas correspondingly. The discussions and findings expose the role of ML in opening the gateway to the profound revolutionary search engines, algorithms, and multiple techniques through the development of technology and its evolution to adapt to new language differences, slang, and changes in language use.

Web Mining and Anemia Classification in Gujarat via WWW

Web mining makes extracting internet data from big databases feasible. Low hemoglobin or insufficient red blood cells are symptoms of anemia. Undernutrition is the most common cause, and it is most common in young children, pregnant women, and women of childbearing age. If it is not treated or doesn't take into consideration, it could lead to an enlarged heart or heart failure. In this study, we gather data from the web about anemia in Gujarat and categorize it in various ways based on its parameter. The dataset builds from web scrapping using Python soup. The dataset we extract can be further classified to make a decision. This research helps in identifying anemia-related information as well as saving lives. Data mining tools predict future trends, help organizations, and present knowledge in the form of the easiest way so that humans can understand. The data can be processed, analyzed, and presented by a specifically trained algorithm for diverse stakeholders, including policymakers, program planners, health service providers, academicians, research scholars, and common citizens, enabling them for evidence-based, data-driven decision-making

A Novel Hybrid Feature Extraction Technique and Spam Review Detection using Ensemble Machine Learning Algorithm by Web Scrapping

A Novel Hybrid Feature Extraction Technique and Spam Review Detection using Ensemble Machine Learning Algorithm by Web Scrapping

Growth of digital brand name through customer satisfaction with big data analytics in the hospitality sector after the COVID-19 crisis

This study examines the best practices for the optimization of the corporate digital brand name by taking into consideration customers’ behavioral big data and web analytics. In the first stage of the study, customers’ satisfaction big data have been extracted with the assistance of a web scrapping tool from TripAdvisor for 189 hotels in Hubei province, and behavioral data from Hubei province hospitality websites have been gathered with the assistance of web analytics platforms for 5.7 million website visitors for the last 18 months. In the second stage of the research, those data have been statistically analyzed including descriptive, correlation, and regression analysis. Then, a fuzzy cognitive map has been created to present the intercorrelation between the parameters and two optimization scenarios have been developed for digital brand name and customer satisfaction. Finally, an Agent-based model has been created in order to simulate the customers’ behavior in the corporate website and TripAdvisor. The results indicated that hotels in Hubei province need to invest less in social media advertisements than search engine advertisements in order to achieve a competitive advantage and improve their digital brand name. Additionally, hotels need to develop their websites with more engaging content to maintain the customer on the corporate website for more time in order to optimize customer satisfaction in contrast to healthcare and libraries websites.

Dynamicity of Web Services and Web Scrapping

This research focuses on dynamic Web service link searching using web scraping (WS). WS is method of extracting required contents from search engine results. The proposed system inputs the functional word to search engine and retrieves required Web services from search engine results. WS method retrieves the links of the Web Service Definition Language from which service and operation elements are extracted and processed to check suitability of Web services. Matched Web services will be clustered and invoked. This research is implemented using J2EE and remote Web services are executed according to user requirement. Analysis of results justified that this will be a suitable search method to search for remote Web services and to execute using open source software.

A web scrapping and AI approach for archeologists to analyze the ancient cities

Studies on machine learning have started to reach a level where we can save a great amount of time and labor by producing structures that can think as a human and have decisions. Deep learning, one of the methods of machine learning, is an artificial intelligence-training technique that can predict the outputs from the given dataset In this study, the use of web scraping technique was investigated to determine the potential of identifying ancient columns, which are one of the most important architectural elements of cultural heritage, by artificial intelligence. In this study, web scraping approach is presented as a digital data acquisition method for archaeology field to collect imagery datasets from web to analyze the ancient cities. For analysis, a free online, and easy-to-use tool ‘Amazon Rekognation’ is used for comparing the number of columns found in the scrapped images. For summarizing the research, simply, we have tried to get the answer the question from PC that ‘which site has the columns most, Perge, Xanthos or Phaselis?’. With this proposed approach, the archeologists can have a primarily knowledge about the sites they will study with use of operational tools for their further comprehensive research.

E-commerce Price Comparison Website using Web Scraping

Price comparison sites are designed to compare the price of goods and services from a range of providers, which will help consumers in making decision to choose products that will save their money through online. Considering the customers’ busy lifestyle especially those who are living in the city area, most of the consumers prefer to buy their needs through the internet because it saves their time. Besides, consumers always go for the cheaper price in purchasing products therefore by using price comparison website, customers do not have to travel from shop to shop only to survey the price offered by different shops for the same product. They can just check it from the price comparison website itself and decide where they should buy the products they need. This project, named as Price comparison website using web scrapping is the place where shoppers could find the great deals on the products. The best deals will be clearly highlighted. To obtain best deals from Price comparison websites web scrapping techniques are used to fetch detailed information. This way, paper aims to provide solution for online customers to buy products at good deal and save their valuable time, effort, and money.

Identifying Fake News via Machine Learning and Web Scrapping

Abstract: After digitalization, the increase in use of social media has led the flow of information among the social network users unchecked. The news is immediately transported to social networks, where it is quickly read, marked with opinions (on Facebook), and shared (on Twitter and Facebook) without being verified as accurate or false numerous times. In the modern world, fake news is an issue that has grown difficult to detect. Its impact on the judgement of commonfolk is noticeable, society has witnessed numerous events where the rampant flow of unverified news had affected the society as a whole. The sharing of fake news has become a major issue to the society. Our project aims to aid in reduction of e-crimes like use of social media for sharing of fake content. This will be achieved by training the machine to identify text based fake content. Using this the time for work of filtering out the fake content will be reduced by great lengths whilst helping the mass to get verified and credible news.