Product matching in digital marketplaces: Multimodal model based on the transformer architecture

Recently, multimodal data analysis in medical domain has started receiving a great attention. Researchers from both computer science, and medicine are trying to develop models to handle multimodal medical data. However, most of the published work have targeted the homogeneous multimodal data. The collection and preparation of heterogeneous multimodal data is a complex and time-consuming task. Further, development of models to handle such heterogeneous multimodal data is another challenge. This study presents a cross modal transformer-based fusion approach for multimodal clinical data analysis using medical images and clinical data. The proposed approach leverages the image embedding layer to convert image into visual tokens, and another clinical embedding layer to convert clinical data into text tokens. Further, a cross-modal transformer module is employed to learn a holistic representation of imaging and clinical modalities. The proposed approach was tested for a multi-modal lung disease tuberculosis data set. Further, the results are compared with recent approaches proposed in the field of multimodal medical data analysis. The comparison shows that the proposed approach outperformed the other approaches considered in the study. Another advantage of this approach is that it is faster to analyze heterogeneous multimodal medical data in comparison to existing methods used in the study, which is very important if we do not have powerful machines for computation.

Digital marketplaces are sites that provide online transaction facilities for their customers. As such, business transactions can be done anytime and anywhere between sellers and buyers through digital platforms, making them very flexible compared to physical stores or traditional marketplaces. Digital marketplaces are not new as they were among earlier e-commerce business models introduced in the 1990s, such as eBay and Amazon. However, local digital marketplace platforms are relatively new in Brunei Darussalam, as most such platforms started only a few years ago, although Bruneians are used to buying products from well-known global platforms. Several reasons have caused the emergence of local digital marketplace platforms, such as the very high internet penetration, the limitations of social media in supporting business transactions, the experience in making transactions using global digital marketplaces, the impact of the COVID-19 pandemic which forced people to conduct online transactions to avoid physical contact and the availability of digital technology for setting up platforms. This chapter discusses Brunei’s digital marketplace. It examines theories of the digital marketplace, such as transaction cost economics and multisided markets. Four examples of Brunei’s digital marketplace are analysed—Al-Huffaz Management, Babakimpo, Naindah and Super Squad Soccer—and opportunities and challenges for them to survive and grow are discussed.

A two-sided market or two-sided network is made up of two distinct user groups that provide each other with network benefits in which they interact through an intermediary or platform. A digital marketplace makes use of a two-sided market where the two distinct groups are the buyers and sellers. A digital marketplace is a type of e-commerce site where the sellers offer products or services to the buyers, and transactions are controlled and processed by marketplace operators. With the rapid development and adoption of the Internet and digital marketplace globally and also regionally, businesses in Brunei Darussalam are slowly incorporating digital marketplace. This chapter provides an overview of the current state of the digital marketplace in Brunei, and thus, case studies of local digital marketplaces are discussed. A qualitative approach, which consists of interviews with companies, is made for the study. The strengths and problems of employing digital marketplace for businesses and analysis using Michael Porter's five models is also covered in this chapter.

In business, traders often do not run their business with a good will, one act that commonly found is passing off well-known mark’s good will. The phenomenon of digital disruption causes the practice of passing off not only found in conventional markets but also in electronic commerce. Digital platform marketplace is one of electronic commerce that often found an act of passing off which conducted by merchants. Based on this background, this research aims to, first gain an understanding of legal protection of well-known marks against passing off on the digital platform marketplace in Indonesia and second, gains an understanding of legal actions that can be done by well-known marks owners towards traders and digital platform marketplace providers towards passing off on the digital platform marketplace. This research is in the form of descriptive analytical using juridical-normative approach and analyzing data with normative-qualitative methods. The data collection technique used is library research by assessing secondary data. The results showed that based on Law No. 20 of 2016 on Marks and GI and Law No. 19 of 2016 concerning Amendments to Law No. 11 of 2008 on EIT, it said that the owner of a well-known marks has legal protection for the practice of passing off carried out in the digital platform marketplace and can take actions such as filing a claim for compensation by requesting compensation both material and immaterial and can file a lawsuit to the Commercial Court as a form of ultimatum remedium. In order to have a full protection, the well-known marks owner must register the trademark first and also before filing a lawsuit can make a complaint to each digital platform marketplace .

Research on multimodal data analysis such as annotated image analysis is becoming more important than ever due to the increase in the amount of data. One of the approaches to this problem is multimodal topic models as an extension of latent Dirichlet allocation (LDA). Symmetric correspondence topic models (SymCorrLDA) are state-of-the-art multimodal topic models that can appropriately model multimodal data considering inter-modal dependencies. Incidentally, hierarchically structured categories can help users find relevant data from a large amount of data collection. Hierarchical topic models such as hierarchical latent Dirichlet allocation (hLDA) can discover a tree-structured hierarchy of latent topics from a given unimodal data collection; however, no hierarchical topic models can appropriately handle multimodal data considering intermodal mutual dependencies. In this paper, we propose h-SymCorrLDA to discover latent topic hierarchies from multimodal data by combining the ideas of the two previously mentioned models: multimodal topic models and hierarchical topic models. We demonstrate the effectiveness of our model compared with several baseline models through experiments with two datasets of annotated images.

Digital Marketplace is the online platform where all organizations can use to find and buy cloud based services. Businesses leverage digital channels such as, Google search, social media, email, and their websites to connect with their current and prospective customers. Digital Marketplace uses digital technologies, mainly on the Internet but also uses mobile phones, and other digital medium. Due to increase in use and rapid growth of digital marketplaces in the world, it became essential to establish regulatory measures and standards in order to protect the rights of the consumers for the regulatory bodies of any country. The expansion of this market is due to the consumer’s choice between a digital and a physical transaction which is governed by the (i) benefits of examining a good in person, (ii) the relative hassle costs between search online and offline, (iii) the benefit of instant product availability, (iv) differences in assortment, and (v) the barriers of regulations. In a world where digital devices are ubiquitous, transactions with not-in-person component, such as flight or hotel purchases, will naturally take place digitally. On the other hand, purchases like furniture, where examining goods in person is valuable that is impossible to check through online. The growth in digital transactions can be driven by innovation spurred by competition between the marketplaces. Digital marketplace also increases the competitiveness of peer sellers and enhances the assortment of options available to the consumers. Now, millions of consumers are involved around the world with these opportunities due to the rapid growth of internet, mobile phones and other digital technologies. However, along with the growth of online activities of the consumers, some serious challenges are created by which consumers are now often cheated and becoming the victims. Therefore, question has been raised from different corners regarding the fraud, deception, and sufferings of the innocent consumers all over the world through online transactions in digital marketplace.

This study delves into the synergy between innovation management, the circular economy, and the development of digital marketplaces, with a particular focus on the reuse of electronic devices. It highlights how digitalisation within the circular economy can spur innovation in managing the reuse of electronic gadgets through digital platforms. Drawing on a robust quantitative analysis of data from over 400 Finnish and Baltic companies, this research is pioneering in its holistic integration of these sectors, addressing the critical need for insights into the operational and managerial dynamics of circular economy digital marketplaces. Employing a comprehensive quantitative approach, our investigation uncovered significant insights regarding the perceived opportunities, challenges, and value-added features of these marketplaces for electronic device reuse. Environmental protection, cost savings, and enhanced recycling opportunities were identified as major advantages by participants. However, the study also pointed out significant barriers to effective marketplace operation, including IT system integration, organisational culture challenges, and resource limitations. A noteworthy contribution of this research is its elucidation of the influence of company size on engagement in the circular economy, pinpointing large and micro-enterprises as potential pioneers in this domain. It also sheds light on the business-to-business (B2B) dynamics within digital marketplaces, exploring the interactions among sellers, buyers, and service providers. Our findings underscore digital marketplaces as pivotal enablers for sustainable practices in electronic device reuse, whereas digital platforms may expedite the transition towards a circular economy. This aligns with principles of innovation management by promoting collaboration, minimising waste, and fostering new business models. The conclusion emphasises the digital marketplace not just as a platform for transactions but as a vital ecosystem enabler, providing a wide array of value-added services. It underscores the importance for businesses, especially in the electronics sector, to integrate circular economy principles strategically, leveraging digital marketplaces to achieve sustainability goals while gaining competitive advantages. The research has directly influenced the launch of a pilot digital marketplace for electronic product reuse, showcasing the practical application of its findings. This emphasises the study’s role in advancing innovation management practices by demonstrating the transformative potential of digital marketplaces in promoting circular economy practices.

Integrating multimodal lung data including clinical notes, medical images, and molecular data is critical for predictive modeling tasks like survival prediction, yet effectively aligning these disparate data types remains challenging. We present a novel method to integrate heterogeneous lung modalities by first thoroughly analyzing various domain-specific models and selecting the optimal model for embedding feature extraction per data type based on performance on representative pretrained tasks. For clinical notes, the GatorTron models showed the lowest regression loss on an initial evaluation set, with the large GatorTron-medium model achieving 12.9 loss. After selecting the top performers, we extracted robust embeddings on the full lung dataset built using the Multimodal Integration of Oncology Data System (MINDS) framework. MINDS provides an end-to-end platform for aggregating and normalizing multimodal patient data. We aligned the multimodal embeddings to a central pre-trained language model using contrastive representation learning based on a cosine similarity loss function. To adapt the language model to the new modalities, we employed a parameter-efficient tuning method called adapter tuning, which introduces small trainable adapter layers that leave the base model weights frozen. This avoids catastrophic forgetting of the pretrained weights. We evaluated our multimodal model on prognostic prediction tasks including survival regression and subtype classification using both public and internal lung cancer datasets spanning multiple histologic subtypes and stages. Our aligned multimodal model demonstrated improved performance over models utilizing only single modalities, highlighting the benefits of integrating complementary information across diverse lung data types. This work illustrates the potential of flexible multimodal modeling for critical lung cancer prediction problems using heterogeneous real-world patient data. Our model provides a strong foundation for incorporating emerging data types, modalities, and predictive tasks in the future. Citation Format: Aakash Tripathi, Asim Waqas, Yasin Yilmaz, Ghulam Rasool. Multimodal transformer model improves survival prediction in lung cancer compared to unimodal approaches [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4905.

Performance of multimodal prediction models for intracerebral hemorrhage outcomes using real-world data.

Entity resolution (ER) is to find the data objects referring to the same real-world entity. When ER is performed on relations, the crucial operator is record matching, which is to judge whether two tuples referring to the same real-world entity. Record matching is a longstanding issue. However, with massive and complex data in applications, current methods cannot satisfy the requirements. A Sequence-rule-based record matching (SeReMatching) is presented with the consideration of both the values of the attributes and their importance in record matching. And with the help of the Bloom Filter we changed, the algorithm greatly increases the checking speed and makes the complexity of entity resolution almost O(n). And extensive experiments are performed to evaluate our methods.

With the rapid technological advances in acquiring data from diverse platforms in cancer research, numerous large scale omics and imaging data sets have become available, providing high-resolution views and multifaceted descriptions of biological systems. Simultaneous analysis of such multimodal data sets is an important task in integrative systems biology. The main challenge here is how to integrate them to extract relevant and meaningful information for a given problem. The multimodal data contains more information and the combination of multimodal data may potentially provide a more complete and discriminatory description of the intrinsic characteristics of pattern by producing improved system performance than individual modalities. In this regard, some recent advances in multimodal big data analysis for cancer diagnosis are reported in this article.

Entity resolution (ER) is to find the data objects referring to the same real-world entity. When ER is performed on relations, the crucial operator is record matching, which is to judge whether two tuples refer to the same real-world entity. Record matching is a longstanding issue. However, with massive and complex data in applications, current methods cannot satisfy the requirements. A Sequence-rule-based record matching (SeReMatching) is presented with the consideration of both which attributes should be used and their importance in record matching. We have changed the Bloom filter and therefore the checking speed is greatly increased. The best performance of the algorithm makes the complexity of entity resolution O (n). And extensive experiments were performed to evaluate our methods.

In a regular retail shop the behavior of customers may yield a lot to the shop assistant. However, when it comes to online shopping it is not possible to see and analyze customer behavior such as facial mimics, products they check or touch etc. In this case, clickstreams or the mouse movements of e-customers may provide some hints about their buying behavior. In this study, we have presented a model to analyze clickstreams of e-customers and extract information and make predictions about their shopping behavior on a digital market place. After collecting data from an e-commerce market in Turkey, we performed a data mining application and extracted online customers' behavior patterns about buying or not. The model we present predicts whether customers will or will not buy their items added to shopping baskets on a digital market place. For the analysis, decision tree and multi-layer neural network prediction data mining models have been used. Findings have been discussed in the conclusion.

Multimodal machine learning for predicting perioperative safety indicators in spinal surgery.

In reflecting upon a dichotomy which pervades the ‘brave new world’1 of the Internet and electronic commerce conducted within the global communications network, the following report written in 1999 and taken from an English website provides a nice illustration: The Queen has opened the last session of the British Parliament this century … Seated on the Throne in the House of Lords, the Upper Chamber of the Westminster Parliament, the Queen unveiled the Government’s programme for the next 12 months. The audience of ermine-clad Lords was joined by members of the House of Commons, summoned to hear the Queen’s speech. She told Parliament that there are 28 planned pieces of legislation in the Government’s third Parliament session … [The Queen said]: ‘To prepare Britain as a dynamic, knowledge-based economy, my Government will introduce a Bill to promote electronic commerce and electronic government, improving our ability to compete in the digital marketplace’.2