Historical Newspaper Collections Research Articles

Understanding the social construction of juvenile delinquency: insights from semantic analysis of big-data historical newspaper collections

AbstractMassive historical newspaper collections contain rich information about the historical development of social issues and constitute a unique resource for studying the social construction of issues such as juvenile delinquency. However, manual analysis of millions of pages of newspaper articles is infeasible. In this paper, we propose a suite of computational methods, including cross-context lexical analysis, dynamic semantic analysis, and valence analysis, to facilitate the study of historical social construction. We apply these methods to ProQuest Historical Newspapers$$^{\textrm{TM}}$$ TM collection in the period of 1790–2006 to study the social construction of juvenile delinquency over this period. Our results show that the proposed methods are effective in revealing insights regarding the social construction of juvenile delinquency, leading to a better understanding of this complex issue and specific hypotheses for further study. Overall, our study shows the great promise of leveraging natural language processing techniques for analyzing historical news data to study social construction of societal issues.

Applying Image Analysis and Machine Learning to Historical Newspaper Collections

Abstract Diving below the surface has its challenges, however. For example, “noise effects” are especially widespread when digital images have been created from earlier microphotographic copies, as is common in historical newspaper collections. Noise effects introduce interference to the primary signals of the pages, both for human vision and computer vision and processing. Various types of noise effects (fig. 1) are common, including unevenly distributed luminosity (i.e., range effects), visible characters from the other side of the page (bleed-through), tilted document scans (skewed orientation), and markings on the newspaper that obscure text (blobs).1 There is a wide range of severity for each of these effects, and images can range from very clean to very noisy within and across datasets.

Computer vision and machine learning approaches for metadata enrichment to improve searchability of historical newspaper collections

PurposeHistorical newspaper collections provide a wealth of information about the past. Although the digitization of these collections significantly improves their accessibility, a large portion of digitized historical newspaper collections, such as those of KBR, the Royal Library of Belgium, are not yet searchable at article-level. However, recent developments in AI-based research methods, such as document layout analysis, have the potential for further enriching the metadata to improve the searchability of these historical newspaper collections. This paper aims to discuss the aforementioned issue.Design/methodology/approachIn this paper, the authors explore how existing computer vision and machine learning approaches can be used to improve access to digitized historical newspapers. To do this, the authors propose a workflow, using computer vision and machine learning approaches to (1) provide article-level access to digitized historical newspaper collections using document layout analysis, (2) extract specific types of articles (e.g. feuilletons – literary supplements from Le Peuple from 1938), (3) conduct image similarity analysis using (un)supervised classification methods and (4) perform named entity recognition (NER) to link the extracted information to open data.FindingsThe results show that the proposed workflow improves the accessibility and searchability of digitized historical newspapers, and also contributes to the building of corpora for digital humanities research. The AI-based methods enable automatic extraction of feuilletons, clustering of similar images and dynamic linking of related articles.Originality/valueThe proposed workflow enables automatic extraction of articles, including detection of a specific type of article, such as a feuilleton or literary supplement. This is particularly valuable for humanities researchers as it improves the searchability of these collections and enables corpora to be built around specific themes. Article-level access to, and improved searchability of, KBR's digitized newspapers are demonstrated through the online tool (https://tw06v072.ugent.be/kbr/).

A data-driven approach to studying changing vocabularies in historical newspaper collections

A data-driven approach to studying changing vocabularies in historical newspaper collections

A data-driven approach to studying changing vocabularies in historical newspaper collections

Abstract Nation and nationhood are among the most frequently studied concepts in the field of intellectual history. At the same time, the word ‘nation’ and its historical usage are very vague. The aim in this article was to develop a data-driven method using dependency parsing and neural word embeddings to clarify some of the vagueness in the evolution of this concept. To this end, we propose the following two-step method. First, using linguistic processing, we create a large set of words pertaining to the topic of nation. Second, we train diachronic word embeddings and use them to quantify the strength of the semantic similarity between these words and thereby create meaningful clusters, which are then aligned diachronically. To illustrate the robustness of the study across languages, time spans, as well as large datasets, we apply it to the entirety of five historical newspaper archives in Dutch, Swedish, Finnish, and English. To our knowledge, thus far there have been no large-scale comparative studies of this kind that purport to grasp long-term developments in as many as four different languages in a data-driven way. A particular strength of the method we describe in this article is that, by design, it is not limited to the study of nationhood, but rather expands beyond it to other research questions and is reusable in different contexts.

Searchable Turkish OCRed historical newspaper collection 1928–1942

The newspaper emerged as a distinct cultural form in early 17th-century Europe. It is bound up with the early modern period of history. Historical newspapers are of utmost importance to nations and its people, and researchers from different disciplines rely on these papers to improve our understanding of the past. In pursuit of satisfying this need, Istanbul University Head Office of Library and Documentation provides access to a big database of scanned historical newspapers. To take it another step further and make the documents more accessible, we need to run optical character recognition (OCR) and named entity recognition (NER) tasks on the whole database and index the results to allow for full-text search mechanism. We design and implement a system encompassing the whole pipeline starting from scrapping the dataset from the original website to providing a graphical user interface to run search queries, and it manages to do that successfully. Proposed system provides to search people, culture and security-related keywords and to visualise them.

Pirates of Empire: Colonisation and Maritime Violence in Southeast Asia, by Stefan Eklöf Amirell

Stefan Eklöf Amirell’s monograph explores the intersections of maritime violence and imperial expansion in South-east Asia, primarily in the period 1850 to 1920. Whereas previous piracy studies have centred on a single colonial power or sub-region, Amirell’s work encompasses five colonial powers (Spain, the USA, the Netherlands, Britain and France) in addition to Asian states (including the Sulu Sultanate and imperial China and Vietnam) and three allegedly ‘pirate-infested’ zones: the Sulu Sea, the Strait of Malacca and Indochina. In the introduction and first chapter, Amirell provides an overview of the history and historiography of regional and global piracy. The remaining three chapters are organised geographically, with each examining the entangled evolution of piracy and colonial rule in one of these piracy zones. The ‘digital revolution’ made this project possible: Amirell’s analysis draws on digitised imperial archives such as Spain’s PARES platform and historical newspaper collections. Amirell demonstrates that the piracy...

How to Do Lexical Quality Estimation of a Large OCRed Historical Finnish Newspaper Collection with Scarce Resources

The National Library of Finland has digitized and made available the historical newspapers published in Finland between 1771 and 1910 (Bremer-Laamanen 2014; Kettunen et al. 2014). This collection contains approximately 1.95 million pages in Finnish and Swedish. The Finnish part of the collection consists of about 2.40 billion words. The National Library’s Digital Collections are offered via the digi.kansalliskirjasto.fi web service, also known as Digi. An open data package of the whole collection was released in early 2017 (Paakkonen et al. 2016). Quality of OCRed collections is an important topic in digital humanities, as it affects general usability and searchability of collections. There is no single available method to assess quality of large collections, but different methods can be used to approximate quality. This paper discusses different corpus analysis style methods to approximate overall lexical quality of the Finnish part of the Digi collection. Methods include usage of parallel samples and word error rates, usage of morphological analysers, frequency analysis of words and comparisons to comparable edited lexical data. Our aim in the quality analysis is twofold: firstly to analyse the present state of the lexical data and secondly, to establish a set of assessment methods that build up a compact procedure for quality assessment after e.g. re-OCRing or post-correction of the material. ResumeLa Bibliotheque nationale de Finlande a numerise et rendu disponible les journaux historiques publies en Finlande entre 1771 et 1910 (Bremer-Laamanen 2014 ; Kettunen et al. 2014). Cette collection contient environ 1,95 million pages en finnois et suedois. La partie finnoise de la collection compte environ 2,40 milliards de mots. Les Collections numerisees de la Bibliotheque Nationale sont offertes sur le service web digi.kansalliskirjasto.fi, egalement appele Digi. Un ensemble de donnees disponibles de la collection entiere est sorti debut 2017 (Paakkonen et al. 2016).La qualite de collections en OCR est un theme important pour les humanites numeriques, puisqu’elle concerne l’utilite et la facilite de recherche de collections. Il n’y a pas qu’une seule methode pour evaluer la qualite de grandes collections, mais des methodes differentes peuvent etre employees pour en estimer la qualite. Cet article discute de methodes differentes d’analyses de corpus visant a estimer la qualite lexicale totale de la partie finnoise de la collection Digi. Les methodes comprennent l’usage d’echantillons paralleles et de frequences d’erreur de mot, l’usage d’analyseurs morphologiques, l’analyse de frequence de mots et les comparaisons a des donnees lexicales redigees comparables. Notre objectif dans l’analyse de qualite est double : premierement, analyser l’etat actuel des donnees lexicales et, deuxiemement, etablir un ensemble de methodes d’evaluation qui constituent une procedure compacte pour l’evaluation de la qualite apres, par exemple, la retransformation en OCR ou apres les apres corrections du materiel. Mots-cles: qualite d’OCR; estimation de qualite lexicale; collection de journaux finnois du 19e siecle

Metadata categorization for identifying search patterns in a digital library

PurposeFor digital libraries, it is useful to understand how users search in a collection. Investigating search patterns can help them to improve the user interface, collection management and search algorithms. However, search patterns may vary widely in different parts of a collection. The purpose of this paper is to demonstrate how to identify these search patterns within a well-curated historical newspaper collection using the existing metadata.Design/methodology/approachThe authors analyzed search logs combined with metadata records describing the content of the collection, using this metadata to create subsets in the logs corresponding to different parts of the collection.FindingsThe study shows that faceted search is more prevalent than non-faceted search in terms of number of unique queries, time spent, clicks and downloads. Distinct search patterns are observed in different parts of the collection, corresponding to historical periods, geographical regions or subject matter.Originality/valueFirst, this study provides deeper insights into search behavior at a fine granularity in a historical newspaper collection, by the inclusion of the metadata in the analysis. Second, it demonstrates how to use metadata categorization as a way to analyze distinct search patterns in a collection.

100 Years of Solitude: A Century of Newspapers in the David M. Rubenstein Rare Book and Manuscript Library

ABSTRACTIn 1898, the Trinity College Historical Society began collecting the first newspapers that would become the foundation of the Rubenstein Rare Book and Manuscript Library's historical print newspaper collection. In recent decades, these newspapers lay underdescribed and largely inaccessible in a Rubenstein alcove. Impending renovation forced action, and in 2012 a three-year, multidepartment project to describe and house the collection was launched. In January 2016, the project successfully concluded, providing for the first time intellectual and physical control of the collection. Four years' worth of circulation data show steady and consistent usage on par with the greater Rubenstein collection.

Information retrieval from historical newspaper collections in highly inflectional languages: A query expansion approach

The aim of the study was to test whether query expansion by approximate string matching methods is beneficial in retrieval from historical newspaper collections in a language rich with compounds and inflectional forms (Finnish). First, approximate string matching methods were used to generate lists of index words most similar to contemporary query terms in a digitized newspaper collection from the 1800s. Top index word variants were categorized to estimate the appropriate query expansion ranges in the retrieval test. Second, the effectiveness of approximate string matching methods, automatically generated inflectional forms, and their combinations were measured in a Cranfield‐style test. Finally, a detailed topic‐level analysis of test results was conducted. In the index of historical newspaper collection the occurrences of a word typically spread to many linguistic and historical variants along with optical character recognition (OCR) errors. All query expansion methods improved the baseline results. Extensive expansion of around 30 variants for each query word was required to achieve the highest performance improvement. Query expansion based on approximate string matching was superior to using the inflectional forms of the query words, showing that coverage of the different types of variation is more important than precision in handling one type of variation.

A National Library for the 21st century – knowledge and cultural heritage online

Since 2004, the National Library of Norway has worked systematically to develop a digital National Library. The core of this activity has been to digitize the whole collection and to seek ways to disseminate the digital content to the biggest possible audience. Being a true multimedia library, the collection holds published material in a wide spectrum of media: from traditional printed material to radio, TV, audio, film and music. The library also harvests the Norwegian internet domain ‘.no’. Today the digital library contains 2.8 million digital objects, which adds up to 4000 TB of unique digital content. By use of extended collective licensing everyone with a Norwegian IP address can freely access and read all books published in Norway up to 2001. In addition, through agreements with the major Norwegian newspapers, all Norwegian libraries can offer access to complete digital historical newspaper collections. The most important matter to make this happen has been a strategic partnership and cooperation with rights holders and publishers. This partnership has built trust and confidence and has resulted in a cooperation where knowledge and costs are shared. The National Library of Norway offers digital library services that provide online access to full-text versions of all books published in Norway up until the year 2001, access to digital newspaper collections from the major national and regional newspapers in all libraries in the country, and opportunities for everyone with internet access to search and listen to more than 40,000 radio programmes recorded between 1933 and the present day. The foundation for this development was laid in 2004 when the National Library adopted a digitization strategy for the creation of a National Library for the 21st century. This was later followed up by the decision in principle in 2006 to digitize the entire collection.

Tramping Through the Baseball Subculture: <i>The Career of Alfred W. Lawson</i>

Since his death in 1954, Alfred W. Lawson has gained renown as a notable American eccentric. This reputation is mainly due to his post-baseball career exploits as a commercial aviation pioneer, Depression-era populist social reformer, and writer of books detailing his all-encompassing folk philosophy of Lawsonomy.1 Major league record keeping has long confirmed his three-game cup of coffee in the tumultuous National League season of 1890, but further details of his 23 years in baseball have been slow to emerge. Until recently, the primary resource for his sporting career has been a handful of selective highlights mentioned by Lawson himself in autobiographical sections of his books. The advent of digitized historical newspaper collections has revealed rich new insights into Lawson's career as player, manager, league organizer, and promoter. These reports reveal a prototypical 19th century ballplayer, as well as an adept innovator of baseball economics, promotion, and marketing.Like the majority of 19th century professional baseball players, Lawson came from a lower-class background. His family had emigrated from the East End of London and settled in Detroit, Michigan. As a child he contributed to the family economy as a bootblack and newspaper boy; he also worked in his father's rag carpet home factory. Lawson attended elementary and trade school (participating in student athletics), but his wanderlust led him to leave home at age 17 to seek his fortune. By his own account, in 1887 he arrived via freight train at Frankfort, Indiana, and successfully pitched a game for their independent club against a heavily favored league club. He was rewarded with a $40-per-month contract. That same summer, he moved from the Frankfort club to a team in South Bend.2In the late 1880s, the Upper Midwest region where Alfred Lawson started playing baseball supported early incarnations of regional minor leagues: the Western League (with cities as far-flung as Denver and Milwaukee); the Interstate League (Iowa, Illinois, and Indiana); the Michigan League; the Ohio League; and the International League (western New York, Ontario, northern Ohio). The composition of these regional leagues fluctuated from year to year-if they survived more than one season. Larger cities sup- ported multiple semipro local leagues. Chicago, for example, had a City League and a handful of company-sponsored leagues, including the Garden City League, the Market Street League, and the Wholesale Grocers League. The sports pages from cities with local leagues would often announce open dates as opportunities for unscheduled games, or announce the hastily arranged games themselves.Lawson played for eight different teams over the next two years (1888-1889), moving between towns in Indiana, Illinois, and Wisconsin. The reasons for the jumps probably varied: better contract offers, league or club failures, or disagreements with management. The number of clubs he played for was probably not unusual for itinerant professional ballplayers. It is possible that injuries or poor performances might have caused him to be released from some teams, but the incomplete records that have been uncovered so far indicate that he was a pitcher whose talent was on the rise. His appearances for Sterling (Illinois) and Appleton (Wisconsin) in 1889 were phenomenal.3 It is likely that by the time Appleton's season ended at the end of September, Lawson had been advised to head south to play winter ball in Florida in order to hone his skills for the next season.One can not imagine a better year for a young ballplayer to hope to get his first major league contract than the year 1890. The revolt of the major league union, the Brotherhood of Professional Base Ball Players, led to the formation of the Players League that year, which depleted the National League of more than 55 percent of its players and took a similar percentage from the American Association. Consequently, the NL and the AA were scouring the nation for young talent in the early months of 1890. …

Connecting to the past: Newspaper digitization in the Nordic countries

Digital newspaper projects are a hot topic around the world. One pioneer in the field of digitizing newspapers is the Nordic TIDEN project, launched in 2001. Today the amount of online pages has risen from 400,000 at launch to 1.6 million. In Finland and Sweden, full text search is available. The Historical Newspaper Library in Finland is highly regarded among researchers and the public. It is easy to use in its Google-like approach. The users are connected to places, questions, nations and human life over centuries. The process of digitizing newspapers includes microfilming, digitization and optical character recognition, identification and database import. The speed and automation of the production environment is essential. The automated processes of today offer the coordinates to each word in the paper and the users get better service as the words are highlighted. Images are of a better quality as grayscale images can be handled automatically. In digital futures, the digital collection management will be an exciting challenge. The infrastructure is changing fast as well as the expectations. We have to improve access in various ways, including automatic translation to old holdings and meeting places for users: connecting them to each other and to holdings in other heritage sectors. Newspaper collections are targets that have a great demand on the internet from researchers as well as the public at large. Our digitized historical newspaper collections and the born digital ones are connecting the users to places, questions, nations and human life over centuries. Incidents from the past are suddenly easily accessible. The past is living in the present.