GATE team hosts its first ATRIUM TNA research visit: Using NLP to understand trends in political and social debate

In December 2024, we hosted research visitor Tasos Galanopoulos as part of  the ATRIUM project (Advancing fronTier Research In the arts and hUManities) TransNational Access scheme. ATRIUM's aim is to bridge 4 leading research infrastructures in: arts and humanities (DARIAH), archaeology (ARIADNE), language technology (CLARIN), and open scholarly communication in the social sciences and humanities (OPERAS). The Transnational Access (TNA) scheme offers fully funded placements for researchers across Europe. This initiative is designed to support Arts and Humanities researchers by providing access to expert knowledge, mentorship, and tools from leading Data Management organisations. Successful applicants have the opportunity to visit one of 14 different host organisations across Europe in order to conduct their research, benefiting from direct contact, knowledge sharing and network building. 

Tasos describes his visit below...

How can NLP tools and large language models be used to understand trends in political and social debate around major issues of the day? 

 What is the relationship between 'distant reading' and the layered understanding that these tools offer for large volumes of data, and 'close reading', understanding aspects of these topical issues?  

What role can these modern tools play in the humanities and in everyday journalistic practice?  

Questions such as these, on the occasion of a project on "Analysis of textual data from newspapers on the agreement of Greece's accession to the European Economic Community EEC (1961)", in the context of my postgraduate studies in Digital Humanities at the Open University of Greece, brought me to the School of Computer Science at the University of Sheffield at the end of November (23/11/2024 - 7/12/2024), to collaborate with members of the GATE team.

Despite the short period of the stay, the impressions were the best: the patience and goodwill of all the team - with Dr Maynard at the forefront - helped me to "navigate" the tools offered by the GATE Cloud and the European Language Grid, to understand a bit better the processes required, and the wider field, to learn a bit more about its "alphabet" and requirements. At the same time, through the regular meetings of the team I was able to get a "glimpse" of the modern, specialised, and valuable research being carried out at the university.  

In relation to the actual subject of the research, the findings from the processing with tools such as NamedEntity Recognition, N-gram detection and their visualization with wordclouds, Topic Classification, Sentiment Analysis, Multidimensional analysis with LIWC-22, Persuasion techniques were very interesting, giving answers and insights to our questions that had to do with the attempt to develop a methodology to identify, document and frame named entities in the context of the investigation of public discourse, Press with different political orientation and political rhetoric in relation to critical events in political life, with reference to the economic and social environment inside and outside the country. Also "identifying" and categorising arguments for and against, and 'bias' for/against in the Press of that time and at a subsequent level , enabled us to explore ways to link entities to key concepts in argumentation.

 Overall, my impressions were therefore the best from this constructive visit, a visit that on a personal level gave me inspiration and opened new horizons, but also created new contacts with remarkable people.

Adapted TextRank for Term Extraction: A Generic Method of Improving Automatic Term Extraction Algorithms

This summer, we presented some of our latest work at SEMANTiCS 2018 in Vienna: "Adapted TextRank for Term Extraction: A Generic Method of Improving Automatic Term Extraction Algorithms".

Zhang, Z., Petrak, J. & Maynard, D. Adapted TextRank for Term Extraction: A Generic Method of Improving Automatic Term Extraction Algorithms. in SEMANTiCS 2018 – 14th International Conference on Semantic Systems 00, 0-000 (2018).

This work has been carried out in the context of the EU KNOWMAK project, where we're developing tools for multi-topic classification of text against an ontology, in order to attempt to map the state of European research output in key technologies.

Automatic Term Extraction (ATE) is a fundamental technique used in computational linguistics for recognising terms in text. Processing the collected terms in a text is a key step in understanding the content of the text.  There are many different ATE methods, but these all tend to work well only in a one specific domain.  In other words, there is no universal method which produces consistently good results, and so we have to choose an appropriate method for the domain being targeted.

In this work, we have developed a novel method for ATE which addresses two major limitations: the fact that no single ATE method consistently performs well across all domains, and the fact that the majority of ATE methods are unsupervised. Our generic method, AdaText, improves the accuracy of existing ATE methods, using existing lexical resources to support them, by revising the TextRank algorithm.

After being given a target text, AdaText:

1. Selects a subset of words based on their semantic relatedness to a set of seed words or phrases relevant to the domain, but not necessarily representative of the terms within the target text. 
2. It then applies an adapted TextRank algorithm to create a graph for these words, and computes a text-level TextRank score for each selected word. 
3. Finally, these scores are used to revise the score of a term candidate previously computed by an ATE method. 

This technique was trialled using a variety of parameters (such as the threshold of semantic similarity to select words, as described in step two) over two distinct datasets (GENIA and ACLv2, comprising Medline abstracts and abstracts from ACL respectively). We also tested it with a wide variety of state of the art ATE methods, including modified TFIDF, CValue, Basic, RAKE, Weirdness, LinkProbability, X², GlossEx and PositiveUnlabeled.

The figures show a sample of performances in different datasets and using different ATE techniques. The base performance of the ATE method is represented by the blachttps://gate.ac.uk/g8/page/show/2/sale/images/blog/Results-by-AdaText-compared-against-the-base-ATE-methods-y-axis-average-PK-for-all.pngk horizontal line. The horizontal axis represents the semantic similarity threshold used in step 1. The vertical axis shows average P@K for all five Ks considered.

This new generic combination approach can consistently improve the performance of the ATE method by 25 points, which is a significant increase. However, there is still room for improvement. In future work, we aim to optimise the selection of words from the TextRank graph, work on expanding TextRank to a graph of both words and phrases, and to explore how the size and source of the seed lexicon affects the performance of AdaText.