Legal Judgment Prediction Research Articles

A case study for automated attribute extraction from legal documents using large language models

AbstractThe escalating number of pending cases is a growing concern worldwide. Recent advancements in digitization have opened up possibilities for leveraging artificial intelligence (AI) tools in the processing of legal documents. Adopting a structured representation for legal documents, as opposed to a mere bag-of-words flat text representation, can significantly enhance processing capabilities. With the aim of achieving this objective, we put forward a set of diverse attributes for criminal case proceedings. To enhance the effectiveness of automatically extracting these attributes from legal documents within a sequence labeling framework, we propose the utilization of a few-shot learning approach based on Large Language Models (LLMs). Moreover, we demonstrate the efficacy of the extracted attributes in downstream tasks, such as legal judgment prediction and legal statute prediction.

A Circumstance-Aware Neural Framework for Explainable Legal Judgment Prediction

A Circumstance-Aware Neural Framework for Explainable Legal Judgment Prediction

Simulating judicial trial logic: Dual residual cross-attention learning for predicting legal judgment in long documents

Legal Judgment Prediction (LJP) plays a vital role in judicial assistance systems, aiming to predict judgment outcomes automatically from the fact descriptions in legal cases. A key challenge in LJP lies in effectively capturing decisive information that influences legal judgments — such as criminal events, behaviors, and consequences — especially from lengthy legal documents. Existing approaches, which incorporate domain-specific knowledge, have attempted to improve the ability to capture decisive information but often fail to adequately address the complexities of longer texts and may introduce noise, leading to incorrect predictions. To overcome these limitations, we propose a novel method, JuriSim, for predicting Chinese criminal legal judgments in long documents by incorporating the knowledge of judicial trial logic. Specifically, JuriSim extracts legal events and generates rationales based on fact descriptions to capture the decisive information that influences judgment outcomes. Then, a dual residual cross-attention mechanism is introduced to interactively process facts, events, and rationales for predicting relevant legal statutes, charges, and term of penalties. This mechanism allows the model to reduce the loss of important information and the retention of incorrect information during the aggregation process. Furthermore, we present a constrained cross-entropy loss, utilizing the topological relationship between charges, terms, and applicable law statutes. Experiments conducted on the publicly available CAIL-Long criminal dataset demonstrate the efficiency of the JuriSim framework in predicting legal judgments, especially for cases involving long documents. JuriSim_Lawformer shows a relative improvement of 3.45% in Macro-F1 for charge prediction and 3.05% for term of penalty prediction, compared to Lawformer.

Distinguish Confusion in Legal Judgment Prediction via Revised Relation Knowledge

Distinguish Confusion in Legal Judgment Prediction via Revised Relation Knowledge

A Study of Legal Judgment Prediction Based on Deep Learning Multi-Fusion Models—Data from China

Legal judgment prediction implies predicting judgment results based on the case description suitable for judgment predictions in legal terms, crimes, fines, and punishment. This study examines Chinese legal cases, focusing on legal predictions based on deep learning multiple fusion models. It aims to analyze case descriptions and predict legal terms and fines. The use of artificial intelligence models to predict the law can solve the problem of increasing workload of legal institutions and personnel because of the increasing number of cases; and simultaneously, reduce the differences in judgment. This will improve the efficiency and fairness of legal judgments. This study used the BDCI2017 dataset for experiments and applied deep learning algorithms to improve the prediction accuracy. Various models, such as TextCNN, TextRNN, Wide & TextCNN, and TextDenseNet classify cases and fines. Results revealed that TextDenseNet is better than the other model structures in terms of predictive accuracy.

Graph contrastive learning networks with augmentation for legal judgment prediction

Graph contrastive learning networks with augmentation for legal judgment prediction

HD-LJP: A Hierarchical Dependency-based Legal Judgment Prediction Framework for Multi-task Learning

In real-world scenarios, multiple subtasks of legal judgment (such as law article, charge, and term of penalty) have strict task logical order and label topological relation, and their results influence and validate each other. However, most existing methods model them as simple classification problems, which ignores the logical and semantic constraints between subtasks. Besides, they mainly focus on the fact description for judgment results, and ignore the standard legal documents (i.e., the established law articles). To this end, we propose a Hierarchical Dependency-based Legal Judgment Prediction framework (HD-LJP), which integrates task judicial logic, label topological relation, and hierarchical semantics knowledge in legal text effectively. Specifically, HD-LJP employs consistency and distinction distillation to model label topological relation among multiple subtasks, and improve the differentiation of each subtask itself respectively. In addition, for simulating the judicial logic of human judges, we define logical dependencies between subtasks, and then utilize the results of intermediate subtasks to make auxiliary prediction of other subtasks. And, hierarchical semantics knowledge is fully integrated and applied in these two processes, which will profoundly affect the creditability and interpretability of the judgment results. The experimental results show that HD-LJP can improve the prediction performance of three LJP subtasks, especially in the term of penalty. Compared with the existing methods, the macro-F1 on CAIL-small is increased by 13.4%, and 6.9% on CAIL-big. In addition, through further case studies, this paper demonstrates that HD-LJP performs better for tail classes and confusing labels than the current SOTA R-former.

Japanese tort-case dataset for rationale-supported legal judgment prediction

AbstractThis paper presents the first dataset for Japanese Legal Judgment Prediction (LJP), the Japanese Tort-case Dataset (JTD), which features two tasks: tort prediction and its rationale extraction. The rationale extraction task identifies the court’s accepting arguments from alleged arguments by plaintiffs and defendants, which is a novel task in the field. JTD is constructed based on annotated 3477 Japanese Civil Code judgments by 41 legal experts, resulting in 7978 instances with 59,697 of their alleged arguments from the involved parties. Our baseline experiments show the feasibility of the proposed two tasks, and our error analysis by legal experts identifies sources of errors and suggests future directions of the LJP research.

Problem Analysis and Application Research in Legal Judgment Prediction Based on Data Mining

The judgments of some hot cases that exceeded public expectations seem to confirm this. Although the legal certainty is questioned, the judgment remains predictable. The predictability of judgments is a way to enhance judicial authority and maintain judicial credibility. It is also a way to achieve overlapping consensus between the judiciary and the public, provide stable value guidance and behavioral expectations for the public, and promote the generation and development of public rational trust. To enhance public legal trust by improving the predictability of judgments, it is necessary to increase the burden of reasoning for judicial judgments, avoid randomness and contingency, and ensure the adequate provision and substantial disclosure of previous judgment information, so that judgment prediction in the era of big data can truly become possible. The research is to use the optimized particle swarm algorithm as the underlying model to carry out joint modeling and prediction research on the analysis and application of problems in the prediction of legal judgments. According to experimental calculations, the optimized particle swarm algorithm can significantly improve the accuracy and universality of legal judgment prediction. After optimization, the convergence speed is increased by about 5%, and the value of the acceleration factor is more significant.

Rethinking the Development of Large Language Models from the Causal Perspective: A Legal Text Prediction Case Study

While large language models (LLMs) exhibit impressive performance on a wide range of NLP tasks, most of them fail to learn the causality from correlation, which disables them from learning rationales for predicting. Rethinking the whole developing process of LLMs is of great urgency as they are adopted in various critical tasks that need rationales, including legal text prediction (e.g., legal judgment prediction). In this paper, we first explain the underlying theoretical mechanism of their failure and argue that both the data imbalance and the omission of causality in model design and selection render the current training-testing paradigm failed to select the unique causality-based model from correlation-based models. Second, we take the legal text prediction task as the testbed and reconstruct the developing process of LLMs by simultaneously infusing causality into model architectures and organizing causality-based adversarial attacks for evaluation. Specifically, we base our reconstruction on our theoretical analysis and propose a causality-aware self-attention mechanism (CASAM), which prevents LLMs from entangling causal and non-causal information by restricting the interaction between causal and non-causal words. Meanwhile, we propose eight kinds of legal-specific attacks to form causality-based model selection. Our extensive experimental results demonstrate that our proposed CASAM achieves state-of-the-art (SOTA) performances and the strongest robustness on three commonly used legal text prediction benchmarks. We make our code publicly available at https://github.com/Carrot-Red/Rethink-LLM-development.

WLEDD: Legal judgment prediction with legal feature word subgraph label-embedding and dual-knowledge distillation

Legal judgment prediction(LJP) has achieved remarkable results. However, existing methods still face problems such as difficulties in obtaining key feature words for charges, which impose limitations on the improvement of prediction results. To this end, we propose a legal judgment prediction model with legal feature Word subgraph Label-Embedding and Dual-knowledge Distillation(WLEDD). Compared with traditional methods, our method has two contributions: (1) To mitigate the impact of overly sparse tail class data and high similarity text representations, we capture the critical features related to the charges by fusing LDA and legal feature word subgraphs. Then we encode them as label information to obtain highly distinguished representations of legal documents. (2) To solve the problem of high difficulty in some subtasks in LJP, we perform subtask-oriented compression of models to construct a student model with lower complexity and higher accuracy through dual knowledge distillation. Moreover, we exploit the logical association between the subtasks to constrain the labels of articles by charge prediction results. It greatly reduces the difficulty of article prediction. Experimental results on four datasets show that our approach significantly outperforms the baseline models. Compared with the state-of-art method, the F1 value of WLEDD for charge prediction has increased by an average of 2.57% . For article prediction, the F1 value has increased by an average of 1.09% . In addition, we demonstrate its effectiveness through ablation experiments and analytical experiments.

A Neural Network model for Indian Supreme Court legal judgment prediction

Industry experts are motivated to collect, collate, and analyse historical data in the legal sector in attempt to predict court case outcomes as the amount of historical data available in this field has increased over time. But using judicial data to predict and defend court judgements is no simple undertaking. Using Machine Learning (ML) models and traditional approaches for categorical feature encoding, previous research on predicting court outcomes using limited experimental datasets produced a number of unexpected predictions. The paper proposes an ensemble model combining Convolutional Neural Network (CNN), attention mechanism and eXtreme Gradient Boosting (XGB) algorithm. This model is primarily based on a self-attention network, which could simultaneously capture linguistic relationships over lengthy sequences like RNN (Recurrent Neural Network) and is nevertheless speedy to train like CNN. C-XGB can obtain accuracy that surpasses the state-of-art model on numerous classification/prediction tasks simultaneously as being twice as speedy to train. The proposed C-XGB model is designed to process the documents hierarchically and calculates the attention weights. Two convolutional layers are used to calculate the attention weights, one at the word level and another at the sentence level. And finally, at the last layer, the XGB algorithm predicts the input case file’s outcome. The experimental results shows that the proposed model outperforms the existing model with 4.67% improvement in accuracy value.

Legal Judgment Prediction via graph boosting with constraints

Legal Judgment Prediction (LJP) is a multi-task multi-label problem in the civil law system, involving the prediction of law articles, charges, and terms of penalty based on fact descriptions. However, most existing research approaches LJP as a single-label scenario, neglecting the correlations between multiple labels and failing to consider cross-task consistency constraints in a multi-label scenario. Moreover, although previous multi-task studies have proposed expert models and coarse-grained topology construction for inter-task relationships, the former neglects rich information exchange among different tasks, and the latter, if one task’s prediction is inaccurate, will affect subsequent tasks. This paper has designed legal label graphs and proposed a novel graph boosting with constraints framework, GJudge, for legal judgment prediction to address these limitations. The framework comprises a multi-perspective interactive encoder and a multi-graph attention consistency expert module. The encoder utilizes bidirectional LSTM, gated attention units, cross attention, and graph attention networks to integrate fact descriptions and label similarity relationships information from legal label graphs for multi-perspective interactive encoding. The expert module utilizes the multiple expert networks and the multi-graph attention network to differentiate between confusing labels and ensure consistent constraints across tasks, this is achieved through the fusion of label consistency constraints and confusion relationships information in the legal label graphs. Experimental results on two real-world datasets across different tasks show an improvement in F1 scores ranging from at least 0.93% to a maximum of 2.97%, illustrating the effectiveness of GJudge compared to the state-of-the-art model.

Legal Challenges and Responses to Artificial Intelligence-Assisted Decision-Making in the International Economic Law System

Abstract Legal judgment prediction is becoming a research hotspot in the legal field as an important artificial intelligence-assisted decision-making tool in legal case management, which is able to predict judgment results. In this paper, data from the 2018 China Law Research Cup competition is gathered, and the dataset is preprocessed in the context of international economic law. Then, a multi-task model for legal verdict prediction is proposed, and the training optimization and prediction of the model are designed using CNN, RNN, and LSTM as the semantic coding layer. The model proposed in this paper achieves a significant improvement of 8% and 6% in the accuracy of the model in the prediction of the charging task and the legal sentence task, respectively. In case outcome prediction, the accuracy of the model proposed in this paper is improved by 14.6% on average compared to the feature model-based modeling approach.

Graph Reasoning With Supervised Contrastive Learning for Legal Judgment Prediction.

Given the fact descriptions of legal cases, the legal judgment prediction (LJP) problem aims to determine three judgment tasks of law articles, charges, and the term of penalty. Most existing studies have considered task dependencies while neglecting the prior dependencies of labels among different tasks. Therefore, how to make better use of the information on the relation dependencies among tasks and labels becomes a crucial issue. To this end, we transform the text classification problem into a node classification framework based on graph reasoning and supervised contrastive learning (SCL) techniques, named GraSCL. Specifically, we first design a graph reasoning network to model the potential dependency structures and facilitate relational learning under various graph topologies. Then, we introduce the SCL method for the LJP task to further leverage the label relation on the graph. To accommodate the node classification settings, we extend the traditional SCL method to novel variants for SCL at the node level, which allows the GraSCL framework to be trained efficiently even with small batches. Furthermore, to recognize the importance of hard negative samples in contrastive learning, we introduce a simple yet effective technique called online hard negative mining (OHNM) to enhance our SCL approach. This technique complements our SCL method and enables us to control the number and complexity of negative samples, leading to further improvements in the model's performance. Finally, extensive experiments are conducted on two well-known benchmarks, demonstrating the effectiveness and rationality of our proposed SCL approach as compared to the state-of-the-art competitors.

Evidence Mining for Interpretable Charge Prediction via Prompt Learning

Nowadays, more and more researchers are committed to applying artificial intelligence technology to the legal field to support decision-making. Charge prediction is a subtask of legal judgment prediction (LJP). Its purpose is to analyze the fact description of natural language text and predict a charge corresponding to a case. At present, most of the research takes charge prediction as a multiclass classification task, which leads to unsatisfactory interpretation due to the weak semantic correlation between the fact descriptions and charge labels. In order to solve this problem, we propose a method of generative evidence mining based on prompt learning. Specifically, in the training phase, we reformulate the charge labels into the prompt template that we design to enhance the semantic correlation between the charge labels and the fact descriptions. In the testing phase, the charge labels are generated via the model based on prompt learning. Meanwhile, we calculate the attention score of each sentence from the multihead self-attention in the transformer encoder and choose the sentence with the highest attention score as the evidence. Our experimental results on a real dataset show that our method is better than the traditional fine-tuning-based classification method.

LegalReasoner: A Multi-Stage Framework for Legal Judgment Prediction via Large Language Models and Knowledge Integration

LegalReasoner: A Multi-Stage Framework for Legal Judgment Prediction via Large Language Models and Knowledge Integration

Explaining legal judgments: A multitask learning framework for enhancing factual consistency in rationale generation

The explanation of legal judgments is crucial for the transparency, fairness, and trustworthiness, aiming to provide rationales for decision-making. While previous works have focused on improving the accuracy of legal judgment prediction, the lack of explainability seriously limits the practical application of these methods. Many researchers have dedicated effort to extracting or generating rationales as explanations for legal judgments, ignoring the factual consistency of these rationales. Inconsistencies between rationales and fact descriptions severely impact their applicability. To address these issues, we investigate Event Chain – ordered sequences of events related to criminal behavior – as an intermediate representation in the fact descriptions, to enhance the representation of causal relationships among events, and focus on crucial events within the fact description. Specifically, we propose a multi-task learning approach, dubbed LegalMind, that introduces Event Chain as an auxiliary task and jointly models Event Chain and rationale in a unified decoder, to improve factual consistency in rationale generation. The experiment results show that our model outperforms the state-of-the-art methods, achieving improvements of 7.65% and 6.65% in AVG-BLEU on the CJO dataset and LAIC2021 dataset respectively compared to BART-C3VG. Furthermore, in comparison to BART-C3VG, our model demonstrated superior performance with increases of 6.17% and 8.33% in factual consistency on the CJO and LAIC2021 datasets, respectively.

Legal knowledge management for prosecutors based on judgment prediction and error analysis from indictments

Legal AI aims to provide improved knowledge management services based on legal documents. Existing legal judgment prediction datasets mainly use court verdicts. However, for prosecutors, the use of indictments for judgment predictions can help detecting inconsistencies between predictions and prosecution, providing prosecutors with more accurate references to laws and charges through error analysis. In this study, we collect a dataset called TWLJP, which contains 342,754 indictments. We compared three possible messaging passing architectures among the law, regulation, and accusation cause prediction tasks, i.e. independent, topological, and interactive. The result shows that interactive message passing among the three tasks achieved the best Macro-F1 performance of 95.2 %, 79.62 %, and 65.84 % for laws, regulations, and accusation cause prediction, respectively. We further improve the prediction of accusation cause from 8.8 % macro-F1 to 62.3 % for underperformed accusation causes via Prompt-Based Learning. Finally, in view of the situation where the charge prediction are written in various ways, we adopted a lenient approach to assess the accusation and improved the accusation performance to 77.2 %.

Chinese legal judgment prediction via knowledgeable prompt learning

In recent years, applying AI techniques in the legal field has attracted researchers’ attention. In particular, Legal Judgment Prediction (LJP), which aims to predict accusations based on given case description texts, has attracted much attention from the natural language processing community. However, most of the existing LJP methods are data-intensive. As we know, data annotation in the legal field is expensive. Prompt learning is a recently prevalent methodology, which often achieves surprising results in few-shot or even zero-shot scenarios. We propose a novel method for Chinese LJP based on prompt learning called KnowPrompt4LJP. The method aligns the Chinese LJP task with the pre-training task of a Pre-trained Language Model (PLM) via a prompt template to stimulate the PLM’s recall of learned knowledge. In addition, the well-designed prompt template can enhance the PLM’s understanding of the Chinese LJP task. We also use an external knowledge base to extract keyword information from the Chinese case description texts and incorporate it into the prompt template, thus enhancing the guidance of the prompt template to the PLM. Experimental results on CAIL2018, a high-quality Chinese LJP competition dataset, show that KnowPrompt4LJP achieves far better results than the baselines in zero-shot, few-shot, and full-size training data scenarios. KnowPrompt4LJP can achieve a macro F1 value of 0.70 in the low-resource scenario, which is comparable to the baselines’ results in the data-rich scenario. In the scenario of using full-size training data, KnowPrompt4LJP can achieve a macro F1 value of 0.81.