An evaluation of NERC learning-based approaches to discover personal data in Brazilian Portuguese documents

Abstract

Governments are issuing regulations and laws demanding that companies protect collected and processed personal data. In Brazil, the federal government sanctioned the General Personal Data Protection law, which defines personal and sensitive data associated with Brazilian citizens. One existing barrier for companies to comply with the law is identifying where personal data is stored inside their infrastructure, mainly concerning personal data inserted into unstructured documents. Named Entity Recognition and Classification (NERC) can support companies in this task by implementing supervised learning models to identify personal data. In this study, we designed an experiment to evaluate machine learning-based NERC using BERT and LSTM approaches to recognize personal data related to Brazil’s context. We established a generic training corpus based on online documents and trained two models for each approach, one considering the original corpus and another after lowercasing it. The study also assessed relation extraction to differentiate personal entities from others. For NERC and relation extraction evaluations, we established a training corpus comprising documents from two organizations related to the education and health sectors. BERT fine-tuned with uncased corpus scored an F1 measure of 0.8 and achieved the best performance in recognizing entities, followed by LSTM based on the same corpus. After applying relation extraction, BERT models achieved better F1 scores than LSTM models. The uncased BERT model achieved an F1 score of 0.85, which was the best. Experiment results also indicated that relation extraction improves the performance of BERT models to discover personal entities.