Simplification Data: Difference between revisions

From Clarin K-Centre
Jump to navigation Jump to search
Line 32: Line 32:
==Dutch municipal data==
==Dutch municipal data==


Repository that contains data and links to resources related to the task of simplifying Dutch (municipal) texts.
The Dutch municipal corpus is a parallel monolingual corpus for the evaluation of sentence-level simplification in the Dutch municipal domain. The corpus was created by Amsterdam Intelligence. It contains 1,311 translated parallel sentence pairs that were automatically aligned. The sentence pairs originate from 50 documents from the Communications Department of the City of Amsterdam that were manually simplified to evaluate simplification for Dutch.


*265 KB
*[https://github.com/Amsterdam-AI-Team/dutch-municipal-text-simplification Github]
*[https://github.com/Amsterdam-AI-Team/dutch-municipal-text-simplification Github]


</translate>
</translate>

Revision as of 09:56, 17 June 2024

ASSET Simplification Corpus

The ASSET simplification corpus (Alva-Manchego et al, 2020) was automatically translated to Dutch (Seidl et al., 2023), and is freely available.

  • Github download
  • Alva-Manchego, F., Martin, L., Bordes, A., Scarton, C., Sagot, B., & Specia, L. (2020). ASSET: A dataset for tuning and evaluation of sentence simplification models with multiple rewriting transformations. arXiv preprint arXiv:2005.00481.
  • Seidl, T., Vandeghinste, V., & Van de Cruys, T. (2023). Controllable Sentence Simplification in Dutch. KU Leuven. Faculteit Ingenieurswetenschappen.

Wikilarge Dataset

Automatic translation of the Wikilarge dataset, useful for automatic simplification (Seidl et al., 2023), freely available. Original dataset from Zhang & Lapata

  • Github download
  • Seidl, T., Vandeghinste, V., & Van de Cruys, T. (2023). Controllable Sentence Simplification in Dutch. KU Leuven. Faculteit Ingenieurswetenschappen.
  • Zhang, X. & Lapata, M. (2017). Sentence Simplification with Deep Reinforcement Learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 584–594, Copenhagen, Denmark. Association for Computational Linguistics.

Comparable Corpus Wablieft De Standaard

UWV Leesplank NL wikipedia

The set contains 2,391,206 pragraphs of prompt/result combinations, where the prompt is a paragraph from Dutch Wikipedia and the result is a simplified text, which could include more than one paragraph. This dataset was created by UWV, as a part of project "Leesplank", an effort to generate datasets that are ethically and legally sound.

Dutch municipal data

The Dutch municipal corpus is a parallel monolingual corpus for the evaluation of sentence-level simplification in the Dutch municipal domain. The corpus was created by Amsterdam Intelligence. It contains 1,311 translated parallel sentence pairs that were automatically aligned. The sentence pairs originate from 50 documents from the Communications Department of the City of Amsterdam that were manually simplified to evaluate simplification for Dutch.