Parallel Multilingual Corpora: Difference between revisions

From Clarin K-Centre
Jump to navigation Jump to search
VisualWikitext
No edit summary
Line 1: Line 1:
<translate>
==EDGeS Diachronic Bible Corpus==
==EDGeS Diachronic Bible Corpus==
A diachronically and synchronically parallel corpus of Bible translations in Dutch, English, German and Swedish, with texts from the 14th century until today.  
A diachronically and synchronically parallel corpus of Bible translations in Dutch, English, German and Swedish, with texts from the 14th century until today.  


Line 5: Line 8:


==PacoMT Parallel Corpora==
==PacoMT Parallel Corpora==
During the STEVIN project PaCo-MT (Parse and Corpus-based Machine Translation), two existing parallel corpora were enriched with syntactic annotations and node alignments. The annotations were generated automatically.
During the STEVIN project PaCo-MT (Parse and Corpus-based Machine Translation), two existing parallel corpora were enriched with syntactic annotations and node alignments. The annotations were generated automatically.


Line 25: Line 29:


==The Open Parallel Corpus (OPUS)==
==The Open Parallel Corpus (OPUS)==
The [https://opus.nlpl.eu/ OPUS corpus] contains a very large collection of parallel corpora, amongst which many contain Dutch.
The [https://opus.nlpl.eu/ OPUS corpus] contains a very large collection of parallel corpora, amongst which many contain Dutch.


==COVID-19 Corpora==
==COVID-19 Corpora==
===COVID-19 - HEALTH Wikipedia dataset. Bilingual (EN-NL)===
===COVID-19 - HEALTH Wikipedia dataset. Bilingual (EN-NL)===
Bilingual (EN-NL) corpus acquired from Wikipedia on health and COVID-19 domain (2nd May 2020). The corpus contains 931 translation units.
Bilingual (EN-NL) corpus acquired from Wikipedia on health and COVID-19 domain (2nd May 2020). The corpus contains 931 translation units.


Line 35: Line 42:


===COVID-19 ANTIBIOTIC dataset. Bilingual (EN-NL)===
===COVID-19 ANTIBIOTIC dataset. Bilingual (EN-NL)===
Bilingual (EN-NL) corpus acquired from the website https://antibiotic.ecdc.europa.eu/. The corpus contains 805 translation units.
Bilingual (EN-NL) corpus acquired from the website https://antibiotic.ecdc.europa.eu/. The corpus contains 805 translation units.


Line 41: Line 49:


=== COVID-19 EC-EUROPA v1 dataset. Bilingual (EN-NL)===
=== COVID-19 EC-EUROPA v1 dataset. Bilingual (EN-NL)===
Bilingual (EN-NL) corpus acquired from website (https://ec.europa.eu/*coronavirus-response) of the EU portal (20th May 2020). This corpus contains 2.391 translation units.
Bilingual (EN-NL) corpus acquired from website (https://ec.europa.eu/*coronavirus-response) of the EU portal (20th May 2020). This corpus contains 2.391 translation units.


Line 47: Line 56:


===COVID-19 EU presscorner v2 dataset. Bilingual (EN-NL)===
===COVID-19 EU presscorner v2 dataset. Bilingual (EN-NL)===
Bilingual (EN-NL) corpus acquired from website (https://ec.europa.eu/commission/presscorner/) of the EU portal (8th July 2020). This corpus contains 6.810 translation units.
Bilingual (EN-NL) corpus acquired from website (https://ec.europa.eu/commission/presscorner/) of the EU portal (8th July 2020). This corpus contains 6.810 translation units.


Line 53: Line 63:


===COVID-19 EUR-LEX dataset. Βilingual (EN-NL)===
===COVID-19 EUR-LEX dataset. Βilingual (EN-NL)===
Bilingual (EN-NL) corpus acquired from website (https://eur-lex.europa.eu/legal-content) of the EU portal (9th July 2020). This corpus contains 22.470 translation units.
Bilingual (EN-NL) corpus acquired from website (https://eur-lex.europa.eu/legal-content) of the EU portal (9th July 2020). This corpus contains 22.470 translation units.


Line 59: Line 70:


===COVID-19 EUROPARL v2 dataset. Bilingual (EN-NL) ===
===COVID-19 EUROPARL v2 dataset. Bilingual (EN-NL) ===
Bilingual (EN-NL) corpus acquired from the website (https://www.europarl.europa.eu/) of the European Parliament (9th May 2020). This corpus contains 887 translation units.
Bilingual (EN-NL) corpus acquired from the website (https://www.europarl.europa.eu/) of the European Parliament (9th May 2020). This corpus contains 887 translation units.


Line 65: Line 77:


===COVID-19 Parallel Global Voices dataset. Bilingual (EN-NL)===
===COVID-19 Parallel Global Voices dataset. Bilingual (EN-NL)===
EN-NL Bilingual COVID-19-related corpus acquired from the website (https://globalvoices.org/) of GlobalVoices (28th April 2020). This corpus contains 675 translation units.
EN-NL Bilingual COVID-19-related corpus acquired from the website (https://globalvoices.org/) of GlobalVoices (28th April 2020). This corpus contains 675 translation units.


Line 71: Line 84:


==Bilingual corpus from the European Vaccination Portal (NL-EN)==
==Bilingual corpus from the European Vaccination Portal (NL-EN)==
NL-EN Bilingual corpus acquired from https://vaccination-info.eu. This corpus contains 494 translation units.
NL-EN Bilingual corpus acquired from https://vaccination-info.eu. This corpus contains 494 translation units.


Line 77: Line 91:


==Bilingual corpus from the Publications Office of the EU on the medical domain v.2 (EN-NL) ==
==Bilingual corpus from the Publications Office of the EU on the medical domain v.2 (EN-NL) ==
EEN-NL Bilingual corpus extracted from the Publications Office of the EU on the medical domain. These are sourced from laws, studies, EC announcements, etc. labelled with concepts like epidemiology, epidemic, disease surveillance, health control, public hygiene, freedom of movement, distance learning, etc. This corpus contains 13.191 translation units.
EEN-NL Bilingual corpus extracted from the Publications Office of the EU on the medical domain. These are sourced from laws, studies, EC announcements, etc. labelled with concepts like epidemiology, epidemic, disease surveillance, health control, public hygiene, freedom of movement, distance learning, etc. This corpus contains 13.191 translation units.


Line 83: Line 98:


==Bilingual corpus made out of PDF documents from the European Medicines Agency, (EMEA) (EN-NL).==
==Bilingual corpus made out of PDF documents from the European Medicines Agency, (EMEA) (EN-NL).==
EN-NL Bilingual corpus made out of PDF documents from the European Medicines Agency, (EMEA), https://www.ema.europa.eu, (February 2020). This corpus contains 762.433 translation units.
EN-NL Bilingual corpus made out of PDF documents from the European Medicines Agency, (EMEA), https://www.ema.europa.eu, (February 2020). This corpus contains 762.433 translation units.


Line 89: Line 105:


==MultiLing EN-NL==
==MultiLing EN-NL==
The multiLing data set is based on six English source texts which are translated into various languages. Four of them (Texts 1-4) are news articles and the other two are (Texts 5-6) sociological texts from an encyclopedia. The Dutch data consists of two parts. ENDU20: Ten Dutch translations of the multiLing set by ten native Dutch recent master's degree translators and ENDU20-MT: Two Dutch machine translations of the multiLing set by DeepL (P20) and Google Translate (P21).
The multiLing data set is based on six English source texts which are translated into various languages. Four of them (Texts 1-4) are news articles and the other two are (Texts 5-6) sociological texts from an encyclopedia. The Dutch data consists of two parts. ENDU20: Ten Dutch translations of the multiLing set by ten native Dutch recent master's degree translators and ENDU20-MT: Two Dutch machine translations of the multiLing set by DeepL (P20) and Google Translate (P21).


Line 95: Line 112:


==Dutch Government Website Corpus==
==Dutch Government Website Corpus==
Parallel (en-nl) corpus of 6.532 translation units.
Parallel (en-nl) corpus of 6.532 translation units.


* [https://live.european-language-grid.eu/catalogue/corpus/2877/ European Language Grid page]
* [https://live.european-language-grid.eu/catalogue/corpus/2877/ European Language Grid page]
</translate>

Revision as of 12:53, 11 June 2024

EDGeS Diachronic Bible Corpus

A diachronically and synchronically parallel corpus of Bible translations in Dutch, English, German and Swedish, with texts from the 14th century until today.

PacoMT Parallel Corpora

During the STEVIN project PaCo-MT (Parse and Corpus-based Machine Translation), two existing parallel corpora were enriched with syntactic annotations and node alignments. The annotations were generated automatically.

Language Pairs: English to Dutch, Dutch to English, French to Dutch, Dutch to French.

The Dutch Parallel Corpus

The Dutch Parallel Corpus (DPC) is a 10-million-word, sentence-aligned parallel corpus for the language pairs Dutch-English and Dutch-French, with Dutch as the central language.

The corpus contains five different text types and is balanced with respect to text type and translation direction. The entire corpus has been aligned at sentence level and further enriched with linguistic information (lemmas and PoS-tags). A small subset of the Dutch-English part has also been manually aligned at the sub-sentential level.

The Open Parallel Corpus (OPUS)

The OPUS corpus contains a very large collection of parallel corpora, amongst which many contain Dutch.

COVID-19 Corpora

COVID-19 - HEALTH Wikipedia dataset. Bilingual (EN-NL)

Bilingual (EN-NL) corpus acquired from Wikipedia on health and COVID-19 domain (2nd May 2020). The corpus contains 931 translation units.

COVID-19 ANTIBIOTIC dataset. Bilingual (EN-NL)

Bilingual (EN-NL) corpus acquired from the website https://antibiotic.ecdc.europa.eu/. The corpus contains 805 translation units.

COVID-19 EC-EUROPA v1 dataset. Bilingual (EN-NL)

Bilingual (EN-NL) corpus acquired from website (https://ec.europa.eu/*coronavirus-response) of the EU portal (20th May 2020). This corpus contains 2.391 translation units.

COVID-19 EU presscorner v2 dataset. Bilingual (EN-NL)

Bilingual (EN-NL) corpus acquired from website (https://ec.europa.eu/commission/presscorner/) of the EU portal (8th July 2020). This corpus contains 6.810 translation units.

COVID-19 EUR-LEX dataset. Βilingual (EN-NL)

Bilingual (EN-NL) corpus acquired from website (https://eur-lex.europa.eu/legal-content) of the EU portal (9th July 2020). This corpus contains 22.470 translation units.

COVID-19 EUROPARL v2 dataset. Bilingual (EN-NL)

Bilingual (EN-NL) corpus acquired from the website (https://www.europarl.europa.eu/) of the European Parliament (9th May 2020). This corpus contains 887 translation units.

COVID-19 Parallel Global Voices dataset. Bilingual (EN-NL)

EN-NL Bilingual COVID-19-related corpus acquired from the website (https://globalvoices.org/) of GlobalVoices (28th April 2020). This corpus contains 675 translation units.

Bilingual corpus from the European Vaccination Portal (NL-EN)

NL-EN Bilingual corpus acquired from https://vaccination-info.eu. This corpus contains 494 translation units.

Bilingual corpus from the Publications Office of the EU on the medical domain v.2 (EN-NL)

EEN-NL Bilingual corpus extracted from the Publications Office of the EU on the medical domain. These are sourced from laws, studies, EC announcements, etc. labelled with concepts like epidemiology, epidemic, disease surveillance, health control, public hygiene, freedom of movement, distance learning, etc. This corpus contains 13.191 translation units.

Bilingual corpus made out of PDF documents from the European Medicines Agency, (EMEA) (EN-NL).

EN-NL Bilingual corpus made out of PDF documents from the European Medicines Agency, (EMEA), https://www.ema.europa.eu, (February 2020). This corpus contains 762.433 translation units.

MultiLing EN-NL

The multiLing data set is based on six English source texts which are translated into various languages. Four of them (Texts 1-4) are news articles and the other two are (Texts 5-6) sociological texts from an encyclopedia. The Dutch data consists of two parts. ENDU20: Ten Dutch translations of the multiLing set by ten native Dutch recent master's degree translators and ENDU20-MT: Two Dutch machine translations of the multiLing set by DeepL (P20) and Google Translate (P21).

Dutch Government Website Corpus

Parallel (en-nl) corpus of 6.532 translation units.

Retrieved from "https://kdutch.ivdnt.org/mediawiki/index.php?title=Parallel_Multilingual_Corpora&oldid=9845"