Data · dataset · 2022
hansards
Listed in Hugging Face Datasets
This release contains 1.3 million pairs of aligned text chunks (sentences or smaller fragments) from the official records (Hansards) of the 36th Canadian Parliament.
Description
The complete Hansards of the debates in the House and Senate of the 36th Canadian Parliament, as far as available, were aligned. The corpus was then split into 5 sets of sentence pairs: training (80% of the sentence pairs), two sets of sentence pairs for testing (5% each), and two sets of sentence pairs for final evaluation (5% each).
The current release consists of the training and testing sets. The evaluation sets are reserved for future MT evaluation purposes and currently not available. Caveats 1.
Read the rest (3 more)
This release contains only sentence pairs. Even though the order of the sentences is the same as in the original, there may be gaps resulting from many-to-one, many-to-many, or one-to-many alignments that were filtered out. Therefore, this release may not be suitable for discourse-related research. 2.
Neither the sentence splitting nor the alignments are perfect. In particular, watch out for pairs that differ considerably in length. You may want to filter these out before you do any statistical training.
The alignment of the Hansards was performed as part of the ReWrite project under funding from the DARPA TIDES program.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/usc-isi/hansards ↗
landing page · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/usc-isi/hansards ↗
metadata API · from Hugging Face
Topics
- From keywords
- Computer Science & AI
- Inferred from text
- Text 75%
Provenance · 1 source records, 8 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | usc-isi/hansards | 12 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].local:modality:text | enrichment · Hugging Face | keyword-concept-rules@1.0.0 | title+description (75%) |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |