Table · dataset · 2024
Taiwan Text Excellence (TTE)
Listed in Hugging Face Datasets
High quality corpus for Taiwanese culture and Traditional Chinese Taiwan Text Excellence (TTE) Contains high quality news and articles in Traditional Chinese.
Description
The data processing pipeline is optimized for LLM performance. Is de-duplicated and cleaned using both rule-based and learning-based filters.
E.g., urls/emails/html tags/abnormal characters are cleaned, and numbers (full-width or half-width) are normalized. Contains ~2 billion tokens, measured using BPE tokenizer… See the full description on the dataset page: huggingface.co/datasets/liswei/Taiwan-Text-Excellence-2B.
Links
Get the data
- Hugging Face dataset page huggingface.co/datasets/liswei/Taiwan-Text-Excellence-2B ↗
Gated (auto) - request access on the dataset page
landing page · request access · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/liswei/Taiwan-Text-Excellence-2B ↗
metadata API · from Hugging Face
Topics
- Stated by source
- text · text generation
- From keywords
- Computer Science & AI
- Inferred from text
- Text 75%
Provenance · 1 source records, 11 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | liswei/Taiwan-Text-Excellence-2B | 13 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].hf_modality:text | source · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].local:modality:text | enrichment · Hugging Face | keyword-concept-rules@1.0.0 | title+description (75%) |
| concepts[task].hf_task:text-generation | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| license | source · Hugging Face | connector:huggingface@1.0.0 | /tags[license:*] |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |