Data · dataset · 2022
Books3
Listed in Hugging Face Datasets
This dataset is Shawn Presser's work and is part of EleutherAi/The Pile dataset.
Description
This dataset contains all of bibliotik in plain .txt form, aka 197,000 books processed in exactly the same way as did for bookcorpusopen (a.k.a. books1). seems to be similar to OpenAI's mysterious "books2" dataset referenced in their papers. Unfortunately OpenAI will not give details, so we know very little about any differences.
People suspect it's "all of libgen", but it's purely conjecture.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/defunct-datasets/the_pile_books3 ↗
landing page · from Hugging Face
Documentation and papers
- arXiv:2101.00027 arxiv.org/abs/2101.00027 ↗
publication · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/defunct-datasets/the_pile_books3 ↗
metadata API · from Hugging Face
Topics
- Stated by source
- fill mask · text generation
- From keywords
- Computer Science & AI
Provenance · 1 source records, 10 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | defunct-datasets/the_pile_books3 | 11 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[task].hf_task:fill-mask | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| concepts[task].hf_task:text-generation | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| license | source · Hugging Face | connector:huggingface@1.0.0 | /tags[license:*] |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |