Data · dataset · 2025
Web 1T 5-gram Version 1
Listed in UC Berkeley Library Dataverse
Web 1T 5-gram Version 1 was contributed by Google Inc. and contains English word n-grams and their observed frequency counts for approximately 1 trillion tokens.
Description
The length of the n-grams ranges from unigrams (single words) to five-grams. This data is expected to be useful for statistical language modeling, e.g., for machine translation or speech recognition, as well as for other uses.
Links
Where it is published
- Dataverse dataset page datasets.lib.berkeley.edu/dataset.xhtml?persistentId=doi%3A10.60503%2FD3%2FK7DLOU ↗
landing page · from datasets lib berkeley edu
- DOI doi.org/10.60503/d3/k7dlou ↗
DOI / persistent id · from datasets lib berkeley edu
Catalogue records · 1
- Dataverse API datasets.lib.berkeley.edu/api/datasets/:persistentId/?persistentId=doi%3A10.60503%2FD3%2… ↗
metadata API · from datasets lib berkeley edu
Topics
- Stated by source
- Social Sciences
- From keywords
- Social Science
- Inferred from text
- Audio 65%
Provenance · 1 source records, 9 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| UC Berkeley Library Dataverse | doi:10.60503/D3/K7DLOU | 10 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| concepts[field].dataverse_subject:social-sciences | source · datasets lib berkeley edu | connector:datasets_lib_berkeley_edu@1.0.0 | /subjects |
| concepts[field].local:field:social-science | mapping · datasets lib berkeley edu | connector:datasets_lib_berkeley_edu@1.0.0 | /subjects |
| concepts[modality].local:modality:audio | enrichment · datasets lib berkeley edu | keyword-concept-rules@1.0.0 | title+description (65%) |
| created_date | source · datasets lib berkeley edu | connector:datasets_lib_berkeley_edu@1.0.0 | |
| description | source · datasets lib berkeley edu | connector:datasets_lib_berkeley_edu@1.0.0 | /description |
| publication_date | source · datasets lib berkeley edu | connector:datasets_lib_berkeley_edu@1.0.0 | |
| title | source · datasets lib berkeley edu | connector:datasets_lib_berkeley_edu@1.0.0 | /name |
| updated_date | source · datasets lib berkeley edu | connector:datasets_lib_berkeley_edu@1.0.0 | |
| version_label | source · datasets lib berkeley edu | connector:datasets_lib_berkeley_edu@1.0.0 |