Table · dataset · 2022
codeparrot/codeparrot-train-more-filtering
Listed in Hugging Face Datasets
CodeParrot 🦜 Dataset Cleaned and filtered (train) Dataset Description A dataset of Python files from Github.
Description
It is a more filtered version of the train split codeparrot-clean-train of codeparrot-clean. The additional filters aim at detecting configuration and test files, as well as outlier files that are unlikely to help the model learn code.
The first three filters are applied with a probability of 0.7: files with a mention of "test file" or "configuration file" or… See the full description on the dataset page: huggingface.co/datasets/codeparrot/codeparrot-train-more-filtering.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/codeparrot/codeparrot-train-more-filtering ↗
landing page · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/codeparrot/codeparrot-train-more-filtering ↗
metadata API · from Hugging Face
Topics
- From keywords
- Computer Science & AI
Provenance · 1 source records, 9 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | codeparrot/codeparrot-train-more-filtering | 12 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].hf_modality:tabular | source · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].hf_modality:text | source · Hugging Face | connector:huggingface@1.0.0 | |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |