Data · dataset · 2022
Code Clippy
Listed in Hugging Face Datasets
Description
This dataset was generated by selecting GitHub repositories from a large collection of repositories. These repositories were collected from seart-ghs.si.usi.ch/ and Github portion of The Pile (performed on July 7th, 2021). The goal of this dataset is to provide a training set for pretraining large language models on code data for helping software engineering researchers better understand their impacts on software related tasks such as autocompletion of code.
The dataset is split into train, validation, and test splits. There is a version containing duplicates (209GBs compressed) and ones where exact duplicates (132GBs compressed) are removed. Contains mostly JavaScript and Python code, but other programming languages are included as well to various degrees.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/CodedotAI/code_clippy ↗
landing page · from Hugging Face
Documentation and papers
- arXiv:2107.03374 arxiv.org/abs/2107.03374 ↗
publication · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/CodedotAI/code_clippy ↗
metadata API · from Hugging Face
Topics
- Stated by source
- text generation
- From keywords
- Computer Science & AI
Provenance · 1 source records, 9 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | CodedotAI/code_clippy | 11 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[task].hf_task:text-generation | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| license | source · Hugging Face | connector:huggingface@1.0.0 | /tags[license:*] |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |