Data · dataset · 2026
Web Archives Data (WARCs) for Retrieval Augmented Generation in Ollama using WARC-GPT
Listed in Borealis
Description
This dataset consists of two (2) WARC files from Archive-It consisting of full site crawls of the The Code4Lib Journal (journal.code4lib.org/) and the International Journal of Digital Curation (ijdc.net/index.php/ijdc). These WARC file are being used for the creation of a retrieval augmented generation (RAG) pipeline as part of a local implementation at UVic Libraries of WARC-GPT from the Harvard Library Innovation Lab (github.com/harvard-lil/warc-gpt).
Links
Where it is published
- Dataverse dataset page borealisdata.ca/dataset.xhtml?persistentId=doi%3A10.5683%2FSP3%2FRZOYV8 ↗
landing page · from borealisdata ca
- DOI doi.org/10.5683/sp3/rzoyv8 ↗
DOI / persistent id · from borealisdata ca
Catalogue records · 1
- Dataverse API borealisdata.ca/api/datasets/:persistentId/?persistentId=doi%3A10.5683%2FSP3%2… ↗
metadata API · from borealisdata ca
Topics
- Stated by source
- Other
- From keywords
- Artificial intelligence · Computer Science & AI
Provenance · 1 source records, 9 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Borealis | doi:10.5683/SP3/RZOYV8 | 10 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| concepts[field].anzsrc:group:4602 | mapping · borealisdata ca | vocabulary-mapper@1.0.0 | keywords['Artificial intelligence'] |
| concepts[field].dataverse_subject:other | source · borealisdata ca | connector:borealisdata_ca@1.0.0 | /subjects |
| concepts[field].local:field:computer-science-ai | mapping · borealisdata ca | connector:borealisdata_ca@1.0.0 | /subjects |
| created_date | source · borealisdata ca | connector:borealisdata_ca@1.0.0 | |
| description | source · borealisdata ca | connector:borealisdata_ca@1.0.0 | /description |
| publication_date | source · borealisdata ca | connector:borealisdata_ca@1.0.0 | |
| title | source · borealisdata ca | connector:borealisdata_ca@1.0.0 | /name |
| updated_date | source · borealisdata ca | connector:borealisdata_ca@1.0.0 | |
| version_label | source · borealisdata ca | connector:borealisdata_ca@1.0.0 |