Data · dataset · 2023
DSSGx 2023 - NRW Bebauungspläne (nrw-bplan-scrape)
Listed in BERD@NFDI Data Portal
This dataset contains all inputs needed as well as outputs of running the full pipeline for creating the NRW land sealing dataset.
Description
This can be reproduced by running this notebook . Dataset Structure nrw bplan features keywords exact_search baunvo_keywords.csv : Results y/n of keywords found in documents relating to baunvo and article 13b. fuzzy_search: keyword_dict_hochwasser.json : Results of keywords found in documents relating to "hochwasser", e.g. hqhäufig and hq100 contains 7 csv files with results of fuzzy key search for keywords.
The file name indicates the key being searched for and the text around this keyword is extracted in a row for each document raw images: images from here can be added to this folder links: NRW_BP.geojson : The file downloaded from the NRV geoportal, containing all raw data on URLs to land parcel bplans. land_parcels.geojson : A processed version of NRW_BP.geojson. NRW_BP_parsed_links.csv : A csv formatted version of NRW_BP.geojson. text: bp_text.json : Raw output of the text text extraction of each pdf.
Read the rest (2 more)
Contains only columns for the filename and the extracted text. document_texts.json : Enriched version of bp_texts.json in which columns about the documents have been appended. pdfs: pdfs extarcted from the NRW Geoportal and are found here , can be added to this folder knowledge_extraction_agent: Contains 6 json files. The filename corresponds to the key looked for in the fuzzy keyword search (e.g. fh.json cooresponds to firsthöhe.csv , gfz.json corrresponds to geschossflächenzahl.csv ).
More info can be found here knowledge_agent_output.json : Is a toy example for 10 files of the output of the pipeline for the knowledge agent (merging of results in nrw/bplan/knowledge_extraction_agent ) clean document_texts.xlsx : See here for more information exact_keyword.xlsx : This corresponds to baunvo_keywords.csv. fuzzy_keyword.xlsx : Is the merged version of the files found in nrw/bplan/fuzzy_search knowledge_agent.xlsx : The .xlsx version of nrw/bplan/knowledge_agent_output.json ) land_parcels.xlsx : See here for more information regional_plans.xlsx : The .xlsx version of the data table found here rplan features: contains regional_plan_sections.json , the output of the pipeline - a more detailed can be found here raw geo: contains regions_map.geojson with information on the geolocations of the regional plans pdfs: contains pdfs of regional plans for NRW - used as input to run the pipeline text: contains text extracted with Tika from all pdf regional plans Contact Homepage: DSSGx Munich organization page.
Links
Where it is published
- BERD@NFDI Data Portal record berd-platform.de/records/z5yeh-wqy70 ↗
landing page · from berd platform de
Catalogue records · 1
- InvenioRDM API berd-platform.de/api/records/z5yeh-wqy70 ↗
metadata API · from berd platform de
Topics
- From keywords
- Economics & Finance
Provenance · 1 source records, 9 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| BERD@NFDI Data Portal | z5yeh-wqy70 | 9 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · berd platform de | connector:berd_platform_de@1.0.0 | |
| concepts[field].local:field:economics-finance | mapping · berd platform de | connector:berd_platform_de@1.0.0 | |
| concepts[modality].local:modality:image | enrichment · berd platform de | keyword-concept-rules@1.0.0 | title+description (65%) |
| concepts[modality].local:modality:tabular | enrichment · berd platform de | keyword-concept-rules@1.0.0 | title+description (65%) |
| concepts[modality].local:modality:text | enrichment · berd platform de | keyword-concept-rules@1.0.0 | title+description (75%) |
| description | source · berd platform de | connector:berd_platform_de@1.0.0 | /metadata/description |
| publication_date | source · berd platform de | connector:berd_platform_de@1.0.0 | |
| title | source · berd platform de | connector:berd_platform_de@1.0.0 | /metadata/title |
| updated_date | source · berd platform de | connector:berd_platform_de@1.0.0 |