Data · dataset · 2026
Evolving Scientific Software in Long-Running Observatories: Lessons from the TERENO Sensor Management Migration
Listed in Jülich DATA
Scientific software in the geosciences often evolves organically, growing from small, purpose-built tools into long-lived infrastructure.
Description
Using the example of the TERENO observatories and their integration into the Earth & Environment DataHub, this contribution reflects on the challenges and opportunities of transforming legacy systems into sustainable, community-ready platforms. A central task was the migration of sensor metadata from a tightly coupled legacy system to the Helmholtz Sensor Management System, requiring a clear separation of data and metadata, adaptation to new API-based technologies, and reconciliation of differing data models and vocabularies.
The scale (thousands of devices and tens of thousands of parameters) and the presence of heterogeneous, project-specific edge cases further complicated the process. To address these challenges, we developed the Python-based ODM2SMS tool, enabling configurable, incremental, and reversible migration workflows. Automated processing was complemented by validation routines, staged testing environments, and targeted manual inspection to ensure scientific plausibility.
Read the rest (1 more)
The approach emphasized modularity, explicit documentation, and the enrichment of source data to improve migration quality. From this experience, we derive practical lessons for sustainable scientific software development: prioritize conceptual design over rapid prototyping, involve users early, and avoid tightly coupled database designs. More broadly, the work highlights how incremental refactoring, open tools, and community engagement can turn organically grown systems into robust, reusable infrastructure for future research.
Links
Where it is published
- Dataverse dataset page data.fz-juelich.de/dataset.xhtml?persistentId=doi%3A10.26165%2FJUELICH-DATA%2FGRX… ↗
landing page · from data fz juelich de
- DOI doi.org/10.26165/juelich-data/grxbjx ↗
DOI / persistent id · from data fz juelich de
Catalogue records · 1
- Dataverse API data.fz-juelich.de/api/datasets/:persistentId/?persistentId=doi%3A10.26165%2FJUEL… ↗
metadata API · from data fz juelich de
Topics
- Stated by source
- Earth and Environmental Sciences
- From keywords
- Earth & Environmental Science · Engineering · Life Sciences
Provenance · 1 source records, 10 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Jülich DATA | doi:10.26165/JUELICH-DATA/GRXBJX | 19 h ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| concepts[field].dataverse_subject:earth-and-environmental-sciences | source · data fz juelich de | connector:data_fz_juelich_de@1.0.0 | /subjects |
| concepts[field].local:field:earth-environmental | mapping · data fz juelich de | connector:data_fz_juelich_de@1.0.0 | /subjects |
| concepts[field].local:field:engineering | mapping · data fz juelich de | connector:data_fz_juelich_de@1.0.0 | /subjects |
| concepts[field].local:field:life-sciences | mapping · data fz juelich de | connector:data_fz_juelich_de@1.0.0 | /subjects |
| created_date | source · data fz juelich de | connector:data_fz_juelich_de@1.0.0 | |
| description | source · data fz juelich de | connector:data_fz_juelich_de@1.0.0 | /description |
| publication_date | source · data fz juelich de | connector:data_fz_juelich_de@1.0.0 | |
| title | source · data fz juelich de | connector:data_fz_juelich_de@1.0.0 | /name |
| updated_date | source · data fz juelich de | connector:data_fz_juelich_de@1.0.0 | |
| version_label | source · data fz juelich de | connector:data_fz_juelich_de@1.0.0 |