← Search

Data · dataset · 2026

Evolving Scientific Software in Long-Running Observatories: Lessons from the TERENO Sensor Management Migration

Listed in Jülich DATA

Scientific software in the geosciences often evolves organically, growing from small, purpose-built tools into long-lived infrastructure.

Description

Using the example of the TERENO observatories and their integration into the Earth & Environment DataHub, this contribution reflects on the challenges and opportunities of transforming legacy systems into sustainable, community-ready platforms. A central task was the migration of sensor metadata from a tightly coupled legacy system to the Helmholtz Sensor Management System, requiring a clear separation of data and metadata, adaptation to new API-based technologies, and reconciliation of differing data models and vocabularies.

The scale (thousands of devices and tens of thousands of parameters) and the presence of heterogeneous, project-specific edge cases further complicated the process. To address these challenges, we developed the Python-based ODM2SMS tool, enabling configurable, incremental, and reversible migration workflows. Automated processing was complemented by validation routines, staged testing environments, and targeted manual inspection to ensure scientific plausibility.

Read the rest (1 more)

The approach emphasized modularity, explicit documentation, and the enrichment of source data to improve migration quality. From this experience, we derive practical lessons for sustainable scientific software development: prioritize conceptual design over rapid prototyping, involve users early, and avoid tightly coupled database designs. More broadly, the work highlights how incremental refactoring, open tools, and community engagement can turn organically grown systems into robust, reusable infrastructure for future research.

Links

Topics

Provenance · 1 source records, 10 field assertions
SourceKeyLast seenRaw
Jülich DATAdoi:10.26165/JUELICH-DATA/GRXBJX19 h agoJSON v1
FieldAssertionExtractorEvidence
concepts[field].dataverse_subject:earth-and-environmental-sciencessource · data fz juelich deconnector:data_fz_juelich_de@1.0.0/subjects
concepts[field].local:field:earth-environmentalmapping · data fz juelich deconnector:data_fz_juelich_de@1.0.0/subjects
concepts[field].local:field:engineeringmapping · data fz juelich deconnector:data_fz_juelich_de@1.0.0/subjects
concepts[field].local:field:life-sciencesmapping · data fz juelich deconnector:data_fz_juelich_de@1.0.0/subjects
created_datesource · data fz juelich deconnector:data_fz_juelich_de@1.0.0
descriptionsource · data fz juelich deconnector:data_fz_juelich_de@1.0.0/description
publication_datesource · data fz juelich deconnector:data_fz_juelich_de@1.0.0
titlesource · data fz juelich deconnector:data_fz_juelich_de@1.0.0/name
updated_datesource · data fz juelich deconnector:data_fz_juelich_de@1.0.0
version_labelsource · data fz juelich deconnector:data_fz_juelich_de@1.0.0