Datastores¶

Unlike many ground data systems for space missions, Hermes does not ship with a builtin data persistence solution. Data storage is not a goal of the Hermes project. For this purpose, Hermes relies on an external provider to store its time series data. There are a number of databases that Hermes supports, each with their own use cases.

Before we discuss the Hermes supported datastores, it is important to discuss the data model of Hermes. Hermes uses a generic data representation which can be converted to a chosen database representation. It is also important to note that Hermes does not persist any data in the backend. A plugin can be either built directly into the backend or attached via the gRPC API.

The Hermes downlink data model is described in our core concepts document.

Below is a table of the supported databases that can be used to persist data streaming through Hermes.

Database Overview	Pros/Cons	Use Cases
Open Telemetry Covers all OTEL databases including the recommended Grafana Loki-Grafana-Tempo-Mimir (LGTM) stack.	Pros Free and Open-Source with Enterprise Support Standard data schema Out-of-the-box applications built into Grafana for querying Easy configuration and deployment Cons Limits telemetry to numeric (or complex structures/arrays of numeric) telemetry Does not support 'raw' sample querying Reprocesses telemetry >1Hz into 1Hz data Does not support multiple time columns (i.e. ERT only, no SCLK) Cannot model complex dynamic data structures such as those in data products or image files	Demo, prototype, early development. Supplimentary to another database for events and command + sequence tracing.
Clickhouse High performance database built for online analytical processing (OLAP)	Pros Free and Open-Source with Enterprise Support High performance and scalable Used for large scale analytical applications SQL queries Cons Complex configuration and management Not a good fit for relational data (not typically used by Hermes but may be used on other parts of your mission) No out of the box support in Grafana that conforms to Hermes data model (i.e. raw queries required for now)	V&V, ALTO, Operations
SQLite Single file based embedded database for zero overhead data storage.	Pros No configuration, very simple Used for large scale analytical applications Cons Not meant for server-like deployment (i.e. single process access) Not meant for large scale datasets	FSW Dev, CI, FIT Testing
TimescaleDB Time series extension to PostgreSQL. Supports materialized view for online preprocessing of common queries. Works with Postgres tooling.	Pros High performance and scalable Used for large scale analytical applications Supports PostgreSQL infrastructure. Great for teams already using postgres. Cons Partially Open-Source focusing on Enterprise Support 2.x uses custom query language called flux, 3.x switches to SQL. Complex configuration and management No out of the box support in Grafana that conforms to Hermes data model (i.e. raw queries required for now) Business model rapidly changes with new releases	V&V, ALTO, Operations
InfluxDB High performance open-schema time series nosql column-based database	Pros High performance and scalable Used for large scale analytical applications Low upfront effort for configuration and deployment Cons Partially Open-Source focusing on Enterprise Support Does not support multiple time columns ERT or SCLK, not both Limited support for large binary blob fields, images may need separate store. No out of the box support in Grafana that conforms to Hermes data model (i.e. raw queries required for now) Business model rapidly changes with new releases	V&V, ALTO, Operations