Prometheus has its own local storage using a local on-disk time series database. However, this is not clustered or replicated. i.e. it’s not scalable or durable.

https://prometheus.io/docs/prometheus/latest/storage/

It does provide interfaces to integrate with remote storage. E.g.

https://prometheus.io/docs/prometheus/latest/storage/#remote-storage-integrations

and

https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage

 

One of these options is PostgreSQL and TimescaleDB. Note, TimescaleDB uses PostgreSQL but scales it for better performance using automatic partitioning across time and space:

https://github.com/timescale/prometheus-postgresql-adapter

and https://github.com/timescale/timescaledb

 

Prometheus remote storage adapter for PostgreSQL

1. Install packages (both provided by Timescale):

• remote storage adapter

The adapter is a translation proxy used by Prometheus for reading/writing data to the PostgreSQL/Timescale database. The data from Prometheus arrives as a Protobuf. The adapter deserializes it and converts it into the Prometheus native format (see Prometheus’ Exposition Formats) before inserting it into the database.

A Docker image provides the Prometheus PostgreSQL remote storage adapter: https://hub.docker.com/r/timescale/prometheus-postgresql-adapter/

•  pg_prometheus

pg_prometheusimplements the Prometheus data model for PostgreSQL.

A Docker image which provides PostgreSQL and TimescaleDB: https://hub.docker.com/r/timescale/pg_prometheus/

https://blog.timescale.com/sql-nosql-data-storage-for-prometheus-devops-monitoring-postgresql-timescaledb-time-series-3cde27fd1e07

2. Configure Prometheus to use this remote storage adapter

i.e. add this to prometheus.yml

 

See also this tutorial: https://docs.timescale.com/v0.10/tutorials/prometheus-adapter