Journals allow the results of query statements to be materialised within fixed time intervals for the purposes of pre-aggregation and anomaly detection. For example, a journaled query could aggregate data in hourly or daily time intervals; or could join message data with a list of thresholds to generate performance anomaly indicators for fixed time intervals.
Materialization happens in real-time as and when new data arrives and the results of a journal are retained separately from the source data it was generated from, so that the source data and journal data life-cycle can be managed independently.
Journal results are evaluated at the edge of the network but are defined, managed and queried centrally like any other table or view.
Late Arriving Data
Journals are able to cope with late arriving data, so that any historic intervals affected by late arriving data are updated accordingly.
Journaling can also be used to fold repetitive data into distinct values and occurrence counts in fixed-time intervals, to make such data more amenable to data analysis.