Data which is universal across the network, such as dimension or reference data, is managed centrally via operations which permit rows to be inserted, updated, deleted or loaded from files in a variety of record formats. These operations are performed as ACID transactions allowing changes to multiple rows to happen atomically and durably.
Constraints can be created to enforce primary key, unique and foreign key integrity rules on universal data.
Universal data inevitably changes with time, hence the Edge Intelligence platform provides effective dating of universal data, so that changes can be applied from a given point in time and the history of those changes is retained indefinitely - so that queries can apply temporal conditions.
The use of temporal data is optional, such that changes can be applied across all time, where required; and corrections to invalid historic data can also be applied, if required.
Message data is fact or event data and is treated as immutable, so that message data cannot be updated or deleted. This guarantees that message data maintains evidential quality and accurately reflects the data as it was at time of receipt.
Data acquired from messages arriving at network speed can become extremely voluminous when collected and retained for several weeks, months or years. Edge Intelligence allows message data which falls beyond a useful retention window to be retired to reclaim storage space.
Both structured and semi-structured data rich in textual content can be stored with fast text search available on all textual content, without the need to create or manage any text search indexing.