Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

For more information on how to use the Data Lake, see Data Lake Best Practices.

How It Works

The Data Lake provides table extracts from a client's operational database in the Veson IMOS Platform, transforms those extracts into Report Designer reporting format, and places the transformed extracts into a secure, encrypted download location. The secure location is accessible only to the client and is protected by an industry-standard authentication mechanism. To ensure optimal download performance, the data is stored in the same geographic region as the client: North America, Europe, or Asia-Pacific.

...

The Data Lake is managed by Veson Nautical and is monitored in real-time.

Frequently Asked Questions

Expand
titleHow is the Data Lake different than replicating the VIP operational database to on-premise?

On-premise database replication facilitates direct queries and reports run against the replicated (on-premise) database, to feed BI and data warehouse solutions, or other in-house systems. The replication processes can be labor intensive and unreliable for large data sets, and the client must continue to modify their reporting and data transformation processes when the source VIP database schema changes.

Data Lake solves this issue in two ways. First, Data Lake transforms the Veson IMOS Platform database extracts into Report Designer format prior to making the extracts available to clients, which makes them immune to schema changes. Second, the extracts are sent to the client on a daily or hourly basis, making database replication unnecessary.

Replication is also used traditionally for business continuity, for times when one data center is not available, clients can continue to access data in a replica. Amazon Web Services (AWS) provides redundancy that removes the likelihood of a single point of failure in any part of the Veson IMOS Platform stack (storage, compute, networking), so traditional on-premise replication for business continuity and disaster recovery is not necessary.

...

Expand
titleDoes the Data Lake extract represent a complete copy of the VIP Data Dictionary?

No. The Data Dictionary is another name for our Report Designer schema (also known as the Data Map); it is the complete schema for BI reporting. The Data Lake uses a subset of this schema. Data Lake and Data Dictionary are different by design, since the Report Designer has predefined joins that are not relevant to a BI reporting scenario where you join the data yourself, outside of the system.