One of my municipality clients has a requirement to handle batch data from multiple data sources for visualization and analytics needs. The data source can be ERP system, sensors or internal databases. Sensors load data into time series database continuously. It requires BI dashboard show charts based on latest data in near real-time manner.
Let’s see how we design the architecture to satisfy the requirements.
First, we have data sources listed in the left, including ERP, time series database-based backend systems, other databases and data flow from APIs ;
Second, we have ETL process to load data from data sources to DVA platform. You may choose any ETL tool you like, but we highly recommend Lionsgate Software’s LiveSync Automation here...
Read More