December 15, 2019 Posted in BI, Blogs, Data Visualization and Analytics, Database, LiveSync Automation

One of my municipality clients has a requirement to handle batch data from multiple data sources for visualization and analytics needs. The data source can be ERP system, sensors or internal databases. Sensors load data into time series database continuously. It requires BI dashboard show charts based on latest data in near real-time manner.

Let’s see how we design the architecture to satisfy the requirements.

First, we have data sources listed in the left, including ERP, time series database-based backend systems, other databases and data flow from APIs ;

Second, we have ETL process to load data from data sources to DVA platform. You may choose any ETL tool you like, but we highly recommend Lionsgate Software’s LiveSync Automation here...

September 30, 2019 Posted in Blogs, BPM, Database, Enterprise Integration

The concept and discipline of data governance has grown in importance as organizations are forced to comply with regulations, cut costs, integrate among different COTS (Commercial-off-the-shelf) systems, provide data interoperability through internal and external APIs.

One of my retail clients has large daily transaction data flow among retail, wholesale inventory, financial, auditing and warehouse departments, etc. It has separate COTS systems such as JDA ESO, eCommerce Shopify and Tecsys SCM, etc. It also has in-house store master, product master data, etc. In some system, SKU is defined as 6-digits number; in another, it’s defined as varchar. In such a complex environment, data management turns to be an outstanding issue.

In order to improve customer experience, increase efficiency, estab...

August 31, 2019 Posted in Blogs, ECM, ESX, IT Infrastructure, Performance, SQL Server

One of my banking clients has a large SQL Server farm which hosts support data. Another client uses SharePoint as ECM (Enterprise Content Management) system which relies on underneath SQL Server farm. Both are dependent on the infrastructure operating 24*7*365, both have strong needs to monitor the performance of SQL Server farm.

Let’s see how to build performance monitoring layers for a SQL Server farm. Though this might not be the best approach, it can be a great reference based on technical implementation feasibility under existing IT environment.

A picture is worth a thousand words. Let’s assume that we utilize NetApp as storage, VMware ESXi (vs. Hyper-V) as bare-metal OS. The guest OS is Windows Servers on which SQL Server instances run. As illustrated, scale-out (vs...