January 31, 2017 Posted in Blogs, LiveSync Automation

Often we need track changes for the entire database, so we need design audit system to track changes. There are many best practices, such as separating the auditing table in another database, keep it small and fast, demoralize, etc. But at the end, we are facing how to design the auditing system.

There are a few approaches to design the auditing system:

• Build an auditing table for each base table;
• Build one generic auditing table to track changes of all database tables;
• Write changes to XML files which are stored outside of the database.

Triggers residing on database tables may capture changes and write to auditing table(s) or XML files. There are cons and pros for each one of them. None of them is perfect...

December 29, 2016 Posted in Blogs

For many developers or small teams, it’s ideal to have a free cloud-based source control system for distributed development work. The benefits are obvious, most people enjoy working at home or creating something that may really excite themselves in after hours. This article introduces how to set up such an environment for your development work. Hope you enjoy it!

GitHub is hot but not free for private projects; Heroku is not free for any project with more than 1 contributor.

In this case, GitLab is a lot better. It offers free unlimited repositories for both public and private projects, 10GB disk space per project, unlimited collaborators and complete project management solutions. This is why GitLab.com is our chosen cloud repository platform.

Here’s the steps:

1. Sign up to GitLab...

November 30, 2016 Posted in Blogs

Recently we achieved a project regarding large data replication from Oracle database to PostgreSQL. We are facing the challenge that we have very limited window to complete the large volume data loading on daily basis.

Initially we tried PostgreSQL loading tool psql, it took 4.5 hours to complete transferring approx. 250,000 records. But regular business operations require a lot shorter time to transfer more data, e.g. complete 1 million records replication in 30 minutes. Based on the performance of psql (which is very likely single thread basis), to complete 1 million records replication needs 18 hours (1,080 minutes)! – which is not feasible from business perspectives.

We built up an application based on Quartz Scheduler which does table to table asynchronous replication in nearly rea...