Archive for June, 2010
Over the last few weeks have been working a lot with computer automation. The challenge with automation is to keep it running even when exceptions are happening.
The project I have been working on moves data on a regular interval between MySQL database and a SQL Server database. The MySQL database lives on a linux machine and is accessed via a SSH tunnel.
The following three rules will ensure that the automation runs quite reliable.
- Be able to restart the process at any point.
- Be able to process the same data without causing issues E.g. duplication
- Be able to chunk the data. E.g. if the process has not run in a long time only process a limited number of records at a time.