Orientation on Technology

The load manager is responsible for any data transformation required and for the loading of data into the database. These responsibilities can be summarized as data source interaction, data transformation and data load. The depth and complexity of the requirements placed on the load manager by each of these areas vary radically from data warehouse to data warehouse. This means that the load manager can vary from the simplicity of a few simple load scripts to an enormously involved development involving data transformation tools, database to database gateways and complex load programs.

It is not uncommon for the load manager to start out simple and to grow in complexity as data sources and user requirements are added. It is also possible for the development to go the other way, as data sources merge or are replaced. In particular, if the organization has multiple data warehouses, one can often become a greatly simplified source of data for another.

If a copy management tool is being used to transfer the data from the source system, this task becomes the responsibility of the copy management tool. In this case you need to ensure that the tool is carrying out the relevant checks. For example, you need to ensure that the tool verifies that the whole of a day’s data gets transferred, and not just part of it.