Data Warehouse Support and Enhancement
The Building of Stored Data Is an Iterative Process.
Apr 3, 2009
Just as the building of the data warehouse is an iterative process, the integration of new applications and tools into the production environment is also an on-going process. This phase establishes the blueprint for ongoing support and operation of the data warehouse. It may be conducted by an outside consultant, or an internal support team, that may be independent of the data warehousing project team, and has the following major components :
- Solution Integration – final testing and installation of tools and applications warehouse
- Data Warehouse Management – administration of users, space, backup/recovery, and usage reporting
- Enterprise System Support – coordination with other company operations
- Data Warehouse Capacity Planning – growth projections in system utilization and budgeting for upgrades
- Logical Database Review - periodic review to validate the physical data model
- Physical Database Review – periodic review to validate the physical data model
- Modifying Performance – resolves performance issues with specific applications or queries
- System Audit – identify business value and recommend changes
Solution Integration
Where there is an existing data warehouse, this activity will integrate the new application components and tools with the existing system, and will involve the following eight functions:
- Dependencies on the processing of other systems - establishing correct scheduling information in the system’s automatic job scheduler
- Duration of table maintenance activities of the new system - processing may need to contend with other applications in the “overnight” batch windows and if 24-hour access is required, an alternate table may be built and brought on-line
- Capacity impact – monitoring and scheduling new jobs with critical storage and processing requirements
- Client Software Installation – testing to ensure that installing and removing software is properly managed, and that network setup enables users to access the data warehouse
- Establishment of help desk procedures – to provide on-going user support
- Final documentation revisions – ensuring that user and systems documentation are current and accurately define processes
- Host Software Installation – moving jobs and executable programs from development areas to production areas
- Identification of change procedures -- verification of system procedures for maintaining the system
This Solution Integration process should be executed jointly by the development team and data warehouse support personnel.
Enterprise System Support
This support activity involves integrating the hardware operations into the IT department operation, and will include procedures for startup and shutdown of data warehousing hardware, maintenance of support accounts, and identification of hardware problems such as the failure of a disk drive, a power supply, or other hardware component. If goals are not being met, steps to correct the problem need to be documented. A central organization should monitor the space allocated and used by the different databases in conjunction with capacity planning activities.
Depending on the database product selected for the data warehouse, striping of physical disks may be required, as well as establishment of table spaces across the stripes, and allocation of permanent, temporary, log and index areas across different table spaces.
When space requirements change, the database may need to be recreated and reloaded. During normal operations, the table and indexes may need to be rebuilt to get the entries back into logical = physical sequence. Depending on the size of the tables and the speed of the system, this could mean coordinating a process that makes the system unavailable for hours or days, and requires careful scheduling to avoid impacting users.
 |
