Data Management & Warehousing

Category: Techniques

  • Job scheduling – fixed and relative timing

    —

    by

    macleanwalker
    in

    This article looks at the relatve merits of two types of processing schedule, once based on a fixed times (similar to ‘cron’ on a unix system) and onebased on relative times (similar to ‘at’ on a unix system) Introduction Much of data warehousing is dependent on running regular jobs to get collect data and load…

  • Auditing Data Cleaning Updates

    —

    by

    macleanwalker
    in

    How to track what has been updated by data cleaning processes The Problem A common problem when building a data warehouse is to track and audit which records have had data quality updates or cleaning applied to them. A simple method is to add a bitmap that reflects the data quality updates have been applied.…