Data Management & Warehousing

Category: Techniques

  • Configuring SSH Keys

    —

    by

    macleanwalker
    in

    Overview Once you have installed PuTTY but before you can connect to the CVS server, you must create your SSH authentication keys. SSH authentication keys are a more secure way of controlling access to the CVS server. Once configured they will allow you to connect to the CVS server without entering a log-on name and…

  • Installing the PuTTY SSH Client for Windows

    —

    by

    macleanwalker
    in

    Overview PuTTY is a GUI Windows client that makes it easy to use the SSH (secure shell) protocol to connect your Windows PC to a Linux server hosting a CVS repository Installing an SSH client and generating your personal SSH keys must be done before you can be given a user account to connect to…

  • CVS – A Basic Introduction

    —

    by

    macleanwalker
    in

    CVS – Getting Started Use this document to get started using CVS. It contains a very short intro to CVS and points you to the more detailed installation and configuration procedures you must follow to use CVS from a Windows or Linux PC. These instructions are written for developers who need CVS access for the…

  • Configuration Management & Issue Tracking

    —

    by

    macleanwalker
    in

    One commonly overlooked side of building a data warehouse is the need for configuration management and issue tracking. This article looks at a way to do this. Since this article was written we have upgraded from Bugzilla to Trac and CVS to SVN and integrated in in our Project Services Introduction ETL tool transformations, data…

  • CSV File Format

    —

    by

    macleanwalker
    in

    We all know that we should be writing file extracts as XML but if we really need to get a CSV file then here is how to do it Introduction Comma Separated Values (CSV) files have been used since data first had to be exchanged between two applications. CSV files are an imperfect format that…

  • Building Data Marts

    —

    by

    macleanwalker
    in

    This article descibes building data marts from a data warehouse or transaction repository. It discusses the merits of full and incremental loads for different data marts.   The types of data marts to load Having built our transaction repository or data warehouse the next job is to build our data marts. The design of data…

  • Oracle DBA Crib Sheet

    —

    by

    macleanwalker
    in

    Introduction The Oracle DBA Crib Sheet is not an exhastive list of what a DBA has to do, instead it has been written for a particular client whose support team occasionally need dba commands and don’t have them written down any where else. It features the commands that we have been asked to put in…

  • Data Transformation – Procedural & Non Procedural Solutions

    —

    by

    macleanwalker
    in

    This paper looks at a somewhat awkward data transformation, and at solutions written in SQL and in a procedural language. It describes some techniques which can be used to develop the solution in both languages. It also compares the solutions in terms of ease of development, performance and cost of maintenance. Are we building transformations…

  • Basic Data Quality Checks

    —

    by

    macleanwalker
    in

    This article looks at basic data quality audit that can be done within a database. Examples are given using Oracle syntax however the techniques can also be applied to other databases   Introduction The following article discusses some of the data quality issues that can be addressed by manual scripts on a copy of the…

  • Detecting Changed Data

    —

    by

    macleanwalker
    in

    Introduction When loading data warehouses, it is usually possible to decreases the load time very significantly by processing only changes since the last load, rather than completely refreshing all the data every time. This article describes one approach for detecting changes, which has been used successfully in a number of data warehouse projects. Background There…