Data Management & Warehousing
  • Welcome
    • Contact Us
    • QR Codes
    • Privacy & Cookie Policy
    • Internal
    • Websites
  • White Papers
  • Presentations
  • Blog
  • Our Services
    • Rates
Home » Posts tagged "Netezza"

Tag Archives: Netezza

BI Vendors & Thought Leaders on Twitter

Posted on 17 July 2012 by David M. Walker Posted in Social Media Leave a comment

Our aggregated twitter feed http://datamgmt.com/twitter has all the latest tweets from thought leaders, vendors and some leading BI organisations. You can also subscribe to the list directly on twitter

Don’t forget whilst you are here to have a look at our White Papers, Presentations and Blog

The current list includes (links to their individual twitter accounts):

Thought Leaders

  • Bill Inmon
  • Claudia Imhoff
  • Curt Monash
  • David M Walker
  • Ralph Kimball
  • Tim O’Reilly

Technology Vendors

  • Greenplum
  • IBM Big Data
  • IBM Netezza 
  • Infobright
  • Informatica
  • Jaspersoft
  • Kalido
  • Kognitio
  • Microsoft BI
  • MicroStrategy
  • NeutrinoBI
  • Oracle BI
  • Pentaho
  • PushBI
  • QlikView
  • Roambi
  • SAP BusinessObjects
  • Tableau Software
  • Teradata

Other Organisations

  • BeyeNETWORK
  • ETIS
  • TDWI

If you think someone else should be on here post a comment below and if they meet the general criteria of being a significant thought leader or a vendor with a credible Business Intelligence or Data Warehousing presence then we will add them.

ETIS InfoBright Netezza Teradata Twitter Vectorwise

Implementing Netezza Spatial

Posted on 1 July 2010 by David M. Walker Posted in Presentations Leave a comment

An internal training presentation on implementing Netezza Spatial functionality

What is Spatial Technology?

  • It’s the ability to analyse information in a geographic context:
    • Where is the nearest petrol station?
    • Which road am I on?
    • How many ATMs are in this area?
  • It’s not maps and images
    • These come later with tools that help present the information

This presentation explains what spatial is, how it works and the basics of implementing a solution

Download Implementing Netezza Spatial now

Netezza Programming Spatial Telematics

At last someone we understand …

Posted on 7 November 2009 by David M. Walker Posted in Editorial Leave a comment

It is rare with all the background reading and research that I have to do that I read a contemporary book and say “Yes, this guy gets it” – notice that I say contemporary. There are plenty of good books about the day-to-day business intelligence problems that I see which were written over 20 years ago (The Mythical Man Month by Frederick P Brooks [1975] and Peopleware: Productive Projects & Teams by Tom DeMarco and Timothy Lister [1987] to name but two). So it was with immense pleasure that I read the opening chapters of Netezza Underground: The Unauthorized Tales of Derring-do and Adventures in Resilient Data Warehousing Solutions by David Birmingham[2008]. This book describes, in a hugely readable way, many of the same concepts and ideas that I have been espousing for the last 15 years about how to deliver a successsful data warehouse solution.

David describes the building of data warehouses as the need to think of terabytes, not transactional systems and a discussion of very-large-scale data systems. He says that the rules are different here and yet oddly the same:

  1. Everything is requirements driven
  2. Simplify and clarify
  3. Use correctly powered and scalable systems
  4. Governance
  5. Data management
  6. Strong architectural approach
  7. Building the environment with the expectation of change
  8. Testability
  9. Go Parallel
  10. Never do bulk inside a traditional RDBMS

I would love to drill into each of these in more detail but doing so would simply reprise an excellent book and probably breach copyright! And if the mention of Netezza as a vendor puts you off, then read the book anyway, because whilst the author is an avid enthusiast of the product (and some reviewers dislike the consequentially irreverent style), the approach, techniques and philosophy he describes provide much of the same advice that we have been giving our customers for so long and that can be applied to other technical solutions. Of course if you are interested in implementing a system we are here to help too!

This article was originally published on BIonRails, another Data Management & Warehousing website

Agile Data Warehousing ETL Mythical Man Month Netezza PeopleWare

Data Warehouse Appliances – Fad or Future?

Posted on 3 November 2009 by David M. Walker Posted in Editorial Leave a comment

This article was originally written for Conspectus Magazine in December 2006 and has been updated in November 2009 by the original author

Despite all the hype from vendors the basics of data warehousing have remained fundamentally unchanged – extract data from multiple source systems, reformat the information into an easy to query structure, load it into a dedicated database and add an effective user interface to allow users to query the information. The cost of this environment is substantial and directly relates to the complexity of the Extract, Transform and Load (ETL) process and the volume of data held in the system.

The complexity of the ETL process has two cost impacts: the first is in the cost of the initial design and development and is reasonably well understood. The second is the cost of changes over the lifetime of the system, for example if an organisation have four source systems and each system under goes a change once a quarter then the data warehouse support team have to modify and test an interface every three weeks, and all this without any changes in the users requirements. The volume of data also hits the bottom line, not only in the cost of storage but in the size and (more expensive) skills of team required to support it, especially as data explosion forces the business to enter the very large database arena where load time and user query performance are critical.

Against this background it is unsurprising that vendors are looking to compete by reducing storage, improving query times and simplify administration. Oracle have taken steps to enhance their core database engine with features such as Exadata that improve each of these areas and continue to develop their strategy, however more and more is built into the core of its flagship general purpose engine resulting in software that has many features not needed by a specific application. Sybase have taken the more radical step of creating an entirely new database engine called Sybase IQ that does away with some of the limitations required of a general purpose engine to produce a solution that is both much faster in load and user query performance and far more efficient in its disk usage than other general purpose databases. The other traditional database vendors have all upgraded their product suite to chase the Business Intelligence market

Into this market enters the data warehousing appliance vendors, a breed of dedicated hardware and software solutions designed to solve a business’ data warehousing woes. Such systems use low cost commodity components in large volumes with dedicated business intelligence engines to deliver radically faster load times whilst at the same time reducing the query times and simplifying the systems administration process.

The first hurdle for many organisations is that data warehousing appliances are to some extent proprietary and therefore going against a corporate policy of open systems to allow technology re-use, however a solution built on one of the current market leading platforms, Teradata, is no less so. In fact Teradata can be considered one of the original data warehouse appliances and it is the use of the low-cost commodity components and the ability to achieve massive parallelism by the new-comers that differentiates them.

The second hurdle is credibility – the promises of such large benefits (typically query performance of ten to fifty times faster whilst using three to six times less storage on a platform that only requires a small amount of systems administration support) will be doubted, especially by systems and database administrators who have had to work so hard to maintain the performance of the existing solution. Vendors such as Netezza have overcome this challenge with some key accounts by providing a system on the basis that if it meets agreed performance criteria it will be purchased and thus significantly reducing the risk to the purchasing company.

The final obstacle is migration and its associated cost: an existing solution that is built, for example, on an Oracle database, using Oracle Warehouse Builder and Oracle Discoverer is effectively proprietary and therefore more difficult, but not impossible, to migrate.  This is also a reason to review the existing data warehousing architecture now to ensure that when these and other new technologies come along the business will be able to take advantage of them. Companies will be able to get a clear competative advantage if, by architecting their Business Intelligence systems into functional components, they can quickly change and adopt bigger, faster, cheaper technologies.

Those organisations that have overcome the hurdles report that they are achieving the immediate huge performance gains for their queries without the need for tuning the database whilst lowering the disk footprint and reducing the support costs. The systems also continue to deliver benefit as the fast query times allow more complex data models to be queried, which in turn reduces the need for complex ETL to restructure the data. These changes to the data model and to reduce the complexity of the ETL can be made either as part of the migration project (which delivers the largest benefit quickly but at the greatest risk) or as part of the change management process for the source systems (which delivers benefit over a longer time frame but significantly reduces the risk).

Added to this is the emergence of MapReduce, originally develeoped by Google, it is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key. This is becoming a must-have feature for appliance vendors handling very large data sets.

There are now a significant number of vendors working to produce some form of data warehousing appliance (Netezza, GreenPlum, AsterData, etc. to name but a few) and it is clear that appliances are going to form a key part of data warehouse architectures going forward, the risks of using a smaller vendor and a proprietary solution being outweighed by the business benefit of much more timely information at a significantly reduced cost. Note also that there will be market consolidation and some vendors will disappear.

For further information analyst Curt Monash is just one of a number of analysts who follow this subject and provides regular updates on the market.

Read the original version of the article

This article was originally published on BIonRails, another Data Management & Warehousing website

Agile BI on Rails ETL Google MapReduce Netezza Oracle Sybase Teradata

Conspectus: Data Warehousing Appliances: Fad or Future

Posted on 1 April 2008 by David M. Walker Posted in Articles Leave a comment

The original article for Conspectus magazine.

Conspectus is the UK’s leading independent report which authoritatively addresses the key IT application areas. It is acknowledged as one of the most trusted journals in the IT industry and is used by many organisations to help them in their selection of IT software and technology.

An updated version of the article was also published on BI on Rails in November 2009

Download Data Warehousing Appliances: Fad or Future now

Data Warehousing DWH Appliances Netezza Technical Architecture Teradata

Archives

Random Quote

The best number is 73. Why? 73 is the 21st prime number. Its mirror (37) is the 12th and its mirror (21) is the product of multiplying, 7 and 3. … In binary, 73 is a palindrome, 1001001 which backwards is 1001001.

— Dr Sheldon Cooper (Big Bang Theory)

Contact Details

Tel (UK): 01458 850 433
Skype : datamgmt

Registered Office

Manchester House, High Street
Stalbridge, Sturminster Newton
Dorset, DT10 2LL
United Kingdom

Company Registration

Registered in England and Wales
Registration Number: 3526504
VAT Registration: GB724448236
Directors: DM & HJ Walker

© 1995-2013 Data Management & Warehousing
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish.Accept Read More
Privacy & Cookies Policy