db
Our analytic practice is built on top of the Vertica analytic database as the core platform. Vertica’s columnar MPP architecture is conducive to horizontal scaling, making it ideal for cloud deployments.
Vertica’s design involves the use of projections to compress and efficiently store data. This not only means that much less disk space is used, but also that the need for secondary artifacts such as indexes goes away. Another key design feature is that Vertica is designed to leverage commodity servers in a shared-nothing architecture. This allows for optimization techniques where query performance, disk space and data safety can be balanced by deciding how to store the data across nodes in the cluster, and how to design projections.
We also like Essbase, named one of the top 10 most influential technologies of the decade by Information Age (http://bit.ly/adKyg). We have been building OLAP cubes using Essbase Integration Services since 2002.
We also work with relational databases such as MySQL, Oracle and SQL Server. Typically, these coexist in some form with Vertica at most customer installations we have been involved with.
etl
We rely on scripting tools and languages such as Perl, Awk, Sed, and Ruby for bulk data transformation and loading.
Among packaged vendor solutions, we have used ETL (Extract, Transform and Load) tools such as Informatica, Microsoft DTS, and Talend. For complex source systems such as integrating data from ERP/CRM systems such as Siebel, E-Business Suite, Hyperion and SAP such an ETL tool provides functionality that will be hard to replicate. For “big data” challenges however, sometimes scripting is the way to go for loading bulk data fast.
cloud
The cost benefits of pay-as-you-go cloud computing are best encapsulated by the $6.40 cost for a 64-machine cluster deployed recently by Eli Lilly on Amazon EC2 (http://bit.ly/WpJ1O)
Other than reduced upfront hardware costs, we further leverage cloud computing to shortcut expensive installation cycles and IT staff by leveraging fully built orchestration of ETL, DB and BI servers pre-configured to work together.
For cloud environments, we work with the leader, Amazon EC2. Technology by Rightscale provides the ability to orchestrate a deployment, which can subsequently auto-scale as the installation gets more traction.
frontend and visualization
Our front-end solutions of choice are Tableau and Jaspersoft.
In the past, we have also built reporting solutions using Hyperion Brioquery, Oracle OBIEE, and custom-built ones using Cold Fusion and Perl.
We are exploring emerging visualization technologies such as Processing and Flare.