Overview of ETL in Data Warehouses

You need to load your data warehouse regularly so that it can serve its purpose of
facilitating business analysis. To do this, data from one or more operational systems
needs to be extracted and copied into the data warehouse. The challenge in data
warehouse environments is to integrate, rearrange and consolidate large volumes of
data over many systems, thereby providing a new unified information base for
business intelligence.

The process of extracting data from source systems and bringing it into the data
warehouse is commonly called ETL, which stands for extraction, transformation, and
loading. Note that ETL refers to a broad process, and not three well-defined steps.

The acronym ETL is perhaps too simplistic, because it omits the transportation phase andimplies that each of the other phases of the process is distinct. Nevertheless, the entire process is known as ETL.

The methodology and tasks of ETL have been well known for many years, and are not
necessarily unique to data warehouse environments: a wide variety of proprietary
applications and database systems are the IT backbone of any enterprise. Data has to
be shared between applications or systems, trying to integrate them, giving at least
two applications the same picture of the world. This data sharing was mostly
addressed by mechanisms similar to what we now call ETL.

Posted in: ETL