ETL stands for Extract Transform & Load Part 1 - ETL- iNFORMATICA DEVELOPER

Tuesday, May 21, 2019

ETL stands for Extract Transform & Load Part 1


ktnewslive


What is ETL?

Ø  The process of updating the data warehouse

Ø  ETL is the automated and auditable data  acquisition process from source system that involves one or more sub processes of data extraction, data transportation, data transformation, data consolidation, data integration, data loading and data cleaning. 

Need for ETL

Ø  The process of ETL is required so that data from different heterogeneous sources can be combined and brought into one common source.

Ø  The Advantage of having the process of ETL is that, as data from different sources can be brought together, highly complex and user friendly reports can be generated for decision making

----------------------------------------------------------------------------------------------
Ø  Data stored in different formats in different types of databases.

Ø  Some data sources might be archives while others may be active operational systems

Ø  Data extraction and cleansing - time-consuming and difficult

Ø  Aggregation of data


What ETL is Not ?

·         Data warehouses are not OLTP systems

    Duplication of calculations in Source system & the  data warehouse should not be attempted, as in future the process in the source system can change that will result in asynchronous data.

Feature of ETL Tools

Ø  Support data extraction, cleansing, aggregation, reorganization, transformation, and load operations

Ø  Generate and maintain centralized metadata

Ø  Filter data, convert codes, calculate derived values, map source data fields to target data fields

Ø  Automatic generation of ETL programs

Ø  Closely integrated with RDBMS

Ø  High speed loading of target data warehouses using Engine-driven ETL Tools




Advantages of using ETL Tools



Ø  GUI based design of jobs – ease of development and maintenance

Ø  Generation of directly executable code

Ø  Engine driven technology is fast, efficient and multithreaded

Ø  In-memory data streaming for high-speed data processing

Ø  Products are easy to learn and require less training

Ø  Automatic generation and maintenance of open, extensible metadata

Ø  Support for multiple data formats and platforms

Ø  Large number of vendor supplied data transformation objects

Meta Data

Ø  Data about data

Ø  Needed by both information technology personnel and users

Ø  IT personnel need to know data sources and targets; database, table and column names; refresh schedules; data usage measures; etc.

Ø  Users need to know entity/attribute definitions; reports/query tools available; report distribution information; help desk contact information, etc.








2 comments:

  1. IEEE Final Year projects India Project Centers in India are consistently sought after. Final Year Students Projects take a shot at them to improve their aptitudes, while specialists like the enjoyment in interfering with innovation. For experts, it's an alternate ball game through and through. Smaller than expected IEEE Final Year project centers ground for all fragments of CSE & IT engineers hoping to assemble. Final Year Projects for CSE It gives you tips and rules that is progressively critical to consider while choosing any final year project point.

    Spring Framework has already made serious inroads as an integrated technology stack for building user-facing applications. Spring Framework Corporate TRaining the authors explore the idea of using Java in Big Data platforms.
    Specifically, Spring Framework provides various tasks are geared around preparing data for further analysis and visualization. Spring Training in Chennai


    The Nodejs Training Angular Training covers a wide range of topics including Components, Angular Directives, Angular Services, Pipes, security fundamentals, Routing, and Angular programmability. The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training

    ReplyDelete