.

.

necessities study the give records deeply and Drawing conclusions refers to records this is implied or inferred. ... the use of those clues to give for deeper information And provide the details Conclusions

18 ETL Process Implementation The second phase in ETL process development is the actual implementation of the ETL process specification. It should first be emphasized that only a few approaches exist which enable the automated development of ETL processes in the context of MDD. Generally, in order to enable the automation of the development in accordance with MDD, it is necessary to first map the domain concepts to design concepts and then on to programming language concepts. The way in which the actual automation of software development is achieved (Model Driven Architecture - MDA or Domain Specific Modeling - DSM) is another point of difference between the general purpose approach and the domain-specific approach. In the MDA approach, software development can be partially or fully automated through the successive application of model transformations, starting from the model representing the specification of the system (i.e. the conceptual model) and ending in a model representing the detailed description of the physical realization, from which the executable code can ultimately be generated. The development of ETL processes in accordance with the MDA approach is proposed in (Muoz, Mazn, Pardillo, & Trujillo, 2008; Mazn & Trujillo, 2008). Thus, the conceptual models are defined as platform independent models - PIM which are then automatically transformed into platform specific models - PSM (through a set of formally defined transformations) from which the code (necessary to create data structures for the ETL process in the corresponding platform) can be derived. However, since the PSMS must be specially designed for a certain technology of ETL processes (.e. each PSM must be based on the resources of a specific technology) the proposed approach presumes that a metamodel must be manually defined for each specific tool in order to create the transformations from the proposed conceptual model to each deployment platform. On the other hand, the MDA approach in general, is based on the refinement of models through successive model transformations, yet this process usually also requires that the automatically generated models be manually extended with additional details. These manual extensions could lead to a discrepancy between the original and generated models (i.e. the original models would become obsolete). This discrepancy is further emphasized when the modification of models, that were previously created by partial generation, is required. Since the correct modification of these models remains an unresolved issue, MDA advocates using a single GPML, namely UML, at all the levels (thereby lowering the abstraction levels of models) which not only entails all of the previously discussed issues regarding the use of GPMLs for modeling ETL processes, but also brings additional complexity to the development of model transformations (Fowler, 2010). Thus, an improvement of the proposed approach has been suggested in (El Akkaoui, Zimnyi, Mazn, & Trujillo, 2011) to directly obtain the code corresponding to the target platform, bypassing the need for the defining of an intermediate representation (metamodel) of the target tool. Thus, the conceptual model can be automatically transformed into the required vendor-specific code to execute the ETL process on a concrete platform. On the other hand, in the DSM approach the implementation is automatically generated from the specification (which can be modeled using domain-specific concepts) by code generators which specify how the information is extracted from the models and transformed into code. In other words, the generator reads the model based on the metamodel of the language and maps it to code. The generators are also domain- specific (i.e. they produce the code according to the solution domain) since, according to (Kelly & Tolvanen, 2008), this is the only way to enable full code generation i.e. the generation of code that does not need to be additionally modified. Usually the code generation is further supported by a domain-specific framework which provides implementation concepts, closer to the domain concepts used in the specification, thus narrowing the gap between the solution domain and the problem domain that would otherwise need to be handled by the code generator. The main benefit of DSM according to (Kelly & Tolvanen, 2008) is that generators, along with framework code, provide an automated direct mapping to a lower abstraction level (i.e. there is no need to make error-prone mappings from domain concepts to design concepts and on to programming language concepts) thus providing full code generation instead of resulting in a partial implementation. Because the generated code can be compiled to a finished executable without any additional manual effort, the specification (i.e. model) in fact becomes truly executable. It can, thus, be concluded that, if the goal is to formalize and automate the development of ETL processes to a significant extent, the DSM approach should be adopted not only, because it allows for the formalization of semantically rich abstractions in a form which can be reused, but also because it enables the automatic generation of executable code from models representing the specification of the system. On the other hand, the modeling concepts of GPMLs do not relate to any specific problem domain on the modeling side while on the implementation side, they do not relate to any particular software platform, framework, or component library. Furthermore, MDA assumes the existence of several models at different levels of abstraction obtained through progressive refinement (which can be both automatic and manual) thus automation is usually only partially achieved. An additional benefit is that in the DSM approach, both the models and the code generators can be easily changed (and the code then only needs to be regenerated) which makes the development process more agile. Finally, according to (Kelly & Tolvanen, 2008) domain-specific approaches are reported to be on average 300-1000% more productive than GPMLs or manual coding practices. As a final point, it is argued that the application framework (supporting the implementation of the ETL process specification in the DSM approach) should define specific implementation concepts which are more close to the real domain concepts introduced in the DSLs used for the specification of ETL processes. If both the specification and the application framework use formal concepts close to the real ETL domain concepts