.

.

necessities study the give records deeply and Drawing conclusions refers to records this is implied or inferred. ... the use of those clues to give for deeper information And provide the details Conclusions

2. ANALYSIS OF RELEVANT APPROACHES The existing body of research on ETL process development is constantly expanding to respond to the growing need for the formalization and automation of the development. The leading approach to software development today is Model Driven Development (MDD). The main goal of MDD is to enable the automation of software development in order to increase development productivity, reduce development time and cost, and improve the quality and flexibility of the obtained solution. To this end it promotes the use of abstractions which enable the analysis of a problem at different levels of detail. MDD is based on the premise that the most important product of software development is not the source code itself but rather the models representing the knowledge about the system that is being developed. In other words, in MDD, models are primary software artifacts and the development process is automated through appropriate model transformations which should ultimately result in a concrete implementation i.e. executable code. In light of the complexity of ETL processes and the problems related to their development (as explained in Section it can be stipulated that they should be developed in accordance with the MDD approach. It should be emphasized that in order to enable automatic model transformations the models must be formal. Therefore, in the next subsections an analysis of the most relevant approaches proposed for the formal specification of ETL processes will be given as well as an analysis of the different approaches for the automated implementation of such specifications. Moreover some open issues will be identified and discussed. ETL Process Specification 17 The first phase in ETL process development is the specification (.e. conceptual modeling) of ETL processes. The main goal of this phase is to define "what" the software solution should provide in terms of its basic functionality. In other words, conceptual models are high-level abstractions representing the knowledge i.e. semantics of the system that is being developed. Since ETL process development presumes the active participation of domain experts as they possess the necessary in-depth understanding of the domain) the modeling languages, used for the specification of ETL processes, should be easily understandable not only to software developers but also to the domain experts. According to (Fowler, 2010) communication between software developers and domain experts is the most common source of project failure. Therefore, it can also be stipulated that the models should be expressed in terms of concepts specific to this particular domain i.e. the concepts and terms used by the domain experts. In addition, the modeling languages should also be as simple as possible (that is, they should provide a minimal set of necessary concepts) but at the same time semantically rich to enable the specification of the various aspects of the ETL process domain at the appropriate level of abstraction. Finally, if the modeling languages are to properly represent the domain concepts, along with their semantics, they should also include the rules of the domain to ensure the correct usage of the concepts. These rules should be included in the modeling languages to prevent structural and semantic mistakes. Thus far, two distinctive approaches have emerged for realizing MDD, which differ primarily in the languages used for the specification of the models. One advocates the use of general purpose modeling languages (GPMLs) and their extension, while the other advocates the use of specially designed domain-specific languages (DSLs). It could be said that, in general, the existing body of research on ETL process development can be classified along the same lines. Based on the premise that ETL processes can be regarded as a special type of business process, and emphasizing the need for standardization, the use of existing general purpose modeling languages (such as Unified Modeling Language - UML or Business Process Model and Notation BPMN) has been proposed for the conceptual modeling of ETL processes. These languages have been appropriately extended in order to incorporate the concepts specific to the ETL process domain. More precisely, the extension of UML has been proposed in (Trujillo & Lujn-Mora, 2003; Lujn-Mora, Vassiliadis, & Trujillo, 2004; Muoz, Mazn, Pardillo, & Trujillo, 2008; Muoz, Mazn, & Trujillo, 2009), while the extension of BPMN has been proposed in (El Akkaoui & Zimnyi, 2009; El Akkaoui, Zimnyi, Mazn, & Trujillo, 2011; El Akkaoui, Mazn, Vaisman, & Zimnyi, 2012). However, it can be argued that since GPMLs were envisaged to support the description of the various aspects of any given business process in any given domain (in order to promote standardization) they include a large number of domain-neutral concepts which are defined at a low level of abstraction. According to (Kelly & Tolvanen, 2008) GPMLs do not raise the level of abstraction above code concepts. The complexity of these languages (.e. too many concepts whose semantics are imprecise) along with the fact that they are often too technical for domain-experts to master, lead to a number of issues related to the acceptance, utilization and value of these languages. Furthermore, since the core GPMLs do not contain knowledge of a particular domain, different concepts could be used and connected regardless of the domain rules (Kelly & Tolvanen, 2008). In other words, it is up to the designer to know the semantic rules (e.g. the legal connections and structures, the necessary data etc.) and ensure that they are fulfilled when defining the specification. It can further be argued that the extension of these languages only adds to their complexity while most of the drawbacks of GPMLs still remain. Moreover, in order to extend these languages it is necessary to be familiar with their concepts in order to be able to identify those which can be specialized. At the same time, the use of DSLs which are tailored to a particular domain has also been proposed in (Vassiliadis, Simitsis, & Skiadopoulos, 2002, May; Vassiliadis, Simitsis, & Skiadopoulos, 2002, November; Vassiliadis, Simitsis, Georgantas, & Terrovitis, 2003; Simitsis & Vassiliadis, A methodology for the conceptual modeling of ETL processes, 2003; Vassiliadis, Simitsis, Georgantas, Terrovitis, & Skiadopoulos, 2005; Simitsis, Vassiliadis, Terrovitis, & Skiadopoulos, 2005; Simitsis, 2005; Simitsis & Vassiliadis, 2008). The main benefit of DSLs, according to (Kelly & Tolvanen, 2008) is that, unlike GPMLs, they raise the level of abstraction beyond current programming languages and their abstractions, by specifying the solution in a language that directly uses the concepts and rules from a particular problem domain. Furthermore, they state that with GPMLs it is not possible to know how and when to reuse data from models, check design correctness based on the domain, separate the model data into different aspects relevant in the domain and so on, for the simple reason that these are impossible to standardize as they differ from one domain and company to another. The aim of DSLs is to provide only a minimal set of domain-specific concepts, with clear and precise semantics, along with a set of strict rules controlling their usage and the way in which they can be composed. Since DSLs allow for the inclusion of domain rules (in the form of constraints) both the syntax and the semantics of the concepts can be controlled, thus incorrect or incomplete designs can be prevented by making them impossible to specify. Therefore, in comparison with GPMLs, DSLs are more expressive (i.e. they enable a precise and unambiguous specification of the problem) while at the same time being more understandable and easier to use by domain experts (since they do not include unnecessary general purpose concepts). In addition, the use of such languages facilitates communication among the various stakeholders (from both the business as well as the technical communities) thereby promoting teamwork which is one of the main principles of current agile approaches to software development