consider the following methods:

Search Strategy and Article Selection

Studies were identified by searching MEDLINE, PREMEDLINE, Business Source Premier, ABI, and the Cochrane Library for January 1, 1996 February 28, 2005 using search concepts (with appropriate synonyms): 1) informatics/information systems; 2) patient care management/collaborative care; and 3) chronic illness. This strategy was supplemented by articles identified as key in the reference sections of the studies received and from experts in the field. Abstracts from identified articles were each rated separately by two reviewers into A (exact topic match), B (doubtful match), and C (no match) categories. Agreement was high (kappa0.78, p0.01) and consensus was used to resolve disagreements. All A articles were read and key information was abstracted from them unless: 1) information systems were not a major part of the study; 2) the study was wholly theoretical; or 3) the article did not describe what was done in enough detail to understand important components according to two or more reviewers. Review articles were analyzed separately. Only their general purpose and results were abstracted for comparison with current efforts.

Abstraction Method and Template Development

The primary units of analysis were the components of the system being studied, categorized using the eight Institute of Medicine (IOM) domains (Health Information and Data, Decision Support, Results Management, Communication/ Connectivity, Population Health Management, Order Entry, Patient Support, Administrative Processes).21 When articles compared two or more systems, each system was abstracted individually. Multiple articles discussing the same system were analyzed independently if the components under study were different. An abstraction template and a data dictionary were developed by consensus and iterative review of the team. The abstraction template included study design, setting (clinic, hospital, etc.), disease addressed, information system components and quality of care components. Study quality was evaluated based on study design, with experiments defined as the highest quality. To be classified as an experiment, a study had to have a control or comparison group, measure outcomes, and make appropriate statistical analyses.22 Other study designs included non-experiment observation, description, and case study.

Statistical Analyses

At least one reviewer of seven independently abstracted each study. The first author abstracted all review articles and independently re-reviewed 10% of all the articles as a verification step. Key verification variables included the study type, a summary variable for presence of each IOM domain (8 total), presence of quality, accuracy, and usability evaluations (3 total), and presence of cost-effectiveness analysis. A weighted kappa measurement was used to compare these 13 key indicators of consistency. Several limitations precluded formal meta-analysis. These limitations included a paucity of randomized, controlled trials (RCTs) among the articles reviewed, varying definitions of positive outcomes, inconsistent outcomes measurement, and lack of complete information system descriptions. Therefore, article counting methods were used where appropriate and a weighted sign test used to detect positivity of each studys results. Correlations were used to understand single components most associated with improvements in process and outcomes. Multivariable logistic regression was used to predict a positive effect on quality of care scores (outcomes) with information system components, study designs, and setting/disease treated.

Who were the participants describe appropriate characteristics. Describe the number of participants and the numbers assigned in each condition. Be sure to indicate whether they were randomly assigned to condition.

What was/were the independent variable(s)? How were they operationalized?

What was/were the dependent variable(s)? What tests, scales or instruments were used to operationally define each DV?

Describe what was done to collect data. Describe any attempts made to control confounds or extraneous variables.