Identify the process evaluation article that you chose and explain why you selected this example. Describe the
Question:
Identify the process evaluation article that you chose and explain why you selected this example.
- Describe the purpose of the evaluation, the informants, the questions asked, and the results of the evaluation.
- Identify the stage of program implementation in which the evaluation was conducted.
- Consider why the researchers chose to evaluate at that stage of program implementation. What kind of information would they have received if they had conducted the evaluation earlier or later?
- If you were to replicate the study, would you adjust it in any way for more optimal results?
Answer the question using this article
Introduction Child maltreatment—defined as physical, emotional, and sexual abuse and neglect (World Health Organization, March 1999)—affects millions of children each year (World Health Organization, 2014). Moreover, prevalence rates in low- and middle-income countries (LMICs) are much higher in comparison with those in high-income countries (HICs; Stoltenborgh, Bakermans-Kranenburg, van IJzendoorn, & Alink, 2013). In South Africa, a recent survey of 3,515 youth aged 10-17 years showed lifetime prevalence rates of 55% for physical abuse, 36% for emotional abuse, and 9% for contact sexual abuse (Meinck, Cluver, Boyes, & Loening-Voysey, 2016). This is a serious public health issue due to associations between child maltreatment during early childhood and increased risks of developing negative child, adolescent, and adult outcomes later in life (Font & Berger, 2015).Over the past decade, public health and children's rights organizations have issued numerous calls for the global dissemination of evidence-based interventions to prevent violence against children (World Health Organization & United Nations Children's Fund, 2008). In particular, parenting programs have demonstrated promising evidence in reducing harsh parenting and other factors associated with increased risk of maltreatment (Barlow, Johnston, Kendrick, Polnay, & Stewart-Brown, 2006; Chen & Chan, 2015; Furlong et al., 2013). A recent meta-analysis also showed that evidence-based parenting programs transported to new countries may be as effective as in their country of origin (Gardner, Montgomery, & Knerr, 2015). However, this review identified only one study of a program developed in an HIC and transported to an LMIC (Iran). As a result, there remains limited evidence regarding the effectiveness and feasibility of parenting programs in LMICs (Knerr, Gardner, & Cluver, 2013).The purpose of this mixed-methods process evaluation study was to determine the feasibility of an evidence-informed, locally developed parenting program for low-income families in Cape Town, South Africa. We used the United Kingdom Medical Research Council guidance as a theoretical framework for conducting process evaluations of complex interventions (Moore et al., 2004). Process evaluations have become increasingly relevant as a complement to outcome evaluations (Oakley et al., 2006). They are particularly important when examining issues of program feasibility during early stage intervention development or when adapting parenting interventions for new settings (Lau, 2006). Process evaluations may also provide greater understanding of how a program functions in a given context prior to subsequent testing in larger trials (Moore et al., 2004).Previous process evaluations of parenting programs in HICs have identified three key dimensions that may influence the feasibility and effectiveness of parenting programs: participant involvement, implementation, and acceptability (Berkel, Mauricio, Schoenfelder, & Sandler, 2011; Kazdin, 2000). Participant involvement—the extent to which participants enroll, attend, and engage in a program—has been shown to be an important component in understanding whether an intervention is feasible in a particular context (Andrews & Dowden, 2005). Participation data on enrollment and attendance rates can help determine the minimum number of group sessions necessary to assure positive intervention effects or explain null effects (Flay et al., 2005). In addition, the quality of participant engagement may be equally important in predicting changes in parenting behavior (Nix, Bierman, & McMahon, 2009). Implementation refers to the intervention components delivered to a target population and the quality of delivery by program implementers (Durlak & DuPre, 2008). This is particularly important when examining null effects of interventions, which may either be a result of program ineffectiveness or because the program was not delivered as intended (Borrelli, 2011). Implementation may also be influenced by other factors including training of program facilitators and organizational management dynamics (Moore et al., 2004). Lastly, acceptability can be defined as the extent to which a program is perceived as culturally and contextually relevant by the intended service providers and beneficiaries (Kazdin, 2000). Cultural and contextual factors may include local language; population; metaphors; content; concepts; goals; methods; and the political, economic, and social environment in which an intervention is delivered (Bernal, 2006). If a program fails to accommodate these factors, it may compromise implementation and participant engagement, thus further diminishing program feasibility and effectiveness (Castro, Barrera, & Martinez, 2004).In South Africa, issues regarding program feasibility may be very different from those present in HICs, especially when delivering parenting programs to highly vulnerable families. Many South African families live in isolated communities characterized by extreme poverty and high violence, where access to resources and professional services is limited (Meth, 2013). Service delivery often relies on nonspecialist community workers who are either volunteers or low-paid paraprofessionals (Altman, 2008). In addition, since many children are often raised by multiple caregivers over the course of their childhoods, it is important to investigate whether parenting programs designed for nuclear families are relevant in South Africa (Bray & Brandt, 2007). Furthermore, potential differences in cultural norms and practices regarding caregiving may affect program feasibility. Since corporal punishment is commonly accepted as a normative discipline strategy by South African parents and children (Breen, Daniels, & Tomlinson, 2015; Dawes, De Sas Kropiwnicki, Kafaar, & Richter, 2005), research is necessary to examine the acceptability of more authoritative parenting styles developed in HICs that promote positive reinforcement and nonviolent discipline. Finally, South African cultural values such as respect, social responsibility, and reciprocity may have a significant impact on the feasibility of parenting programs and their relevance to target populations (Bray, Gooskens, Kahn, Moses, & Seekings, 2010).In summary, this study addressed the following research questions regarding the feasibility of a parenting program in Cape Town, South Africa: (a) What is the level of participant involvement, implementation, and acceptability of an evidence-informed, group-based parenting program delivered by community facilitators to low-income parents with children aged 3-8 years? (b) What are the barriers and enablers to program implementation and participant involvement in a low-resource context as perceived by community facilitators and low-income parents? (c) How do South African facilitators and parents perceive the acceptability of a parenting program derived from evidence-based principles and approaches? Method Study Design This mixed-methods process evaluation study was conducted alongside a small-scale randomized controlled trial (RCT) that examined the preliminary effectiveness of the intervention with low-income families with children aged 3-8 years (Lachman et al., 2015). Participants in the RCT were randomly allocated on a 1:1 ratio to a treatment or wait-list control group. Although the control group received no comparative active treatment during the trial, participants were wait-listed to receive the parenting program directly after posttreatment outcome data were collected. As a result, in addition to examining the feasibility of the program for the entire sample, this study compared process data from both groups after all subjects had participated in the program (see Figure 1 for flow diagram).Figure 1. Process evaluation study flow diagram. OPEN IN VIEWER Setting This study was conducted from April to November 2013 in Khayelitsha, a high-density, low-income suburb of Cape Town with a population of approximately 390,000 predominantly isiXhosa-speaking people (isiXhosa is a local, indigenous South African language; Statistics South Africa, 2012). The majority of families in Khayelitsha live in either government-subsidized housing, fabricated homes in the backyards of formal homes, or corrugated tin shacks in informal settlements with limited access to running water, sanitation, or electricity. Khayelitsha is also characterized by high levels of poverty, unemployment, community violence, drug and alcohol abuse, and HIV prevalence (Statistics South Africa, 2012). Similar to many other low-income communities in South Africa, children in Khayelitsha are predominantly raised by female caregivers—mothers, aunts, grandmothers, or older sisters—with minimal presence of fathers in the household (Bray et al., 2010).Participants Sixty-eight parents (age: M = 41.57, SD = 13.15; 98.5% female) were recruited from referrals by a local nongovernment organization (n = 47, 69%) and from a formative evaluation conducted in 2012 (n = 21, 31%; Lachman et al., 2016). Recruitment from the local organization was based on referrals from families enrolled in an orphans and vulnerable children program. The program provided families with ongoing services that included food parcels, paralegal advice, and assistance in enrolling children at early childhood development centers. Recruitment from the formative evaluation included chain referral sampling using community volunteers as original informants who recruited participants through neighborhood contacts (Miernicki & Waldorf, 1981).Inclusion criteria required parents to be (a) living in Khayelitsha, (b) isiXhosa-speaking, (c) aged 18 or above, (d) the primary caregiver of a child between the ages of 3 and 8 (child age: M = 5.40, SD = 1.69; 48.5% female), and (e) a resident in the same household as the child for at least 4 nights per week. Although parents were predominantly single mothers (73.5%), nonbiological parents, relatives, or nonskin foster caregivers who met the other trial criteria were also included. In order to recruit parents who were at risk of using harsh discipline practices to manage their children's behavior, we screened families based on parent report of their child's behavior using the Eyberg Child Behavior Inventory Problem Scale with a cutoff of 11 or more symptoms (Eyberg & Pincus, 1999).Participating families were from highly vulnerable populations, mostly living in informal houses and overcrowded conditions. Seventy-nine percent reported experiencing household hunger more than 5 days during the previous month at baseline (e.g., running out of money to buy food). About one third of the respondents were either caring for a child orphaned by AIDS or AIDS symptomatic themselves; one third reported being victims of intimate partner violence in the previous month; and almost three quarters of the parents had experienced some form of physical abuse as children. With the exception of interviewees living in more crowded households, independent t-tests and χ2 tests found no significant differences at baseline between those who were interviewed and the rest of the sample (Table 1).Table 1. Characteristics of Parents Who Participated in the Study Including Those Who Were Interviewed After Program Delivery.
Variable | Total (n = 68) | Interviewed (n = 15) | Not Interviewed (n = 53) |
---|---|---|---|
Parent characteristics | |||
Parent age, M (SD) | 41.57 (13.15) | 40.47 (11.41) | 41.89 (13.69) |
Parent gender, n female, % | 67, 98.5 | 15, 100.0 | 52, 98.1 |
Marital status, n single, % | 50, 73.5 | 9, 60.0 | 41, 77.4 |
Employment status, n unemployed, % | 66, 97.1 | 14, 93.3 | 52, 98.1 |
Child characteristics | |||
Child age, M (SD) | 5.40 (1.69) | 6.13 (1.41) | 5.19 (1.72) |
Child gender, n female, % | 33, 48.5 | 10, 66.7 | 23, 43.4 |
Relationship to parent, n biological parent, % | 41, 60.3 | 10, 66.7 | 31, 58.5 |
Family characteristics | |||
Type of household structure, n informal, % | 51, 75.0 | 12, 80.0 | 39, 73.1 |
Number of people in household, M (SD) | 5.71 (2.86) | 7.13 (4.22) | 5.30 (2.23)* |
Experience hunger ≥ 5 times in previous month,an, % | 54, 79.4 | 13, 86.7 | 41, 77.4 |
Family affected by HIV/AIDS,bn, % | 21, 30.9 | 5, 33.3 | 16, 30.2 |
Parent experienced intimate partner violence in previous month,cn, % | 23, 33.8 | 5, 33.3 | 18, 34.0 |
Parent experienced physical abuse as a childd, n, % | 49, 72.1 | 13, 86.7 | 36, 67.9 |
Program participation | |||
Number of parents enrolled in program, n, % | 57, 83.8 | 15, 100.0 | 42, 79.6 |
Mean attendance rate of parents, % (SD) | 71.5 | 75.0 | 70.3 |
aBased on the Hunger Scale Questionnaire (Labadarios et al., 2003). bBased on three or more symptoms on the Verbal Autopsy Questionnaire (Lopman et al., 2006). cConflict Tactics Scale (Straus, Hamby, Boney-McCoy, & Sugarman, 1996). dInternational Society for the Prevention of Child Abuse and Neglect Child Abuse Screening Tool-Retrospective (Dunne et al., 2009).
*p < .05 when testing for differences between those who were interviewed and those who were not.
OPEN IN VIEWER Community facilitators were employees of Clowns Without Borders South Africa, a nonprofit organization in South Africa (www.cwbsa.org). Employment qualifications required facilitators to be isiXhosa-speaking parents, have prior experience facilitating groups, basic training in early childhood development, and live in close proximity to the community where the study took place. Facilitators were all mothers with a minimum high school-level education (Age: M = 36.50, SD = 4.90).The Sinovuyo Caring Families Program Program content The Sinovuyo Caring Families Program is a 12-session parenting program that integrates evidence-based parenting principles and practices within a local South African context. It was developed in 2012 using community-based participatory approaches with multiple stakeholders in South Africa (Lachman et al., 2016). It is based on social learning theory in which parents play an important role in modeling and reinforcing child behavior (Bandura, 1977). Evidence-based parenting approaches are presented using a cultural metaphor of constructing a "Rondavel of Support"—a traditional hut with mud walls and a thatched roof that is found in many rural African communities (Figure 2). The initial sessions address the development of positive parent-child relationships (i.e., mud walls) prior to learning nonviolent discipline strategies (i.e., thatch roof) during the latter half of the program. Each session is introduced using a traditional Southern African story related to the specific session's theme. The first six sessions of the program focus on (a) establishing parent goals around child behavioral outcomes; (b) spending special time with children through child-led play; (c) descriptive commenting; (d) communicating about emotions; (e) using specific, labeled praise; and (f) using rewards to encourage positive behavior. The final six sessions address limit setting and nonviolent discipline skills by (a) giving positive and clear instructions, (b) establishing household rules, (c) ignoring negative attention-seeking behavior, (d) using time-out for aggressive behaviors or noncompliance, (e) using realistic consequences, and (f) involving children in problem-solving. The program also addresses local issues identified by stakeholders regarding supervision of children in high-crime communities and developmentally appropriate ways to communicate with children about HIV/AIDS and poverty. Finally, participants learn simple mindfulness exercises such as body relaxation techniques to reduce stress associated with parenting and other social factors (Kabat-Zinn, 1994).Figure 2. "Building a Rondavel of Support" program model for the Sinovuyo Caring Families Program. OPEN IN VIEWER Program design Each session of the Sinovuyo program follows a similar format and is approximately 2.5 hr-long. At the beginning of each session, parents are asked to share about their experience implementing parenting skills at home during the previous week. When parents report having challenges with home practice activities, the facilitators involve them in a group problem-solving process that includes role-playing of solutions. Next, parents are introduced to the session's core parenting skill (e.g., using specific, labeled praise). Then, parents are guided through illustrated stories, or scenarios, that depict scenes of typical South African families demonstrating parenting practices either correctly or incorrectly (Figure 3). Next, the parents practice these skills in role-plays with one parent acting as the "child" and the other as the "adult." Finally, sessions close with parents receiving assignments to practice these parenting skills at home. Example of an illustrated storyboard with a grandmother reinforcing her granddaughter's compliance with instructions by using labeled praise and tangible rewards (illustrations by Shifrah Getz from Inside Out Inspired Designs™).OPEN IN VIEWER Program delivery The Sinovuyo program was developed as a low-cost intervention capable of being delivered by community-based facilitators with limited professional experience prior to training. Facilitators receive initial training in collaborative facilitation methods and parenting principles (30 hr), weekly training on specific session content during implementation (20 hr), and experiential training by participating in the program prior to facilitation (24 hr). A key aspect of the training is reinforcing the program's collaborative learning approach to promote parental participation. This method requires the following expertise from program facilitators: (a) an understanding of social learning principles, (b) the ability to actively engage participants in identifying parenting principles and developing effective child behavior management strategies, (c) the ability to facilitate discussions with open-ended questions in order to generate collective problem-solving, (d) the capacity to model positive parenting behaviors, (e) and an empathic attitude toward each parent's individual experiences (Hutchings, Gardner, & Lane, 2004). Facilitators also participate in weekly supervision sessions that involve reviewing video recordings of sessions, problem-solving challenges experienced while delivering the program during the previous session, and role-playing solutions to improve the quality of delivery. Scaffolding The Sinovuyo program uses programmatic "scaffolding" to support implementation and participant involvement. Traditionally, scaffolding refers to a parent's active and structured support of child cognitive and behavioral development using problem-solving and encouragement (Wood, Bruner, & Ross, 1976). This concept can also be applied to the extent to which a program is able to support facilitators with the skills needed to engage parents in the behavior change process. Accordingly, the Sinovuyo program includes a facilitator manual and parent handbook to strengthen treatment fidelity and participant adherence to prescribed activities. Weekly text messages serve as boosters to remind parents of home practice activities and encourage attendance at sessions. Facilitators also perform home visit consultations to support parents who are having difficulties with program material or attending group sessions. In order to create a peer support network outside the group sessions, the program also encourages participants to form "Sinovuyo partner" relationships with other parents in the program. Finally, participants receive financial support for public transportation and lunch at each session. Ethical Procedures This study was granted ethical approval by University of Oxford Central University Research Ethics Committee (ref SSD/CUREC2/11-40) and the University of Cape Town Psychology Department Research Ethics Committee (ref 2012_12_01). Research assistants with prior experience working on research projects in South Africa conducted informed consent procedures with the participants. They were fluent in isiXhosa and extensively trained in ethics, informed consent, and interviewing techniques. All participants were told that they had the right to decline to participate and that they could drop out of the feasibility study at any time and continue participating in the program and the RCT study .Quantitative Data Collection and Analysis Participant involvement was based on enrollment, attendance, and dropout rates as well as the rate of parental engagement in home practice activities. Enrollment rate was based on the percentage of participants in the study who attended at least one session. Attendance was measured using signed attendance registers at each session and by examining home visit consultation reports for parents who missed a group session. Dropout from the program was based on parents who missed three consecutive sessions and were unavailable for home visit consultations. Finally, parental engagement was measured in each session by calculating completion rates of assigned home activities for those who had attended the previous session. Program fidelity was assessed using self-report checklists completed by facilitators who viewed video recordings from each session and indicated whether or not they delivered a particular component as specified in the program manual. Video recordings of a random sample of four sessions per parent group were examined in order to verify the accuracy of self-report data. Program fidelity scores were created by calculating a ratio of number of activities implemented to the number of prescribed activities. Parents' perceptions of overall program acceptability were assessed using a 36-item questionnaire adapted for a South African context from the Incredible Years Parent Program Satisfaction Questionnaire (Webster-Stratton, 1989). Using a Likert-type scale of 0-5, participants reported on whether the program fulfilled their overall expectations (10 items, e.g., "achievement of one's goal"), the acceptability of delivery methods (10 items, e.g., "practice of play skills at home"), the acceptability of parenting skills (10 items, e.g., "naming feelings"), the quality of program facilitation (4 items, e.g., "quality of interaction"), and the supportiveness of the parenting group (2 items, e.g., "group's support of parent"). Research assistants who were not associated with program delivery or the RCT administered the survey at the end of program completion. Items were summed to create an overall program satisfaction rating as well as satisfaction ratings for each subscale. Total scores were based on weighted means out of 100 for comparison purposes across subscales. We conducted independent t-tests and χ2 tests to examine potential differences between the initial treatment and wait-listed groups for participant involvement, implementation, and acceptability data. Tests for significance were set at the p < .05 level. Qualitative Data Collection Data collection We collected qualitative data from four sources: (a) individual interviews with 15 randomly selected parents who had attended at least one session of the program (treatment: n = 8; waitlist: n = 7), (b) focus group interviews with facilitators, (c) transcripts from parenting group sessions, and (d) transcripts of supervision session of facilitators. All interviews took place in the homes of participants during the month after post program data collection and lasted approximately 1 hr. An English-speaking researcher conducted the interviews with the assistance of a professional isiXhosa-speaking interpreter. The interpreter was instructed to translate the English questions and isiXhosa answers verbatim during each interview. The interpreter was also provided with the interview guide prior to the interviews in order to familiarize herself with the purpose and structure of the interview. Interviewees were given a small food parcel as compensation for participation (approximate value: US$5). Focus group interviews with community facilitators (n = 8 participants) took place at a community center after the completion of each round of program implementation, lasted approximately 2.5 hr, and were conducted in English by the first author. Interviews were captured on digital recorders and transcribed verbatim with written notes as backup. Research assistants also documented parent report of engagement in weekly home practice assignments at each parenting group session. Likewise, a research assistant recorded detailed minutes of each supervision session of facilitators. Interview guide We used a semi structured interview guide with an open-ended approach to examine the following themes: (a) perceived changes in parenting style, child behavior, and parent-child relationships; (b) perceived appropriateness of program content, design, and method of delivery; and (c) perceived barriers and enablers to participation and engagement in the program. Additional probing explored the extent to which respondents understood and utilized the parenting skills taught during the program, and whether they found the material culturally acceptable and appropriate to their context. Qualitative data analysis We used a contextualism framework to guide qualitative data collection and analysis (Braun & Clarke, 2006). As a pragmatic approach situated between essentialist and constructionist epistemologies, a contextualism framework allows for the integration of theory-driven, deductive approaches with data-driven inductive methods (Ferriday & Muir-Cochrane, 2006). Two data coders used thematic analysis to manually assess, code, and group data into clusters of themes (Braun & Clarke, 2006). Each coder read through the transcripts identifying initial codes that emerged from the texts. Queries arising from interpretation or context were discussed and clarified with isiXhosa-speaking research assistants. Initial codes were grouped into categories based on the following a priori classification: program content, delivery, and structure. These categories were further divided into themes regarding perceived program feasibility based on participant involvement, implementation, and acceptability. After data convergence of coding schemes was reached by the research team, we reexamined the transcripts to identify coherent patterns and divergent viewpoints. The entire team then discussed findings in relation to the overall data set and in the light of existing literature, with particular focus on the representativeness of individual themes. Finally, we selected data extracts to illustrate key emergent themes. Trustworthiness We used a number of measures to strengthen the trustworthiness of findings. First, to increase the diversity of perspectives and experiences, we collected data from multiple sources that included parents and facilitators during and after program implementation. Second, we attempted to reduce selection bias by randomly selecting the participants for interviews from all of the parents who had attended at least one session. Third, the interviews were conducted in the participants' homes in order to increase the comfort level of respondents. Fourth, audiotapes with verbatim transcription ensured that we accurately captured the data. We also translated the isiXhosa portions of the transcripts into English to verify the interpretation. Fifth, we maintained an audit trail that included a registered trial protocol (NCT01802294; PACTR201302000455414), coding matrix, and minutes from research team meetings. ResultsLevels of Participant Involvement, Implementation, and Acceptability Results from quantitative analyses of participant involvement, implementation, and acceptability are summarized in Table 2 and described in detail below. Table 2. Levels of Participant Involvement, Implementation, and Acceptability for Overall Sample as well as by Initial Treatment and Wait-Listed Groups.
Variable | Total (n = 68) | Treatment (n = 34) | Waitlist (n = 34) |
---|---|---|---|
Participant involvement | |||
Enrollment, n enrolled, % | 57, 83.8 | 32, 94.1 | 25, 73.5* |
Mean attendance, n sessions,a % | 8.58, 71.5 | 9.00, 75.0 | 8.00, 67.0 |
Attended 6 or more sessions,an, % | 47, 82.5 | 28, 87.5 | 18, 72.0 |
Attended 11 or more sessions,an, % | 19, 33.3 | 11, 34.4 | 8, 32.0 |
Program dropout,an, % | 8, 14.0 | 4, 12.5 | 4, 16.0 |
Engagement in home practice activities, %b | 87.9 | 86.9 | 88.9 |
Implementation | |||
Activities implemented, % | 92.6 | 90.6 | 96.6* |
Acceptabilityc | |||
Total satisfaction score, M (SD) | 94.07 (5.35) | 94.22 (5.12) | 93.83 (6.83) |
Ability to meet goals of parents, M (SD) | 88.77 (9.09) | 89.55 (8.27) | 87.56 (10.42) |
Appropriateness of parenting skills, M (SD) | 95.91 (6.72) | 96.00 (5.81) | 95.78 (8.17) |
Format of delivery, M (SD) | 94.85 (10.30) | 94.41 (11.97) | 95.56 (7.08) |
Quality of delivery, M (SD) | 98.62 (3.08) | 98.62 (3.24) | 98.61 (2.87) |
Supportiveness of group, M (SD) | 95.60 (7.77) | 97.24 (3.79) | 92.96 (11.31) |
aBased on number of enrollees. bBased on completion rates of assigned home activities for those who had attended the previous session. cIncredible Years Program Satisfaction Questionnaire (Webster-Stratton, 1989).
*p < .05 when testing for differences between initial treatment and wait-listed groups.
OPEN IN VIEWER Enrollment, attendance, and engagement Quantitative data demonstrated a high rate of overall program enrollment: 83.8% of the participants (n = 57 of the 68) included in the study attended at least one session (treatment: 94.1%, n = 32/34; waitlist: 73.5%, n = 25/34). Reasons for nonenrolment included obtaining full-time employment, relocation outside Cape Town, and refusal to participate. χ2 tests showed that parents allocated to the wait-list control group enrolled at a significantly lower rate than those in the initial treatment group, χ2(1, N = 68) = 5.31, p = .02.Enrolled parents attended an average of 8.58 of the 12 sessions (treatment: 9/12; waitlist: 8/12). Of a total of 57 enrollees, 47 (82.5%) attended six or more sessions (treatment: n = 28/32, 87.5%; waitlist: n = 18/25, 72.0%), and 19 (33.3%) missed only one or no sessions (treatment: n = 11/32, 34.4%; waitlist: n = 8/25, 32.0%). Reasons for missed sessions included part-time employment, days on which government grants had to be collected, parent or child illness, severe weather, funerals, and political unrest. Eight participants (14.0%) were characterized as program dropouts after having missed at least three consecutive sessions and not returning to the program (treatment: n = 4/32, 12.5%; waitlist: n = 4/32, 16.0%).Attending parents also engaged in an average of 87.9% of the assigned activities during the previous week (treatment: 86.9%; waitlist: 88.9%). Reasons for not completing home practice included missing the previous session, parent or child illness, or a lack of perceived self-efficacy in applying the assigned parenting skills. Independent t-tests found no significant differences between initial treatment and wait-listed groups for attendance, dropout, or home practice engagement rates (p > .05).Program fidelity Analyses of self-report of fidelity checklists showed that an overall 92.9% of the manualized activities were implemented by facilitators during program delivery. This was verified with 100% reliability by examining four randomly selected video recordings per parent group. Independent t-tests showed significantly higher program fidelity in the wait-listed group (M = 0.96, SD = 0.01) than in the initial treatment group (M = 0.91, SD = 0.01), t(3) = 4.21, p = .024, d = 4.86, 95% CI [0.80, 8.92].Participant satisfactionOf a total score of 100, participants reported a high overall satisfaction with the program (full sample: M = 94.07, SD = 5.35; treatment: M = 94.22, SD = 5.12; waitlist: M = 93.83, SD = 5.83). Parents gave high ratings to the program's ability to meet their goals and expectations (full sample: M 88. = 77, SD = 9.09; treatment: M = 89.55, SD = 8.27; waitlist: M = 87.56, SD = 10.42), the appropriateness of parenting skills taught during the program (full sample: M = 95.91, SD = 6.72; treatment: M = 96.00, SD = 5.81; waitlist: M = 95.78, SD = 8.17), the format of delivery (full sample: M = 94.85, SD = 10.30; treatment: M = 94.41, SD = 11.97; waitlist: M = 95.56, SD = 7.08), the quality of delivery by community facilitators (full sample: M = 98.62, SD = 3.08; treatment: M = 98.62, SD = 3.24; waitlist: M = 98.61, SD = 2.87), and supportiveness of the group (full sample: M = 95.60, SD = 7.77; treatment: M = 97.24, SD = 3.79; waitlist: M = 92.96, SD = 11.31). There were no significant differences between the initial treatment and the wait-listed groups. Barriers and Enablers of Participant Involvement, Implementation, and Acceptability Qualitative analyses identified seven emergent themes that influenced participant involvement, implementation, and acceptability (Figure 4).Figure 4. Conceptual model for program feasibility based on results. OPEN IN VIEWER Supporting participant involvement Respondents identified a number of program components that strengthened participant involvement. Parents reported that the preprogram home visit consultations helped overcome potential barriers to enrollment by orienting them to program goals and structure: "It was important because they explained to me about the program. If they didn't come [to my house] I wasn't going to go to the program" (parent #2, interview). Parents explained how the parent handbook also supported engagement and adherence to program content: "The handbook helped me a lot because sometimes I would go back and look at maybe where I went wrong" (parent #65, interview). Respondents also described how the weekly text messages reminded parents of their home practice, reinforced core principles, and created a sense of program continuity during the week. This had limitations, however, as those who did not own a cell phone were not able to benefit from the text message reminders. Parents also appreciated the role of individual home visit consultations in maintaining participant involvement for those who missed a group session. Conversely, facilitators reported that parents who missed many sessions were often unavailable for home visits. Parents also described how peer networks within the group provided additional opportunities for learning and support: "Sometimes it was helpful. If there was something I didn't understand, I was phoning her and asking what must I do" (parent #2, interview). However, not all parents interacted with their support network: "I'm not gonna lie. I never contacted her. She never contacted me" (parent #70, interview). Lastly, respondents emphasized that providing transportation, childcare, and lunch helped encourage consistent program attendance. Engaging parents in the collaborative approach Respondents demonstrated a high level of receptivity to the program's emphasis on using a collaborative learning approach to engaging parents in evidence-based parenting skills. In spite of their relative lack of experience, facilitators displayed a proficiency in adopting reflective listening techniques to elicit parent's views and needs: "Before, I would correct people, but I learned that it blocks them from expressing themselves. Accepting and exploring creates an atmosphere where people can be open" (facilitator #1, focus group). Many parents were also receptive to the program's use of collective problem-solving to resolve challenges parents encountered when learning new parenting practices: "It was helpful for us as a group to talk together and try to solve our problems, to come up with a solution" (parent #51, interview).The program's combination of illustrated stories, role-plays, and home practice activities was also perceived to be an effective approach to promote participant involvement. In particular, respondents viewed the illustrated stories as a useful strategy to engage parents in the collaborative learning process. As one facilitator explained, "the illustrated stories were one part where you see [the parents] come alive. You really see them sharing and thinking and talking" (facilitator #3, focus group). Parents also articulated how the scenarios helped communicate parenting skills in a clear and understandable manner: "I can see the granny with the child, how she is sitting, and how they are talking ... so, it helps to understand that you have to go to that level when you talk to the child" (parent #43, interview). This was especially relevant to parents with low levels of literacy: "The pictures were helping a lot. Even though I don't understand the language in terms of reading" (parent #34, interview). Finally, respondents reported that the role-plays helped parents understand the utility of skills and experience parenting from the perspective of the child. Many parents also said that the role-plays increased their sense of confidence in applying the parenting skills at home: "It was helpful because we were practicing it there [during the session], and then we came home and did it with our kids" (parent #11, interview).Challenges delivering nonviolent discipline content Although they expressed confidence in their capacity to deliver most of the content, facilitators articulated a lack of self-efficacy regarding more complex content such as teaching nonviolent discipline techniques to manage difficult child behavior. In order to competently deliver content on nonviolent parenting techniques, facilitators needed to clearly distinguish which approach would be most appropriate for a specific problem behavior. For instance, whereas "ignore" was recommended for negative attention-seeking behaviors such as temper tantrums, the program suggested using a "5-min cool down" as a more effective strategy for aggressive or noncompliant behaviors. However, during the initial implementation to the treatment group, facilitators struggled to make this distinction: It was a really difficult concept. We are not used to these things. I could feel that the way that I delivered it, there was no separating the behaviors for Ignore, Cool Down, and Consequences. I was not as convinced as a facilitator. We could see that [the parents] were listening to us, but they were just not getting it. (facilitator #1, focus group)Nevertheless, facilitators articulated an increased sense of competency with these more complex techniques when implementing the program a second time for the wait-listed group: "[The parents] can feel when you are not sure about something. But now at least we have that confidence of okay, we're gonna do it ... So, even they feel like this is okay. We were more competent" (facilitator #5, focus group). As a result, increased self-efficacy was perceived to be associated with improved delivery of and parental receptivity toward program content. Reinforcing program fidelity and improving facilitator competency Thematic analyses also identified a number of factors that were perceived to have strengthened program implementation. Facilitators described how the program manual assisted them in preparing for and delivering sessions by describing how to lead activities and suggesting possible questions for group discussions. Likewise, the additional training provided during program delivery was viewed as essential for maintaining fidelity to program protocols. As one facilitator explained, "The [practical] sessions helped remind us of everything we learned. They helped us prepare and understand how to deliver the program" (facilitator #2, focus group). The facilitators also described how viewing video recordings of sessions provided an opportunity to improve their delivery: "It was very useful because it was where I saw what I can improve on. It's where we were able to catch mistakes" (facilitator #4, focus group). Finally, the facilitators explained that the weekly supervision sessions significantly contributed to improvements in their facilitation skills. The supervisor's modeling of core program delivery methods and principles was perceived as particularly useful: "The encouragement and praise [during supervision sessions] helped with our confidence ... Also, the way questions were asked to help us to think more about how to deliver the program" (facilitator #1, focus group).Contextualizing content within a cultural framework Respondents reported that the cultural idioms, metaphors, and traditional forms of expression embedded within the program increased the acceptability of core program principles and content. For instance, respondents expressed a strong affinity with the "Building a Rondavel of Support" program model (a "rondavel" is a traditional hut made of mud walls and a thatch roof). This provided a cultural reference point familiar to isiXhosa-speaking families with ties to rural communities: "Whenever we go home to the Eastern Cape, the first place we sleep is in a rondavel. It is the center of the family" (facilitator #6, supervision notes). In addition, participants highlighted the importance of contextualizing the program within culturally relevant forms of interaction. Respondents described how traditional stories and songs helped communicate parenting principles in a culturally resonant manner and promoted a sense of collectivity. Parents also reported that the traditional stories and children's songs provided opportunities for positive parent-child interaction while reinforcing core program principles. Receptivity to strengthening existing parenting skills Qualitative data also showed an overall high degree of receptivity to program content that was perceived to strengthen or refine existing parenting skills. These practices included spending quality time with children, using praise and rewards to encourage positive child behavior, enforcing household rules consistently, giving positive and specific instructions, and applying stress reduction techniques. For example, one mother explained how the program increased child compliance by helping her rephrase how she gave instructions: "Before [the program], I use to say in a negative way, 'do not leave this house before you clean up and make up the room.' But now [my child] listens because I give clear and positive instructions" (parent #70, interview). Many parents were also receptive to the social learning principles on which the program was based. They perceived how changes in their own behavior led to improved relationships with their children, thus underscoring the importance of modeling appropriate behavior toward their children: "Now my children recognize and respect me as their mother because I'm not shouting at them. It's the way I talk with them and spend a lot of time with them" (parent #29, interview).Resistance to introducing new parenting skills While parents were receptive to program content that strengthened existing parenting practices, some parents initially resisted the introduction of new parenting skills that challenged normative parenting behavior. For instance, facilitators perceived that parents were at first reluctant to engage in child-led play: "Parents were not used to playing with their children. It was a new thing to them" (facilitator #2, focus group). However, this resistance often diminished when parents were able to perceive the impact of child-directed play on their children's behavior: "When we spend special time together, I can see that he is very happy, and it helps him to do positive things" (parent #55, interview).Facilitators also reported that some parents resisted using language to communicate about emotions: "In our culture, we don't talk about our feelings ... We do not have those kinds of words, like, excited or mad ... So, how can you help your child to understand their feelings when you cannot deal with your own situation?" (facilitator #6, focus group). Nevertheless, some parents were receptive to communicating about emotions, which was also associated with an increased sense of empathy in their children: "I tell him, 'my child, this is how I feel.' He also notices how I feel and tells me, 'Mama, I can see that you are not happy'" (parent #29, interview).Many respondents also initially resisted the nonviolent discipline techniques emphasized by the program, especially when they contradicted more normative practices such as corporal punishment: "When we learned that we must not beat the kids, in our culture, it's not normal" (parent #11, interview). Parents also articulated that the new skills were often difficult to understand and that they struggled to implement consistently at home. On the other hand, parents explained that their resistance to nonviolent discipline decreased when they were able to personally experience its efficacy in managing problematic child behavior: "I used to spank him just the way I was also raised, which was not working at all. Now, I found that talking to my child works" (parent #11, interview).Discussion and Implications for Practice This study is one of the first of its kind to use a mixed-methods approach to examine the feasibility of a locally developed, evidence-informed parenting program in sub-Saharan Africa. The combination of quantitative and qualitative data allows for an in-depth investigation of the program's applicability when delivered in low-income communities to highly vulnerable families for whom it was developed. The study also offers an opportunity to examine program feasibility from multiple theoretical domains that include participant involvement, implementation, and acceptability. Overall, results indicate that a parenting program derived from evidence-based programs developed in HICs can be delivered with a high level of feasibility in South Africa. These findings are promising, given the vastly different cultural and contextual circumstances that low-income, isiXhosa-speaking parents experience in South Africa in comparison with families from HICs. Program enrollment, attendance, and completion rates were high in spite of the multiple life stressors experienced by participants. Although low socioeconomic status has often been cited as a barrier to involvement (Heinrichs, Bertram, Kuschel, & Hahlweg, 2005), 83.8% of parents participated in at least one session of the program. Moreover, the intervention had a low dropout rate (only 13.8% of those who initially enrolled) in comparison to parenting programs in HICs (Axford, Lehtonen, Kaoukji, Tobin, & Berry, 2012). Similarly, the mean attendance rate of 71.5% (i.e., 8.58 of the 12 sessions) was comparable to trials of parenting programs of similar length in the United Kingdom and United States (Hutchings et al., 2007).In regard to implementation, community facilitators with limited professional qualifications and prior knowledge of evidence-based approaches were able to deliver the parenting program with a high degree of fidelity to manualized protocols. This finding has strong policy implications due to the limited availability of professional clinicians and social workers in LMICs. By engaging paraprofessionals in service delivery, implementers of parenting programs may be able to expand their reach in vulnerable populations. Since facilitators were aware that their performance was being monitored, it is possible that recording videos of sessions influenced the rate of fidelity. This suggests that establishing overt monitoring systems and systematic supervision may be essential to ensure adherence to program protocols. Comparisons between initial treatment and wait-listed groups showed significant differences in program enrollment by parents and program fidelity by community facilitators. The lower enrollment rates in the wait-listed group may have been due to the duration of the waitlist in which participants received the intervention 7 months after the beginning of the study (Handley, Schillinger, & Shiboski, 2011). On the other hand, fidelity data show improvements in program adherence to intervention protocols during delivery to the wait-listed group. Qualitative data also indicate increased competence of community facilitators, as they gained more experience and self-confidence in facilitating content on nonviolent discipline skills. These results suggest that while community-based workers may initially struggle to competently deliver family programs that require advanced facilitation skills, implementation may improve over time, as they gain experience along with consistent training, supervision, and fidelity monitoring. Current findings support literature that highlights the important role of programmatic scaffolding to promote implementation fidelity and participant involvement. For example, preprogram consultations to address practical and psychological concerns of participants have also been shown to improve engagement in other parenting interventions from HICs (Ingoldsby, 2010). Furthermore, many programs have successfully overcome structural barriers to participant involvement by providing transportation, childcare, and food to low-income families in HICs (Axford et al., 2012). Recent research has also demonstrated the utility of using text messages to improve program participation with parents at high risk of child maltreatment in the United States (Murray, Woodruff, Moon, & Finney, 2015). Likewise, the provision of manuals has been shown to increase adherence to treatment protocols and engagement in program content (Elliott & Mihalic, 2004). Finally, the reported benefits of experiential training and supervision of program facilitators are consistent with prior research on their role in maintaining treatment fidelity and improving the quality of delivery (Sethi, Kerns, Sanders, & Ralph, 2014).Results also highlight the potential benefits of integrating social learning theory principles throughout all levels of program implementation. For instance, the program used the same approaches for skills acquisition and utilization to train both facilitators and parents. These skills included the modeling of expected behavior, using collaborative group problem-solving methods, reinforcing positive behavior, and practicing skills in role-plays prior to using them in live situations. Furthermore, these approaches were perceived as particularly effective in addressing challenges with more complex content. Although the facilitators initially articulated low self-efficacy in delivering content on nonviolent discipline, the supervision sessions and experiential training provided them with the opportunity to build confidence and competency by practicing these skills in a supportive environment. The same methods were also perceived as useful in overcoming parental resistance to adopting these practices with their children at home, thus emphasizing the dynamic relationship between the quality of implementation and participant involvement. Findings also confirm the importance of integrating evidence-based approaches within a local cultural context in order to increase program acceptability (Lau, 2006). This mirrors research on Latino populations in the United States showing higher receptivity to programs that embedded content within a culturally relevant context (Parra Cardona et al., 2012). For instance, parents strongly identified with the program's culturally sensitive framework that provided a familiar reference point for understanding the program's structure and content. Likewise, the collaborative learning approach may have resonated with African values of shared responsibility and collectivism as exemplified by the indaba, or communal gathering, traditionally used to solve problems in South Africa (Sulamoyo, 2010).More in-depth adaptation and development may be necessary to adequately engage participants in parenting skills that challenge preexisting norms of parenting behavior. On the one hand, findings demonstrated high engagement in content that reinforced everyday parenting practices in the community (i.e., praise, household rules, instruction giving, and stress reduction). However, facilitators reported low self-efficacy with new practices such as nonviolent discipline techniques, which were subsequently resisted by parents who perceived them as culturally dissonant and too complex. It is possible that additional facilitator training combined with more time allocated to deliver content in sessions is required to address concerns regarding participant engagement and cultural appropriateness. For example, an adaptation of the Incredible Years Teacher Training Program in Jamaica required extra sessions in order to deliver new concepts and skills to beneficiaries with an acceptable degree of proficiency (Baker-Henningham, Scott, Jones, & Walker, 2012).It is important to emphasize that these challenges are not unique to the South African context. For instance, a qualitative study of a comparable parenting program with disadvantaged families in Ireland found similar difficulties in implementing complex content such as time-out (Furlong & McGilloway, 2012). Furthermore, none of the parenting skills were rejected in totality by the parents; rather, they utilized them to varying degrees. This selective engagement of parenting skills corresponds to research in the United States in which Latino immigrant parents experimented with different parenting strategies depending on their personal preferences and family contexts (Holtrop, Parra Cardona, & Forgatch, 2014). Thus, instead of labeling content as either culturally acceptable or inappropriate, future research using in-depth ethnographic or participant observation methods may provide more detailed information regarding the utilization of discrete parenting strategies within the context of local South African cultural norms, values, and practices.This study has several limitations. First, one must caution the generalizability of findings across other contexts and population groups. The sample was limited to predominantly unemployed, low-income, isiXhosa-speaking mothers living in an urban South African setting. It is possible that programs implemented in more rural and isolated communities may encounter additional barriers to participant involvement or increased resistance to new parenting approaches. Moreover, since the study only recruited female caregivers, results may not be generalizable across gender lines. Given the context of family systems in South Africa in which many children do not have regular contact with their fathers, it is not surprising that only women participated in the program (Bray & Brandt, 2007). Other studies have found it necessary to include content specific to fatherhood, employ male facilitators, and deliver sessions to exclusively male groups in order to increase participation of men (Panter-Brick et al., 2014). Second, the study design limited our ability to test the causal effect of discrete program components or delivery methods on program feasibility. While improvements in the delivery of the parenting program were reported between implementation waves, other potential factors may have confounded these effects. Future research may benefit from using an active treatment as a comparison group within the RCT or a factorial experimental design that allows for a more robust component analysis. Third, the reliance of an isiXhosa interpreter during qualitative interviews may have compromised the trustworthiness of thematic analyses (Tsai et al., 2004). We attempted to mitigate this issue by providing rigorous training to the interpreter and instructed her to translate verbatim. Nonetheless, future research would benefit from involving an isiXhosa researcher as a part of the scientific team who could conduct data collection and analyses directly in the home language of participants (Berman & Tyyskä, 2011). Fourth, one must consider potential respondent biases due to social desirability, particularly with client satisfaction surveys and qualitative interviews (Brestan, Jacobs, Rayfield, & Eyberg, 1999). Nevertheless, the study's mixed-methods approach, which allowed for triangulation of qualitative and quantitative data from multiple sources, respondents, and analytic approaches, increases the reliability of results. Fifth, the random selection of enrolled participants for the qualitative interviews included few participants with low attendance rates. A more purposive sampling approach that included participants who had poor attendance or were dropouts may have provided additional perspectives regarding barriers to participation and engagement (Watters & Biernacki, 1989). Finally, the study did not directly examine the experiences and perceptions of children. Future research that includes children as respondents may provide a more child-led perspective on the feasibility and acceptability of the program (Bray et al., 2010).This study nonetheless provides a valuable contribution to existing literature on implementation science and social work practice. By combining dynamics of participant involvement, implementation, and acceptability, we were able to examine multiple dimensions of program feasibility at the same time. While many studies have explored these dimensions separately (Dumas, Nissley-Tsiopinis, & Moreland, 2007), it is equally important to consider how factors associated with feasibility are interlinked and reciprocally influence one another (Koerting et al., 2013). For instance, the challenges implementing the nonviolent discipline components of the program illustrate how the low self-efficacy of facilitators may negatively influence engagement of parents and perceptions of cultural acceptability. On the other hand, the integration of evidence-based parenting principles within a culturally relevant framework was perceived to have positively influenced the quality of implementation and participant involvement.In conclusion, this study's findings have important implications for further research on parenting programs and other family interventions in South Africa and other LMICs. Additional research is necessary to examine predictors of program enrollment, attendance, and engagement. This will enable service providers to identify South African families who are at a higher risk of poor participation and thus may require extra support. Observational assessments of program implementation may also provide a more objective assessment of program fidelity and quality of delivery (Eames et al., 2009). These studies should be nested within larger experimental trials with sufficient statistical power to test the associations between program feasibility and effectiveness in reducing the risk of child maltreatment. Finally, factorial research designs may provide further understanding on the dimensions of program feasibility and how they relate to intervention effectiveness (Collins, Murphy, Nair, & Strecher, 2005). By testing multiple active delivery components in a factorial design, we may be able to optimize parenting programs according to cost effectiveness and scalability as well as their effectiveness in reducing the risk of child maltreatment. This may further reduce the cost of delivery while increasing the reach of parenting programs for vulnerable families living in low-resource contexts throughout LMICs.
Management Accounting Information for Decision-Making and Strategy Execution
ISBN: 978-0137024971
6th Edition
Authors: Anthony A. Atkinson, Robert S. Kaplan, Ella Mae Matsumura, S. Mark Young