PPN Home > Programs that Work > National Guard Youth ChalleNGe Program

Programs that Work

National Guard Youth ChalleNGe Program


Program Info
Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail
Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

 

Program Info

Outcome Areas
Healthy and Safe Children
Children Succeeding in School

Indicators
Students graduating from high school
Children and youth not engaging in violent behavior or displaying serious conduct problems
Children experiencing good physical health

Topic Areas

     Age of Child
       Adolescence (13-18)
     Type of Setting
       High School
       Out of School Time
     Type of Service
       Instructional Support
       Mentoring
       Youth Development
     Type of Outcome Addressed
       Cognitive Development / School Performance
       Juvenile Justice
       Physical Health
       Poverty / Welfare
       Substance Use and Dependence
       Teen Sex / Pregnancy

Evidence Level  (What does this mean?)
Proven

Back to topTop  



Program Overview

The National Guard Youth ChalleNGe Program (ChalleNGe) is an intensive residential program that aims to "reclaim the lives of at-risk youth" who have dropped out of high school and give them the skills and values to succeed as adults. Developed by the National Guard Bureau in the U.S. Department of Defense, ChalleNGe operates in more than half of the states in the country. Over 100,000 young people have completed the program since it was launched in 1993.

The program is 17 months long and divided into three phases: a two-week Pre-ChalleNGe Phase, which is a demanding orientation and assessment period; a 20-week Residential Phase; and a one-year Post-Residential Phase. The participants live at the program site, often a military base, during the first two phases. The curriculum for the Residential Phase focuses on eight core components of positive youth development: leadership/fellowship, responsible citizenship, service to community, life-coping skills, physical fitness, health and hygiene, job skills, and academic excellence. At the end of the Residential Phase, participants work with staff to arrange post-residential placement, such as employment, education, or military service. During the Post-Residential Phase, participants return to their families and receive structured mentoring from qualified mentors identified by themselves within their own community. While the program environment is described as "quasi-military," participation in ChalleNGe is voluntary, and there are no requirements for military service during the program or afterward (Millenky et al., 2011).

Back to topTop  



Program Participants

ChalleNGe serves youths between the ages of 16 and 18 who have dropped out of school and are unemployed, drug-free, and not heavily involved with the justice system. The program is open to both males and females. There are no income-based eligibility criteria.

Back to topTop  



Evaluation Methods

ChalleNGe was evaluated using a randomized controlled research design. Because the number of applicants to ChalleNGe was greater than the number of spaces available, program admittance was decided by a lottery in which eligible applicants were randomly assigned either to an intervention group that was offered admission to ChalleNGe or a control group that was not offered admission. During the study period of 2005-2007, 2,320 applicants across 10 program sites were assigned to the treatment group and 754 applicants were assigned to the control group. Program sites were not selected randomly, but rather were selected for stable staffing and a tendency to receive more applications to participate than spaces available. Among applicants assigned to the ChalleNGe group, 1,575 (67.9 percent) actually enrolled. The analyses included those applicants who did not enroll in or finish the ChalleNGe program.

Baseline data were collected via a two-page questionnaire shortly before the applicants were randomly assigned. These data provide a snapshot of the study participants' demographic information. Three follow-up studies have been conducted:

  • The first wave follow-up surveys were administered an average of nine months after the participants had entered the study, shortly after the treatment group had completed the Residential Phase. The response rate was 85 percent (Bloom et al., 2009).

  • The second wave follow-up surveys were administered approximately 21 months after the participants entered the study. The follow-up surveys included education outcomes (high school enrollment, graduation rate, college attendance), employment outcomes (work status), health outcomes (self-reported overall health and body mass index, or BMI), and social outcomes (arrest rate and conviction rate). The response rate for this follow-up was 79 percent (Millenky et al., 2010).

  • The third wave follow-up surveys were conducted three years after program enrollment, more than one year after the Post-Residential Phase was complete for participants. The response rate for this survey was 78 percent. These surveys assessed the same outcomes that were included in the second wave follow-up (Millenky et al., 2011).

In addition, the research teams gathered information about program implementation and participation through site visits and the ChalleNGe Data Management and Reporting System (DMARS), the national Web-based program tracking system used by all ChalleNGe programs.

Back to topTop  



Key Evaluation Findings

First wave (nine month) findings (Bloom et al., 2009)

Nine months after the participants entered the study, the treatment group (14.6 percent) was significantly more likely than the control group (2.6 percent) to have obtained a high school diploma. Similarly, the treatment group (30.9 percent) was also significantly more likely than the control group (7.5 percent) to have earned a General Education Development certificate (GED). The control group (35.5 percent) was significantly more likely than the treatment group (16.3 percent) to have returned to high school but not to have obtained a diploma or GED.

  • The treatment group (51.2 percent) was significantly more likely than the control group (42.1 percent) to be employed and to be taking college courses (10.9 percent versus 2.7 percent)

  • The treatment group (14.2 percent) was significantly less likely than the control group (20.0 percent) to have been arrested since the random assignment. Similarly, the treatment group (6.5 percent) was also significantly less likely than the control group (11.0 percent) to have been convicted.

  • The treatment group (76.7 percent) was significantly more likely than the control group (68.3 percent) to report their overall health as being "very good" or "excellent." Further, the treatment group (8.4 percent) was significantly less likely than the control group (12.8 percent) to be obese (BMI of 30 or more), though the two groups did not demonstrate any statistically significant differences in the likelihood of being overweight (BMI of 25 to 29).

  • The treatment group (11.0 percent) was significantly more likely than the control group (7.0 percent) to report high self-efficacy and social adjustment scores (defined as one standard deviation above the average score).

Second wave (21-month) findings (Millenky et al., 2010)

  • Twenty-one months subsequent to enrollment in the study, ChalleNGe participants were significantly more likely than their control group counterparts to have earned a high school diploma or GED (60.5 percent versus 36.4 percent). They were also significantly more likely to have earned some college credit (24.8 percent versus 9.6 percent) and to have received vocational training (29.7 versus 22.9 percent)

  • A higher percentage of participants were currently working relative to the control group (55.0 percent versus 50.1 percent), and a greater proportion of participants were currently working full-time (43.7 versus 38.8 percent). Participants were more likely to be earning an hourly wage of $8 to $9.99 (22.6 percent versus 18.2 percent) or to be earning $10 or more per hour (9.1 percent versus 6.2 percent).

  • There was no difference between program and control groups on likelihood of being arrested and charged with a crime in the past year, but the ChalleNGe group was less likely than the control group to have been convicted of a crime (8.9 percent versus 13.2 percent).

  • On self-reported delinquency, ChalleNGe participants were equally as likely as the control group to report any violent incidents in the past year, but the average number of violent incidents was lower in the ChalleNGe group compared with the control group (2.0 versus 2.3 incidents). ChalleNGe participants were less likely to report any property damage incidents (27.2 percent versus 35.1 percent), and, similarly, the average number of property damage incidents was lower in the ChalleNGe group (0.9 versus 1.3 average incidents per participant)

  • There were no significant differences between the groups on health, sexual activity, or drug use outcomes, with the exception of binge drinking. ChalleNGe participants were less likely to report binge drinking in the last 14 days (2.8 percent versus 4.7 percent).

  • Significantly more ChalleNGe than control group participants reported living in their own home or apartment (versus with their parents or a friend; 19.0 percent versus 15.0 percent).

  • The 21-month survey also asked questions related to life-coping and leadership. While most of the questions showed no significant differences between the ChalleNGe and control groups, the ChalleNGe group showed positive impacts relative to the control group on the following measures: "Has learned about organizing time and not putting things off" (ChalleNGe participants ranked this an average of 3.6 out of 4, compared with 3.5 for the control group); "Has learned how to better control temper" (3.5 out of 4 versus 3.4); "Has learned the challenges of being a leader" (3.7 out of 4 compared with 3.5); "Encourages different points of view without worrying about agreement (ChalleNGe participants scored 3.7 out of 5, versus 3.5 out of 5 in the control group).


Third wave (3-year) findings (Millenky et al., 2011)

  • At three years subsequent to enrollment in the program, a range of positive outcomes related to academic performance and employment were observed:

    • ChalleNGe participants were still significantly more likely than their control group counterparts to have earned a high school diploma or GED (71.8 percent versus 55.5 percent). They were also significantly more likely to have earned some college credit (34.9 percent versus 18.8 percent) and to have received a college degree including associate's degrees (0.9 percent versus 0.0 percent).

    • Participants were also more likely than the control group to have been employed in the last 12 months (88.4 percent versus 84.5 percent) and to be currently employed (57.8 percent versus 50.7 percent). Participants had been employed for more total months on average (8.1 months versus 7.2 months) and earned more in the previous year on average ($13,515 versus $11,248, averages include zero amounts for those who do not work).

    • More ChalleNGe than control group participants reported living in their own home or apartment (versus with their parents or a friend, 25.0 percent versus 20.0 percent), but there were no other differences observed across the groups in living arrangements, marital status, or parental status.

    • In questions related to civic engagement, ChalleNGe participants reported more collective civic efficacy than the control group, averaging 3.31 out of 5 compared with 3.23 in the control group. According to study authors, "this aspect of civic engagement was measured based on the respondent's strength of endorsing beliefs that, by working with fellow members of his or her community, he or she could be effective in addressing community problems" (Millenky et al., 2011).


  • The study evaluated a range of health and lifestyle-related outcomes. For several of these outcomes, the ChalleNGe group appears to be doing statistically significantly worse. Study authors did not comment on possible reasons for these differences:

    • While there was no significant difference across the groups in terms of obesity (BMI of 30 or above), the ChalleNGe group was significantly more likely to be overweight than the control group (defined as BMI between 25 and 29.9; 32.1 percent in the ChalleNGe group versus 25.9 percent in the control group). There were no other significant differences in overall physical or mental health.

    • ChalleNGe participants were significantly less likely to report that they "always use birth control" than the control group (49.8 percent versus 57.8 percent) and were significantly more likely to report never using birth control (16.1 percent versus 12.4 percent). There were no significant differences in the proportion of ChalleNGe or control group members who reported being sexually active.

    • Finally, a higher proportion the ChalleNGe group reported ever having used illegal drugs other than alcohol or marijuana (28.2 versus 23.2 percent). Other drug and alcohol measures were not significantly different.


  • There was no difference across the treatment and control groups in enlistment history or current enlistment.

  • There were no differences across the two groups in being charged or convicted of a crime, or self-reported incidents of delinquency.

  • Twelve questions related to leadership and life-coping skills were asked of both groups; there were no significant differences across the groups on responses to any of the questions.

Back to topTop  



Probable Implementers

This program is implemented by states under a Master Cooperative Agreement with the National Guard Bureau.

Back to topTop  



Funding

Perez-Arce et al. (2012) conducted a cost-benefit analysis of ChalleNGe, finding that ChalleNGe program generates increased earnings and other benefits which result in a $2.66 return for every dollar expended on the program. Researchers found that the ChalleNGe program is estimated to increase the present discounted lifetime earnings of ChalleNGe admittees by $45,231 (in 2012 dollars).

Back to topTop  



Implementation Detail

Program Design

  • Although there is considerable room to tailor the program model to local conditions, the basic structure of the ChalleNGe program is the same in all states.

  • Most states operate a single "100-bed" ChalleNGe program, serving a total of about 200 participants per year in two class cycles (starting in January and July).

  • During the two-week Pre-ChalleNGe Phase, candidates are introduced to the program's rules and expectations; learn military bearing, discipline, and teamwork; and begin physical fitness training.

  • The 20-week Residential Phase environment is quasi-military: Participants are called "cadets" and are divided into platoons and squads, live in barracks, have their hair cut short, wear uniforms, and are subject to military-style discipline. The cadets are closely supervised by staff at all times.

  • Cadets who successfully complete the Residential Phase move into the one-year Post-Residential Phase, during which they receive structured mentoring from a mentor who is nominated by the cadet and screened and trained by the program staff.


Staffing

Typically, the ChalleNGe program staff includes a Program Director, who oversees all elements of the program and focuses particularly on external matters, such as marketing, fundraising, and government and community relations; a Program Deputy Director, who oversees all internal affairs, including program management, human resources, and cadet affairs; cadre or team leaders, who directly supervise all daily activities of the cadets; academic instructors, who teach GED courses, responsible citizenship, and job skills; counselors, who conduct psychological counseling and career counseling; and recruitment, placement, and mentoring (RPM) coordinators, who are responsible for recruiting and screening applicants, screening and training mentors, and interacting with cadets and mentors during the Post-Residential Phase. Most administrative staff are military veterans, military retirees, or members of the National Guard and reserves. Academic instructors and counselors are hired through various channels, such as the local school district, community colleges, or direct hiring.

Curriculum

During the Residential Phase, academic instructors teach the GED subject courses, including math, science, writing and language arts, and computer skills. The curricula vary by program site and no particular example is mentioned.

Back to topTop  



Issues to Consider

This program received a "proven" rating. The research, conducted during 2005-2007, was implemented according to rigorous design standards and included a treatment group of 2,320 participants and a comparison group of 754 participants.

The program evaluation used a randomized controlled trial. However, several issues regarding the study sample should be noted. First, the 10 programs—roughly half of the state programs in existence at the time—included in the evaluation study were not chosen randomly. Rather, they were programs that had stable staffing and tended to have more applicants than they could serve, a prerequisite for conducting a random assignment. Thus the results presented in this study do not represent the overall impact of ChalleNGe nationally but rather the impact of selected ChalleNGe programs. Second, applicants who were under age 16 and a half were excluded from random assignment (though not from the program) so that they would not be barred from reapplying for ChalleNGe in the future if they were originally assigned to the control group. As a result, the results reported in the evaluation study pertain only to ChalleNGe participants who were age 16 and a half or older at the time of application. It is possible that younger participants of ChalleNGe have had different experiences from their older counterparts. Finally, in most cases the first wave of follow-up surveys, from which conclusions about the early impacts of ChalleNGe were drawn, were administered only to the first cohort of random assignment for each site despite the fact that random assignments had been conducted for two or more cohorts at most sites.

It should also be noted that, in several areas related to health and lifestyle, the ChalleNGe group appears to perform significantly worse than their control group counterparts. Study authors did not comment on reasons for these differences, which included a higher likelihood of overweight among ChalleNGe participants, a higher proportion of ChalleNGe participants ever having used illegal drugs compared to their control group counterparts, and a lower proportion of ChalleNGe participants reporting that they always use birth control.

Back to topTop  



Example Sites

Arizona, California, Florida, Georgia, Illinois, Michigan, Mississippi, New Mexico, North Carolina, Texas, Virginia, and Wisconsin

Back to topTop  



Contact Information

MDRC Evaluation Study Contact

Dan Bloom and Megan Millenky
MDRC
19th Floor
16 East 34 Street
New York, NY 10016-4326
Phone: 212-532-3200
Email: dan_bloom@mdrc.org

National ChalleNGe Contact

Joe Padilla
Air National Guard
Office of Athletics and Youth Development
Jefferson Plaza 1, Room 2456
1411 Jefferson Davis Highway
Arlington, VA 22202-3231

Back to topTop  



Available Resources

The National Guard Youth ChalleNGe Program website (www.ngycp.org) offers comprehensive information about the program models and implementation experiences for youth, parents, government agencies, and the public.

Back to topTop  



Bibliography

Bloom, Dan, Alissa Gardenhire-Crooks, and Conrad Mandsager,  Reengage High School Dropouts: Early Results of the National Guard Youth ChalleNGe Program Evaluation,  New York: MDRC, 2009. 

Millenky, Megan, Dan Bloom, and Colleen Dillon,  Making the Transition: Interim Results of the National Guard Youth ChalleNGe Evaluation,  New York: MDRC, 2010. 

Millenky, Megan, Dan Bloom, Sara Muller-Ravett, and Joseph Broadus,  Staying on Course: Three-Year Results of the National Guard Youth ChalleNGe Evaluation,  New York: MDRC, 2011. 

Perez-Arce, Francisco, Louay Constant, David S. Loughran, and Lynn A. Karoly,  A Cost-Benefit Analysis of the National Guard Youth ChalleNGe Program,  Santa Monica, CA: RAND Corporation, 2012. 

Back to topTop  



Last Reviewed

February 2012

Back to topTop