AN OUTCOME EVALUATION OF A YOUTH DEVELOPMENT PROGRAMME

INTRODUCTION The Child Care Act of South Africa (1983) requires children who are in children’s homes to leave these places of care once they reach 18 years of age. Research indicates that youths aging out of foster care are more likely to experience homelessness, unemployment, substance abuse and lack of basic healthcare services (Courtney, Drowsy, Ruth, Havelock & Boost, 2005). Atkinson (2008:195) points out that as a result a life marked by traumatic experiences and their lengthy time on the streets, foster care youths often lack the basic skills necessary for independence such as keeping appointments, managing a bank account, finding housing, shopping for groceries, cooking meals, driving a car and taking public transportation.

include the opening of a bank account, obtaining identification documentation and the development and adherence to time-and money-management schedules.

Experiential outdoor learning
Experiential outdoor learning consists of camps and other outdoor activities which are aimed at increasing self-awareness, teamwork and trust.Programme staff organise and administer these activities.

Health awareness
Health awareness workshops cover education in basic nutrition, sex, mental health, and drug and alcohol abuse.Programme staff conduct the health awareness workshops and monitor practical elements such as the cooking activities.A free health assessment is also offered to each of the youths by a men's clinic situated close to the children's home.

Career guidance
In this component youths are exposed to appropriate career options.Programme staff assist the youths to identify career paths and relevant education and skill requirements.Programme staff offer financial aid and mentorship where youths choose to enrol for career-enhancing courses.Visits to career fairs are organised and the youths learn how to write informative curricula vitae.

Relationship building
Under the guidance of programme staff and where feasible youths are assisted to re-establish family ties through home visits and other family-based outings.Youths are also guided to participate in community activities at least once a quarter.

Internships
Prior to the youths leaving the children's home, programme staff identify suitable skills training and education for the youths.Once the youths have left care and are adequately educated and trained, programme staff identify host companies for part-time internship programmes.After placement, programme staff monitor internship progress and support the youths to complete the internship.If possible, companies are encouraged to provide permanent employment to the interns.

Mentorships
Programme staff, former graduates and other role models within the community are identified to provide emotional and practical support to the youths before they age out of care.Ongoing support is also provided after the youths have graduated from the children's home.Mentoring discussions take place in group or individual sessions and typically deal with peer pressure, overcoming the frustration of not being able to find employment, uncertainty about their future and low self-esteem.

Accommodation
Youths unable to return to their families are assisted by programme staff to find suitable accommodation after aging out of care.Youths are either provided with funds to rent a room or are placed in a hostel where they will stay until they are gainfully employed.In some instances, the youths are provided with funds to build a house in an informal settlement.
In summary, these activities of this youth development programme focus on life-skills development, skills building for employment and preparation for independent living.

Programme theory
The first task of an evaluation is to understand the programme's theory of change (also called the programme theory).According to Bickman (1987:5), a programme theory is "the construction of a plausible and sensible model of how a programme is supposed to work".Donaldson (2001:22) defines programme theory as "the process through which programme components are presumed to affect outcomes and the conditions under which these processes are believed to operate".In the light of these definitions, a programme's theory serves to describe the link between the services provided by the programme and how these services will influence change in the target participants.
The youth development programme's theory as provided by the programme stakeholders is presented in Figure 1.

Youth Development Programme
Work experience via internships

Find suitable accommodation
Gainfully employed youths who are able to live healthy independent lives

Medium-term Outcomes Long-term Outcomes
This theory purports that if youths at this children's home are exposed to the activities of the programme, they may gain work experience by means of internships and find suitable accommodation.Thereafter they may find employment and lead healthy and independent lives.

Plausibility of programme theory
In order to test the plausibility of the client's programme theory, the literature and evaluations focusing on children aging out of foster care are summarised below.
According to Avery (2010:183-212), young people who age out of foster care to independent living are more likely to experience homelessness, unemployment, unplanned pregnancy, legal system involvement, substance abuse and lack of basic health care services.Youth development programmes may alleviate these problems and help youths to develop survival competencies (Lerner, Fischer & Weinberg, 2000;Roth & Brooks-Gunn, 2003).Programmes containing strategies which include systematic skills assessment, training for independent living and developing connections with birth families and the community may assist youths aging out of care to cope with independent living (Massinga & Pecora, 2008).
However, programme staff on youth development programmes need to be aware of the following intractable problems often encountered by participants who age out of care:  homelessness and a lack of suitable housing (Atkinson, 2008;Lenz-Rashid, 2006;Mamelani Projects Annual Report, 2009);  failure to secure and maintain employment (Atkinson, 2008;Courtney & Dworsky, 2006;Lenz-Rashid, 2006);  lack of educational achievement (Atkinson, 2008;Collins, 2004;Gerber & Dicker, 2006); and  lack of access to healthcare (Atkinson, 2008;Collins, 2004;Courtney & Dworsky, 2006;Gerber & Dicker, 2006;Lenz-Rashid, 2006) When testing the plausibility of a programme theory, it is useful to review previous evaluations of youth development programmes.These evaluations, although mainly North American, may provide information regarding programme activities and the effectiveness of the programme.
Life-skills training plays an important part in shaping the youths' progression after they have transitioned out of foster care.Survival Skills for Youth (SSY) is a programme that is designed specifically for at-risk youths who are between 14 and 21 years of age (Thurston, 2002).Atrisk youths are defined as youths who are not in school or at risk of school failure or drop out.The programme consists of ten sessions and its key activities include goal setting and developing life plans; leading a healthy lifestyle (health habits, nutrition, dieting, emotional health); employment (finding a job, interviewing, networking, job lifeline, job exploration, selfassessment, keeping a job) and life-skills development (communication with others, managing money, survival skills, reflection and assessment, and group support).Thurston (2002) evaluated the programme by using pre and post-test measures for each activity.He found that all youths who participated in the programme activities showed improved life skills.
Going for the Goal (GOAL) is a life-skills development programme that is designed to give adolescents a sense of personal control that helps them to develop a positive outlook on their future (Forneris, Danish & Scott, 2007).The intervention is generally taught in schools by peer educators (commonly two older students to ten to twelve younger students).The GOAL programme focuses on the development of three skills, namely goal setting, problem solving, and the ability to seek and obtain social support.An evaluation of GOAL (Forneris et al., 2007) indicated that those who completed GOAL understood how to set life goals, worked towards attainment of these goals, identified specific problem-solving strategies and knew where to seek support when encountering problems which they could not solve.
Adolescent Decision Making for the Positive Youth Development Collaborative (ADM-PYDC) is a structured after-school programme for youths with low levels of parental monitoring.Programme activities include coping with stress by means of stress-reduction strategies, effective decision making, knowledge of tobacco, alcohol and other drugs, and applying the decision-making process to one's life through identifying positive personal attributes, dealing with job and school stressors, setting positive goals for healthy living, and enhancing one's social networks and resources (Tebes, Feinn, Vanderploeg, Chinman, Shepard, Brabham, Genovese & Connell, 2007).Results of an evaluation (Tebes et al., 2007) indicated that an ADM-PYD intervention which included an evidence-based, substance use prevention component adapted for an urban after-school setting was effective in preventing adolescent substance use.Adolescents participating in the intervention were significantly more likely to view drugs as harmful at programme exit (about seven months after enrolment), and demonstrated reduced incidence of past-thirty-day use of alcohol, marijuana or other drugs, as well as any drug use one year after programme enrolment.
Future Cents is a life-skills programme which helps at-risk youths to secure employment (Matsuba, Elder, Petrucci & Marleau, 2007).Participants take up a part-time, paid position which serves the purpose of transitioning them from unemployment to full-time, paid employment.An evaluation by Matsuba et al. (2007) utilised pre-and post-tests measuring life satisfaction, loneliness and self-esteem of the participants.All participants showed improvements on these measures of psychological wellbeing after the intervention.In addition, 88% of the youths who completed the programme found employment or were enrolled in educational institutions 12 weeks after the programme (Matsuba et al., 2007).
The literature suggests that youths who participate in community service activities have a stronger self-image and value themselves more highly than adolescents who do not participate (Lakin & Mahoney, 2006).Lakin and Mohoney (2006) evaluated a community service programme which was included as part of the academic curriculum for sixth grade students.The programme consisted of three main components, namely skills building, planning and action.Each component focused on providing students with a sense of empowerment and community while doing community service.The participants and a control group completed pre-and post-tests measuring self-efficacy, a sense of responsibility, intent to be involved in future community action and empathy (Lakin & Mahoney, 2006).Results of the evaluation indicated that youth community service programmes with programme activities which engendered a sense of empowerment and community had the potential to promote positive youth development.
Mentoring relationships, or consistent connections between caring non-parent adults and at-risk youths, may also foster a sense of social connectedness (Munson & McMillen, 2009).An evaluation of a mentoring programme suggested that most of the older youths in foster care could identify non-kin supports in their lives and that these supportive relationships were associated with some positive psychological outcomes (Munson & McMillen, 2009).The evaluation's findings also suggested that mentoring could be an important ancillary intervention that may keep young adults feeling connected to society and other helpful adults.
From the literature and previous evaluations it would seem as if training in life skills, preparation for future employment and building quality relationships with mentors are effective programme activities for youth development programmes of at-risk youths.Finding appropriate accommodation seems to remain a problem for these programmes.The current programme under evaluation contains all these activities and it could be concluded that its programme theory is sound.However, the evaluators would like to introduce three additions in order to strengthen the client's programme theory.
Firstly, educational outcomes should be clarified for these youths who have missed formal schooling because of a life on the streets, drug abuse and trauma.In the South African context further education and training (rather than academic schooling) may be a better option.
Secondly, although the client's programme theory was captured earlier on, the evaluators suggest that the outcomes in this programme theory be disaggregated (Kusek & Rist, 2004) in order to monitor progress better.
Thirdly, it is suggested that long-term outcomes be added to the programme theory.Long-term outcomes are those outcomes that apply after the youths have transitioned out of care and have left the programme.This addition is important as the programme purports to prepare youths for life after residential care.
The revised programme theory is presented in Figure 2.

FIGURE 2 REVISED PROGRAMME THEORY Evaluation Questions
This evaluation focused on whether the youth development programme prepared the beneficiaries for life after the programme.According to Rossi, Lipsey and Freeman (2004:17), the ultimate goal of a social programme is to bring about change in the lives of its beneficiaries.This change is called the outcome of the programme and is assessed by means of an outcome evaluation (Chen, 2005;Rossi et al., 2004;Weiss, 1998).Unlike a process evaluation which focuses on how the programme works, an outcome evaluation assesses whether a programme works (Rossi et al., 2004).Meaningful information about the outcome of a programme can be obtained by utilising specific evaluation questions about short-, medium-and long-term programme outcomes (Rossi et al., 2004).
Following the advice of the authors mentioned here, the following short-, medium-and longterm evaluation questions were formulated to provide a framework for this evaluation:  1) While still in residential care, did youths on the 2007-2009 programmes acquire improved: -life skills -skills to live healthy lives -career-identification skills -relationship-building skills? 2) In the first six months (a time period chosen because of time constraints for the evaluation) after these youths had left residential care, did they: -gain work experience via an internship -attend mentorship sessions -have access to suitable accommodation?

Long-term Outcomes
Youth Development Programme Improved life-skills

Improved health
Improved social relationships

Identify career options
Access educational support

Have healthy lifestyle
Contact with support structures

Work experience via internship
Further education

Maintain healthy lifestyle
Use support structures  3) In 2010, were these youths: -living independently -gainfully employed -maintaining a healthy life style -furthering their education and -using support structures?

METHOD
The evaluation questions formulated above will be used to present the sub-sections of the method.

Data providers
The Mamelani Projects Annual Reports of 2006-2009 were the programme records used to answer the first and second evaluation questions.
The programme director and programme manager assisted with queries relating to these programme records.
As regards the short-and medium-term outcomes, the sample of beneficiaries showed no treatment attrition (Shadish, Cook & Campbell, 2002).A total of 30 male programme beneficiaries completed the 2006-2009 programmes.Their ages ranged between 18-25 years.
They had low levels of education: 25 had primary school education, five had high school education and none had completed the National Senior Certificate.
Table 2 presents the data providers and method of data collection used to answer the third evaluation question.The first evaluator met with the programme director on three separate occasions and with two of the programme staff on four separate occasions.Unstructured interviews were used during these occasions to elicit missing data and to verify data.

TABLE 2 DATA PROVIDERS AND METHODS
The programme director indicated that of the original 30 beneficiaries who completed the programme, two were in jail and one had died in a fatal stabbing in 2006.Four beneficiaries had relocated to the Eastern Cape and no contact details were available for them.Another nine beneficiaries did not attend the mentorship session during which the data were collected and did not respond to attempts to contact them.As regards the long-term outcomes, the final, contactable sample consisted of 14 beneficiaries who were available during a specific mentoring session.
In captive samples (e.g.youths in foster care) or where programme documents are used to obtain outcome measures, treatment attrition can be overcome (Shadish et al., 2002).However, when at-risk participants (e.g.prisoners, street people, youths in foster care, etc.) leave a programme, measurement attrition often occurs (Shadish et al., 2002).This pattern was evident in this outcome evaluation: no treatment attrition but relatively high measurement attrition in the long-term outcome assessment.

Materials and procedure
The programme director agreed to the outcome evaluation and allowed the first evaluator access to specific programme documents.The University of Cape Town's Commerce Faculty Ethics Committee approved the evaluation proposal.
An ideal outcomes map, depicting short-, medium-and long-term outcomes and indicators, was developed by the evaluators.This map is represented in Table 3.The long-term outcomes on the outcome map were measured by means of a standard questionnaire, the Quality of Life Questionnaire developed by Bigelow, Gareau and Young (1991).The original questionnaire contained 12 sub-scales.However, for this evaluation only eight sub-scales pertaining to making decisions, getting along with others, getting along with family, sharing problems with others, handling work, spending leisure time, alcohol use and drug use were used.These eight sub-scales contained 42 items.Nine additional items were added to the questionnaire by the evaluators.The items developed by the evaluators are described in Table 4. Age in years 4 Options The final questionnaire, which contained 51 items, is available from the evaluators.
The first evaluator contacted the programme manager to assist with the distribution of the questionnaires to youths with whom they still had contact.A week before being asked to answer the questionnaire, the programme manager read a cover letter which explained the purpose of the evaluation to each one of the youths.The letter also stipulated that the information supplied by the youths would be kept anonymous.A week after being told about the questionnaire and after their mentorship session, the programme manager assisted youths who elected to answer the questionnaire.Each questionnaire took 30 to 35 minutes to complete.Completed questionnaires were collected and placed in a box.The box was sealed to ensure that no one other than the evaluators had access to the completed questionnaires.
The questionnaires were completed during July-August 2010.All the beneficiaries of the 2006-2009 programmes had transitioned out of care at this time, albeit some earlier than others.The completed questionnaires were collected at the end of August 2010.Data from these questionnaires were then captured in the statistical programme SPSS.

RESULTS AND DISCUSSION
The results of this evaluation will be reported and discussed according to the evaluation questions.
Evaluation question 1: While still in residential care, did youths on the 2006-2009 youth development programmes acquire improved life skills, healthy living skills, career identification skills and relationship-building skills?
Table 5 presents the programme beneficiaries' short-term outcomes.Each of the short-term outcomes will be discussed in more detail.

Improved life skills
While all 30 youths completed the life-skills training, none of them submitted a life-skills checklist (the checklist was developed only during late 2009).The checklist will be used to monitor the programme's beneficiaries for the 2010 programme and beyond (Mamelani Projects Annual Report, 2009).Life skills are the basic skills necessary for successful independent living (Massinga & Pecora, 2008) and therefore the current evaluators would like to encourage programme staff to use the checklist and to assess the development of life skills on a regular basis.In addition, such a monitoring and assessment process would identify youths who are lagging behind in their life-skills development and who would need further intensive life-skills training (Thurston, 2002).

Health checks
No records were available to indicate whether the youths enrolled for the 2006-2009 programmes had visited a health clinic or doctor prior to their aging out of care.With research (Collins, 2004) indicating that children in foster care are more likely than their peers to have health problems, it is important for each of the beneficiaries to obtain a health record prior to aging out of care, as this may assist in their treatment if they were to get sick after leaving the foster home.In 2009 programme staff suggested that youths enrolled for future programmes be required to visit a clinic for a general check-up and submit a copy of the non-confidential portion of their respective health cards prior to their transitioning out of foster care (Mamelani Projects Annual Report, 2009).

Career identification skills
Twenty-three of the programme's 30 beneficiaries had developed well-written curricula vitae (CVs) prior to aging out of care.This is encouraging and may assist in finding employment.However, only four of the programme's 30 beneficiaries attended a career fair.The four beneficiaries were all enrolled in the 2006 programme.Attending career fairs is an informal way of exposing youths to job-seeking skills.Research has shown that job-seeking skills are generally undeveloped for most at-risk foster care youths prior to their aging out of care (Atkinson, 2008;Courtney & Dworsky, 2006).The evaluators suggest that the programme staff budget for this intervention, so that it can take place regularly in future.

Relationship-building skills
According to available records, 20 of the programme's 30 beneficiaries visited their family homes prior to aging out of care (Mamelani Projects Annual Reports, 2006-2009).However, no records exist regarding the quality of these visits.Development of strong relationships with immediate and extended family has been identified as one of the most important needs for youths in foster care (Atkinson, 2008).The evaluators suggest that the staff in the children's home, who have a statutory responsibility for home visits, try to refine assessment of this outcome.
Building relationships by getting involved in community-based activities promotes a sense of empowerment (Lakin & Mahoney, 2006).In terms of community involvement, 13 beneficiaries out of 30 took part in community activities during the 2006-2009 periods prior to ageing out of care.The evaluators suggest that programme staff try to engage more youths in these activities in future.
No conclusion can be drawn about the effect of the programme on improved life-skills, healthy living skills and some of the career identification skills, as programme records of these activities did not exist.There is evidence that the programme succeeded in assisting youths to develop well-crafted CVs.There is also evidence that beneficiaries engaged in relationshipskills-building activities, namely visits to family and involvement in community service.However, no records exist for the quality of these relationship-skills-building activities.
Evaluation question 2: In the first six months after these youths had left foster care, did they gain work experience via an internship, attend mentoring sessions and have access to formal housing?
Table 6 presents the programme beneficiaries' medium-term outcomes.The results for evaluation question 2 will be discussed in terms of the relevant medium-term outcomes, namely internships, mentoring sessions and accommodation.

Internships
Between 2006 and 2009 12 of the programme's 30 beneficiaries were assisted to enrol for internships after aging out of care.The low internship enrolment number could be attributed to three factors: firstly, some of the programme's beneficiaries were still at school after they had left care (Mamelani Projects Annual Report, 2007), secondly, some of the youths dropped out of school while in residential care and might not have had the required educational levels for an internship and thirdly, the internship programme was only implemented in 2009.The evaluators strongly suggest that serious effort be invested in improving the programme beneficiaries' educational outcomes to enhance their chances of obtaining an internship position prior to their aging out of care.
Nine of the 12 youths recruited for internships completed their internships.This is a positive outcome for the programme, as completion of internships helps youths to develop work experience and seek out future employment opportunities with the organisation where the internship is based (Atkinson, 2008;Matsuba et al., 2007).The evaluators would like to suggest that the programme staff forge strong relationships with the organisations that offer internships and find more organisations that are willing to make this offer, as more internships could contribute to an increase in the programme's employment outcomes after the youths have transitioned out of foster care.

Attending mentoring sessions
No programme records were available for attendance of one-on-one mentoring sessions.

Access to accommodation
Between 2006 and 2009 11 youths secured accommodation within either a room at a hostel or a rented room in a flat.Pendlebury, Lake and Smith (2009:98) would call this type of accommodation 'habitable housing'.It is indeed a positive outcome that about a third of the beneficiaries lived in such housing.However, 10 youths were assisted to build informal houses according to Pendlebury et al., (2009:98) informal housing usually consists of shacks in backyards or informal settlements, caravans or tents.Another nine youths returned to their family homes.Moving to informal housing and going back to a home from which the youths were removed do not constitute positive accommodation outcomes.It is vital that youths find suitable accommodation when they leave the children's home.However, in a country where such accommodation is acutely scarce, this may be very difficult to attain.This is supported by research which has indicated that finding suitable housing for at-risk youths may be one of the most difficult youth development programme outcomes to attain (Lenz-Rashid, 2006).
Can we deduce from these results that the programme works in terms of its medium-term outcomes?First we have to clarify what is a 'good enough' outcome for at-risk youths.Rossi et al. (2004:228) suggested that one way of interpreting outcome data could be to ask the question: are the programme beneficiaries better off post-programme than they were preprogramme?The evaluators would like to add further context to this question and ask: given their poor educational levels and at-risk status, can we set the 'success threshold' (Rossi et al., 2004:228) for these beneficiaries at 30%?In other words, if we have evidence that 30% of youths are now better off than they were before the programme, then we could assert that the programme worked for them.Therefore, the programme worked for the 12 beneficiaries who secured an internship and the 11 beneficiaries who lived in habitable housing.However, the evaluators would like to suggest the following two improvements for a stronger programme effect: the timing of doing an internship needs some refinement, as the beneficiaries who are still at school cannot take up an internship.Also, service delivery in terms of habitable housing could be improved.
In conclusion, it would seem as if the programme is struggling to attain its medium-term outcomes for most of the beneficiaries, mainly because internships and accommodation are difficult outcomes to achieve.In order to improve the effect of the programme in this regard, the evaluators http://socialwork.journals.ac.za/ : http://dx.doi.org/10.15270/48-2-97suggest that programme staff extend and strengthen their relationships with organisations that provide internships and contact local and regional housing authorities in order to explore the feasibility of creating waiting lists for formal housing for youths who age out of care.
Evaluation question 3: In 2010, are these youths: living independently, gainfully employed, maintaining a healthy life style, furthering their education and using support structures?
In this section the programme beneficiaries' long-term outcomes are presented and discussed.Only 14 youths completed the questionnaire measuring these outcomes and therefore the small sample was not divided by year of programme, but treated as a single sample.

Living independently
In terms of accommodation, only four respondents were living in habitable housing (one in a room in a hostel and three in a room in a house).Nine respondents were living in an informal house in 2010.Research (Atkinson, 2008;Lenz-Rashid, 2006) has shown that finding suitable accommodation for participants transitioning out of foster care remains a major challenge for most youth development programmes.It was no different for the programme under evaluation.Scarcity of habitable housing restricted most of the youths to either return to their family homes or build an informal house after they had transitioned out of care.
Despite poor living conditions, mean scores on the accommodation satisfaction sub-scale for this sample tended towards 'satisfied' on the 4-point Likert scale.It would seem as if these youths had relatively low expectations regarding housing quality.Or perhaps they just exhibited realistic attitudes in the light of unemployment and housing scarcity.

Employment
Five respondents were employed in 2010 (and nine were unemployed).These findings suggest that youths may not have acquired the required skills or education to find a good job after they transitioned out of care.Atkinson (2009:209) pointed out that sufficient education, which atrisk youths often lack, may be the most important requirement for finding employment.In the light of research findings and the low educational levels of the current sample, it is suggested that the programme prioritises the improvement of education outcomes for these youths after their aging out of care.
The mean scores for the employment satisfaction sub-scale for the five employed respondents were located on the higher end of the 4-point scale indicating that respondents were generally satisfied with their current jobs in terms of the work load and interaction with work peers.Although all of the employed respondents were satisfied with their working conditions, three of the five respondents felt that their current incomes were inadequate to cater for their present needs.This suggests that these poorly skilled respondents were being paid a low salary prior to their aging out of care.Research has shown that earning a low salary exposes the respondents to the risk of obtaining extra money through illegal means such as drug dealing (Atkinson, 2008:193).Four of the five employed respondents also indicated that they were quite worried about their future incomes.This may suggest that respondents were aware of how poor their chances were of finding higher-income employment with their current employment skills.applicable to the 2006-2008 intakes.Research (Atkinson, 2008) has shown that community involvement fosters a sense of empowerment and community, and provides a social support structure other than the youths' immediate family.Furthermore, the 2009 youths were able to retain the social networks they established while in care.This meant that they did volunteer work in communities which were familiar to them after they had aged out of care.
In summary, has the programme attained its long-term outcomes for the sample?In order to answer this question, we shall employ our 'good enough' standard again.However, this time the sample is 14 (and not 30).We have evidence from 14 beneficiaries and cannot make assumptions regarding the evidence the other 16 would have supplied.Therefore the 30% standard is calculated on the basis of 14 respondents and should be interpreted with care.Our conclusion is that the programme has been successful for these 14 beneficiaries in terms of employment, responsible alcohol use, resistance to drug use, safe sex, attending mentoring sessions and involvement in community activities (this latter outcome for the 2009 cohort only).However, the programme has not been successful in attaining its habitable accommodation and education outcomes.It could very well be that for this poorly educated sample living in a developing country, the education and housing standards were overambitious.In conclusion, the programme was relatively successful in attaining some very difficult long-term outcomes for at least 30% of its beneficiaries.

LIMITATIONS OF THE EVALUATION
The lack of monitoring data in the programme records precluded firm conclusions regarding the short-and medium-term outcomes of the programme.The small sample size for evaluation question 3 (14 out of 30 beneficiaries) meant that the evaluators had to exercise care when interpreting the effect of the programme for long-term outcomes.When collecting monitoring data in future, the evaluators suggest that pre-programme data be collected to be used as baseline data.The post-programme data could then be compared to the pre-programme data and a firmer conclusion could be reached when making a judgement whether beneficiaries were better off after the programme.
Given the poor educational background of this particular sample, one could question the suitability of the questionnaire that was used.In some instances (e.g.type of housing, community service/involvement) it became clear that the respondents did not always understand the terminology used.The evaluators recommend that a user-friendly questionnaire aimed at respondents with low functional literacy be used in future.
Self-report data were used to measure the long-term outcomes of the programmes.As some of these outcomes dealt with sensitive issues (alcohol and drug abuse, risky sexual behaviour, etc.) the results of this section may reflect some social desirability bias.This evaluation did not focus on the quality of the programme outputs (i.e.adequacy of services provided by programme staff).It is suggested that a follow-up evaluation should focus on programme implementation and specifically on the quality of service delivery.
While one would have preferred to see higher employment rates of these youths, two factors should be kept in mind here, namely their poor educational levels and the depressed state of the employment market at the time of the evaluation.Within this context, the long-term outcome of employment (five out of 14 respondents) could be assessed as positive.
Finally, a best practice, plausible programme theory may not always be suitable for poor children in a developing country.The educational and housing outcomes specified in this evaluation would seem to be over-ambitious.

EVALUATION CONTRIBUTION
Although several evaluations of youth development interventions have been conducted within the developed world, the same cannot be said for South Africa.The current outcome evaluation contributes to the literature on at-risk youths within a South African context.It provides a realistic picture of how difficult it is to make a youth development programme for at-risk youths work.It also provides a standard of what is 'good enough' in terms of long-term outcomes for such programmes.
In order to overcome the lack of monitoring data for short-and medium-term evaluations, the evaluators designed an outcome map which the programme staff could use in future.As data collection using the outcome map becomes more reliable, the map could be the basis of a predictive theory regarding the programme activities which contribute most to the programme's long-term outcomes.This evaluation focused on the intake of 2006-2009.These years could be described as the first period of development of the programme.By 2011, the programme had changed significantly in terms of increased staffing and budget.This also meant that attention could be focused on monitoring and evaluation.The main contribution of this evaluation is the manner in which the programme staff utilised it.The programme manager reported in 2011 that the systematic nature of the outcome map and the introduction of individual development plans assisted them in implementing a comprehensive monitoring system.In 2011 the programme staff also started deliberations with the Department of Social Development to develop an aftercare programme which could be used more widely.The staff indicated that the evaluation provided them with systematic information to engage in these deliberations.

CONCLUSION
Outcome evaluations are critical in securing additional funds for youth development programmes and providing an evidence base for programme activities for future programmes (Shannon, Walker & Blevins, 2009).Evidence from the current evaluation has highlighted how difficult it is to attain ambitious accommodation and employment outcomes for youths aging out of care.Improvement of educational outcomes for these youths has also been indicated as a priority programme activity.It is recommended that a follow-up evaluation should focus on refining and introducing programme activities related to education, housing and employmentthose intractable problems which confront every youth development programme.