A PROCESS EVALUATION OF A PROGRAMME FOR STREET PEOPLE

The challenges which homeless people face every day are numerous and are not likely to be overcome with any ease. These challenges include hunger, malnutrition, disease, crime, abuse, violence, mental illness, exposure to the elements and difficulty in accessing government services. The homeless may also experience isolation and difficulties integrating into mainstream society. The severity of these challenges in combination with the lack of data on South Africa’s homeless suggests that permanent solutions to homelessness will be challenging.


INTRODUCTION
The plight of the homeless in South Africa has been graphically described by Cross and Seager (2010a:18) as "the proverbial skeletons at the feast, the excluded poorest who enter unobserved and stand by gaunt and starved, terrifying to the invited guests but deprived of any capacity to join the party." The challenges which homeless people face every day are numerous and are not likely to be overcome with any ease.These challenges include hunger, malnutrition, disease, crime, abuse, violence, mental illness, exposure to the elements and difficulty in accessing government services.The homeless may also experience isolation and difficulties integrating into mainstream society.The severity of these challenges in combination with the lack of data on South Africa's homeless suggests that permanent solutions to homelessness will be challenging.Seager and Tamasane (2010) conducted a survey of the health problems of the street homeless in South Africa.The results of the survey showed that one out of every five homeless people had experienced assault or had been injured in some way within the previous six months.Twelve per cent of homeless adults and almost 7% of street children under the age of 18 could be considered to be disabled.More than half of the adults had experienced symptoms of depression during the previous 30 days.It was also noted that street people are at risk for contracting the human immunodeficiency virus (HIV) or other sexually transmitted diseases as "they tend to be sexually active from a young age, have more partners, are less likely to use condoms, tend to use alcohol and drugs more frequently than other groups (which may impair their decision making about safe sex), and they are vulnerable to rape and 'survival sex' (sex in exchange for food and other essentials)" (Seager & Tamasane, 2010:76).
These challenges for the homeless are not restricted to South Africa.Results of a needs assessment (Herman, Streuning & Barros, 1994) among the homeless in New York City showed that the most common need was finding a place to live, while the next most pressing need was for employment or income.Other needs included the resolution of health problems and help in obtaining public assistance.The authors argue that the multiple needs of the homeless suggest that any programme which seeks to help them must be able to grant access to multiple services.
In an evaluation of a multi-site programme in the United States, Rog (1999) found that homeless families had multiple needs, including access to mental and physical health services, substance abuse treatment, and education and training.She also noted that domestic violence and poor mental health were the most prominent challenges which homeless women faced.
Numerous organisations have risen to the challenge to assist homeless people to leave a life on the street and integrate into mainstream society.These organisations have to address the many difficulties which the homeless face on a daily basis.
The implementing organisation of the programme under evaluation is one such an organisation which faces these challenges in its attempts to assist clients to move from the streets into mainstream society.This article aims to provide a process evaluation of its street person's programme.

Programme description
The following programme description was obtained from the implementing organisation's website, programme documents and discussions with the programme manager.
The implementing organisation is a non-governmental (NGO), faith-based organisation which is located in Cape Town, South Africa.It receives funding from its umbrella organisation and the Western Cape Provincial Government.The aim of the NGO's street person's programme is to help clients move into permanent accommodation.In order to achieve this, it is believed that the clients need to achieve three objectives.First, they have to find employment; secondly, they need to find temporary or long-term shelter; and thirdly, they need to assess the reason why they are living on the street and if possible, to find a permanent solution to this state.
Underpinning these three objectives are additional services offered by the NGO.These additional services do not have a direct causal relationship with the goal of getting clients off the street, but without these services it is likely that the programme would be unsuccessful.These five services entail the provision of food, hygiene, primary health care and emotional support, and assisting clients to access relevant state services.These services, plus a drug rehabilitation service, are offered from the premises of a single centre.The centre does not offer sleeping facilities.
The target population of the programme is the street homeless over the age of 16 years.Four categories of service are offered to the target population: provision for physical needs; provision for social needs; skills training and the creation of employment; and spiritual programmes.However, these services are resource-dependent and as such are not always readily available.Furthermore, the NGO generally offers services to clients only for three months in order to avoid creating dependence.These services are described in more detail below.

Provision for physical needs
The NGO attempts to address the physical needs of their clients by providing storage lockers, showers, toilets and laundry facilities.Food parcels are handed out and clients can cook food on the NGO's premises.
Clients are offered primary health care including oversight of the administration of medication for pulmonary tuberculosis (TB) and HIV.Clients who attend out-patient rehabilitation treatment are given letters which they may present to a nearby shelter to secure a bed.
Limited funding is provided to clients who wish to travel back to their original home.A special fund was created for providing clients with building materials for erecting shacks.

Provision for social needs
Clients are offered a voluntary psychosocial evaluation in order to determine their needs.A social worker administers this evaluation and also offers other services including the running of support groups, helping clients attend hospital visits, and reuniting clients with their families when appropriate.
The NGO tries to care for their clients' social needs by providing an outpatient rehabilitation programme for substance abusers.This programme is run by a health professional.Personal counselling is also offered by a member of the programme staff.
The NGO also offers advice on how to obtain identity documents, pensions, grants and how to access social services.

Skills training and employment
The NGO offers a six-month employment contract as a street sweeper with a City Improvement District (CID).The client has to volunteer consistently to do cleaning jobs and other tasks at the NGO's centre in order to be placed on the waiting list for this job.
Clients are also assisted to search for other jobs by helping them prepare résumés.

Provision for spiritual needs
The NGO has a second goal apart from helping homeless people live permanently off the street and this aim is to convert clients to Christianity.In order to receive the midday meal, clients must attend a religious service.This service consists of prayer, songs and a sermon.

Programme theory
The programme activities described above can be presented as a programme theory.Such a theory serves to "explain why the programme does what it does and provides the rationale for expecting that doing so will achieve the desired results" (Rossi, Lipsey & Freeman, 2004:135).If the assumptions underlying the programme theory do not represent a credible approach to bringing about the intended results, it is unlikely that the programme will be effective (Rossi et al., 2004).
The theory underlying the NGO's programme to reduce homelessness is straightforward: if a client is given access to accommodation and employment, and is able to address the causes of his or her homelessness, then the client will leave the streets and reintegrate into society.The provision of accommodation, employment and psychosocial support is accompanied by the provision of food and basic hygiene.The theory behind the NGO's aim to convert people to Christianity is that if a person is told the gospel message, then he or she might convert.The NGO's programme theory was elicited by means of interviews with, and e-mails from, the programme manager and is depicted in Figure 1.Rossi et al. (2004:102) state that "in evaluating a social program, it is essential to ask whether it addresses a significant social need in a plausible way and does so in a manner that is responsive to the circumstances of those in need."It is doubtful that not being an adherent of Christianity constitutes a significant social need.Likewise, it is not clear how conversion is an action responsive to the circumstances of those in need.

Evaluating spiritual programmes
Another problem with evaluating a programme which attempts to convert people to Christianity is that there is no way of objectively measuring if the outcome has been achieved.As a result of these two problems, this publication will not focus on the spiritual activities of the programme.

Evaluation questions
A programme theory provides the evaluator with a basis for formulating useful evaluation questions.The current evaluation is a process evaluation.In other words, it will attempt to discover "whether the program is delivered as intended to the target recipients" (Rossi et al., 2004:431).Any process evaluation is concerned with three programme areas, namely the programme's service utilisation, service delivery and organisational support (Rossi et al., 2004).Service utilisation refers to the extent to which the programme reaches its target audience, while a programme's service delivery refers to its ability to implement the programme as intended.The last evaluation area for a process evaluation is to examine whether the programme has enough resources to implement it as planned.Based on this, the following evaluation questions have been formulated:

Service utilisation
1. Who has utilised the programme's services?2. Which programme services are the clients utilising?
3. What are the clients' perceptions of services offered by the NGO?
4. What has assisted the clients who have entered the CID employment track to become success cases?

Service delivery
5. What are the actual programme activities?Were these the same as the intended activities?
6. How many clients use the NGO's services?
7. Are the activities delivered in an effective manner by the staff to enable clients to find employment, accommodation and to explore the reasons for homelessness?
8. Are the referral activities delivered in an effective manner so that clients can access primary health care, treatment and rehabilitation, social grants, and obtain an identity document?
9. Is the programme delivered according to social development standards?
10. Does the NGO adhere to the conditions stipulated by the CID for the employment of street people as street cleaners?

Organisational support
11. Does the NGO have sufficient resources (financial, human and infrastructure) to deliver the programme effectively?
12. How do the staff organise their activities?Do their efforts complement the activities?

METHOD Design
This evaluation had a dual design.All evaluation questions except Evaluation Question 3 were answered within a descriptive framework.Babbie and Mouton (1998:105) define descriptive research as "the precise measurement and reporting of the characteristics of some population or phenomenon under study." For Evaluation Question 3, however, an exploratory framework was used.This means that the answer to this question does not constitute a precise measurement.This question aimed to find out how clients perceive the services made available to them and the question is therefore an attempt to increase understanding.This meets the criterion of exploratory research which has been described as "the attempt to develop an initial, rough understanding of some phenomenon" (Babbie & Mouton, 1998:105).

Participants
For Evaluation Questions 1-3 (Service Utilisation), all clients who made use of the NGO's services for the week of 22 June to 28 June 2011 were approached to answer structured questions (n=37).Thirty-eight clients were interviewed but one interview containing ambiguous and meaningless answers was omitted.
For Evaluation Question 4 (Service Utilisation), sampling was based on a variation of Brinkerhoff's (2003) Success Case Method (SCM).An assumption was made that the clients who had entered the CID employment track demonstrated that they had begun to utilise the employment opportunities offered by the NGO.They could therefore be deemed as success cases.Four success cases were interviewed.
Responses to Evaluation Questions 5-10 (Service Delivery) and Evaluation Questions 11-12 (Organisational Support) were provided by the programme manager.

Materials
Evaluation Questions 1-3 (Service Utilisation) For the service utilisation questions, data were collected by means of structured interviews with the clients.It could not be assumed that the clients were literate and therefore a self-administered questionnaire was not appropriate.Babbie and Mouton (1998) note that an interview has a number of advantages over a self-administered questionnaire: it usually allows researchers to gather more responses; the interviewer can clarify confusing questions and if the interviewer is allowed to probe for answers, it can decrease the number of "don't know" answers.Although there might be some doubt regarding self-report data from the homeless, there is evidence that this population may report answers with some accuracy (Calsyn, Morse, Klinkebenberg & Trusty, 1997;Gelberg & Siecke, 1997).However, this accuracy is likely to decrease if the answers are complex or if the respondent feels that the answer will result in stigmatisation (Gelberg & Siecke, 1997).Calsyn et al. (1997) and Schumacher et al. (1995) have cautioned that self-report results on mental health and substance abuse services are likely to be underestimates.
The implications of these studies are that the questions which are asked of the homeless should be fairly simple, the interviewer should avoid questions which are seen as potentially stigmatising and that the researcher needs to be aware that underestimates may occur in certain conditions.A checklist, which accommodated all these implications, was designed for the interviews.The checklist allowed the evaluator to determine the age and sex of the respondent and which services the respondent was utilising.It also allowed the evaluator to determine the clients' perceptions of the services utilised.The complete checklist is available from the evaluators.

Evaluation Question 4 (Success Cases)
This question was answered through structured interviews with clients who had entered the CID employment track.The interview was based on an outcome map, which is presented in Table 1.As is evident from the outcome map, the CID employment track consisted of five outcomes.The outcome map allowed the evaluator to determine which outcomes had been achieved.After determining the highest successful outcome, the evaluator asked designated follow-up questions of the participant.
The outcome map and follow-up questions were designed in an attempt to fill "success case interview buckets" (Brinkerhoff, 2003:142).The filling of these buckets allowed the evaluator to determine what factors had contributed to the participant becoming a success case.The first bucket sought to discover what aspects of the employment initiative they used.The second bucket sought to find out what outcomes were achieved, while the third bucket was used to find out what benefits the project had.Bucket four aimed to find out what aspects of their environment were used by success cases to achieve their success.The fifth bucket was optional and was a way for success cases to give suggestions for improvement of the project.

Evaluation Questions 5-10 (Service Delivery)
These evaluation questions were answered by means of an interview based on a service delivery checklist, which is available from the evaluators.The checklist allowed the evaluator to determine whether or not the intended activities were being delivered to clients.If the activities were being implemented as planned, the checklist allowed the evaluator to determine what percentage of clients was using these services.The checklist was also used to determine if the services had been used effectively.The interview did not cover a specific period of time (for example, the year to date), but it was designed to give an overview of service delivery in general.The checklist is available from the evaluators.

Evaluation Questions 11-12 (Organisational Support)
The primary evaluator interviewed the programme manager on the basis of an organisational support checklist, which was designed to determine if there was sufficient funding, staff and infrastructure to support the activities.The organisational support checklist is available from the evaluators.

Procedure
The University of Cape Town's Ethics in Research Committee granted approval that the evaluation could be carried out.The programme manager of the NGO granted written permission for a process evaluation.The programme manager also reviewed and approved all the checklists.Client participants remained anonymous and all participation was on a voluntary basis.Participants who agreed to participate were given a food package sponsored by the primary evaluator.

Evaluation Questions 1-3 (Service Utilisation)
The programme manager introduced the evaluator to the clients and informed them that the evaluation was offered to the NGO free of charge and that it was intended to benefit the organisation.The programme manager stressed that the evaluator was trustworthy and that participation was voluntary.
The interviews were not recorded as the act of recording might have been intimidating to the respondents.

Evaluation Question 4 (Success Cases)
The success case interviews took place after the service utilisation interviews.

Evaluation Questions 5-10 (Service Delivery) and 11-12 (Organisational Support)
The evaluator interviewed the programme manager and recorded his responses on the checklist.

RESULTS
Despite the guidelines followed for designing data-collection tools for the homeless, it should be noted that the data-collection process for this sample was onerous.As will become clear from the paucity of the reported results, the participants provided a minimum of data.Despite probing, participants often did not give direct answers to the interview questions.The primary evaluator adapted his interview style and frequently replaced words on the checklist with synonyms.Researchers wishing to study this population should be aware of the difficulties of collecting valid and reliable data.
The evaluation questions formulated earlier will be used to present the subsections of the results.

Service Utilisation
Evaluation Question 1: Who has utilised the non-governmental organisation's services?
The majority of the respondents were male.There were 28 (75.7%)male respondents and nine female respondents.
The mean age of the respondents was 35.6 (n=35).Two instances where the age of the respondent was not known were omitted from the calculation.The mean age for the women was 29.1 years, while the mean age for the men was 37.9.
Evaluation Question 2: Which programme activities are the clients utilising?Which activities were not sufficiently utilised?
Table 2 presents the services offered by the NGO and the extent to which clients accessed these services.a n=35.Two of the respondents worked for the NGO and therefore their responses were omitted from the total number of respondents.b n=36.There was one response which was unclear and it was therefore omitted.
If an arbitrary cut-off point of 50% is used, then the following services were not sufficiently utilised: wash clothes; build a shelter; sign up for CID contract; work at CID; discuss homelessness; see social worker/staff member regularly and referrals to obtain a social grant, clinics and alcohol counselling.
Evaluation Question 3: What are the clients' perceptions of services offered by the programme?
Table 3 presents client perceptions of the services offered by the programme.
Again, if an arbitrary cut-off point of 50% is used, then clients had positive perceptions of every service except the referral to alcohol counselling.Only six of 37 (33.3%)respondents indicated that this service was helpful.Note.The word used in the questionnaire to describe the quality of the service is included in brackets.a n=36.There is one instance of missing data: a client indicated that he had used a locker, but his perception of the locker remains unclear.His response has been excluded.b n=36.One of the responses by the client was unclear and it was therefore omitted from the analysis.
Evaluation Question 4: What assisted the clients who have entered the City Improvement District employment track to become success cases?When asked what has helped them get employment as a street sweeper, three of the four respondents indicated that either volunteering at the NGO or consistent presence at the premises had helped.
All four respondents indicated that their lives were different after receiving employment, but it was unclear if Respondent 2 was indicating that employment had made the difference or if it was the result of other factors.
None of the respondents were able to identify hindrances in obtaining employment and there were mixed responses as to how the NGO could have helped other people to become street sweepers.

Service Delivery
Evaluation Question 5: What are the actual programme activities?Were these the same as the intended activities?
Table 5 presents all of the NGO's activities and whether or not the programme manager believed that they had been delivered.For each activity which had been delivered, the programme manager also indicated whether he believed that the programme was delivered effectively.The estimated number of clients who used these services is also included.According to Table 5, the programme manager was under the impression that all of the initial and referral activities were delivered in an effective manner.Only two of the core services were delivered effectively and these were related to the provision of employment as CID street sweepers.
Evaluation Question 6: How many clients use the non-governmental organisation's services?
Table 6 presents the comparison between the programme manager's estimates of how many clients used each service per month, and the respondents' indicated usage.
The programme manager estimated that the initial services were the most used services and this was confirmed by the clients.However, and with the exception of the provision of meals, there was a large discrepancy between the estimated and recorded rates of utilisation for each activity.In each case, the estimated rate of utilisation is lower than the recorded rate.Note that the estimate of service utilisation was based on the number of clients using a service each month, while the recorded rate did not take into account a time frame.Evaluation Question 7: Are the activities delivered in an effective manner by the staff to enable clients to find employment, accommodation and to explore the reasons for homelessness?
According to the programme manager, every initial service was delivered in an effective manner to the clients (Table 5).Two core activities, the provision of opportunities to volunteer and to work as a CID street sweeper, were delivered in an effective manner.
Another two core activities (the provision of a shack and seeing a social worker) were not delivered at all.The programme manager did not know if the provision of accommodation, psychosocial counselling and long-term support were effective.
Evaluation Question 8: Are the referral activities delivered in an effective manner so that clients can access primary health care, treatment and rehabilitation, access social grants, and obtain an identity document?
The programme manager reported that all of the referral services were delivered effectively (Table 5).
Evaluation Question 9: Is the programme delivered according to social development standards?
The Department of Social Development does not place onerous demands upon nongovernmental organisations.According to the programme manager, the NGO does ensure that it adheres to the mandated fiscal control policies and occupational health demands.
Evaluation Question 10: Does the programme adhere to the conditions stipulated by the City Improvement District for the employment of street people as street cleaners?
The NGO provided a weekly report to representatives from the CID and ensures that it adheres to contractual obligations.

Organisational support
Evaluation Question 11: Does the non-governmental organisation have sufficient resources (financial, human and infrastructure) to deliver the programme effectively?
Table 7 presents the activities offered by the NGO and the programme manager's perception of whether or not these activities were supported by sufficient staff, budget and infrastructure.The NGO depends on volunteers to help provide meals and without volunteer support it would not be able to provide this service.b There is insufficient budget to provide soap to clients but the shower is always available for use.c There is insufficient budget to provide washing powder to clients but the facilities are available for clients to use.
Table 7 shows an organisation which is significantly under-resourced.The NGO did not have sufficient staff, budget or infrastructure to run their programme as planned.
Evaluation Question 12: How do the staff organise their activities?Do their efforts complement the activities?
According the programme manager, the staff spent most of their time involved in food management and preparation.The collection and distribution of donations was the second most time-consuming activity.The third most time-consuming activity was the counselling of clients in one-on-one settings.
In summary, the results relating to service utilisation indicated that the programme's clients were mostly male and that they had generally favourable perceptions of the services offered to them.The clients made use of a mixture of initial, core and referral services.The results relating to service delivery suggested that all of the initial and referral activities were implemented as planned.However, there were two core services which were not delivered at all (build a shelter and seeing a social worker), and three core services which were delivered but not in an effective way (find accommodation, psychosocial counselling, long-term support).Finally, the NGO did not have sufficient resources to implement the programme as planned.

DISCUSSION
The programme areas which constitute a process evaluation will be utilised to present the discussion.

Service Utilisation
All participants were over the age of 16 and thus fit the age specification of the intended target population.The men were significantly older than the women.This is in contrast with the results of studies by Kok, Cross and Roux (2010) and Olefumi (1999), which showed no age difference.In the current evaluation one can only speculate that homelessness affects men and women at different stages of their lives or that men remain homeless for longer than women.
There were significantly more men than women.This is in line with a survey conducted by Kok et al. (2010), which found that there were more homeless men than women in South Africa.The reason for this gender discrepancy is unclear.It could be that women spend less time on the street because the streets are not perceived to be safe (Kok et al., 2010).Another possible reason is that the pathways into homelessness may affect men more than women.
One implication of having a male-dominated population is that the NGO may be able to tailor their services to problems which are more likely to affect the male homeless.For example, coloured homeless men are more likely to experience high blood pressure, cholesterol, diabetes, emphysema or bronchitis and TB when compared to the general population of South Africa (Seager & Tamasane, 2010).
From the results of the evaluation it was unclear whether all of the participants were indeed homeless and therefore it is not known to what extent the NGO reached its target audience.It should be noted that the NGO has opened up the rehabilitation centre on its premises to the local community.Clients who are not homeless, but utilise the rehabilitation centre, can access the same services as the homeless.No client is requested to identify whether he or she is homeless or from the local community.This coverage is admirable, but has implications for funding and service delivery.
As regards utilisation of initial services, the provision of meals was the most utilised service.Of all the services made available to clients, the provision of food is one which is most obviously related to survival.The only initial service which was not sufficiently utilised was the option to wash clothes at the centre.According to the programme manager, this under-utilisation is possibly due to the fact that washing powder is not always freely available to clients.
The only core service which was sufficiently utilised was the opportunity to volunteer at the centre.This may be due to the fact that clients are often given additional food if they volunteer.It may also be because it was one of only two core services which had sufficient staff, budget and infrastructure to implement as planned (Table 7).Although less than half the participants utilised the CID employment scheme, this activity should be considered fully subscribed, given that only seven clients are employed at any given time.Therefore the only two core services which were fully utilised were those which had sufficient staff, budget and infrastructure (Table 7).
It is worrisome that the core activities which are central to getting participants off the street, namely building a shelter and seeing a social worker, were not utilised sufficiently.The programme manager reported that although there was a fund for building shelters in previous years, there was no budget for this in 2011.No social worker was appointed because of lack of funding.
The NGO has had mixed success in its attempts to refer clients to additional services.The referrals for drug rehabilitation and referrals to obtain identity documents were both sufficiently utilised, but the remaining three referral activities (social grants, clinic and alcohol rehabilitation) were under-utilised.
There does not seem to be a direct link between the referral service utilisation and the resourcing of these services.For example, the referral to obtain an ID document was sufficiently utilised, but there was insufficient budget or infrastructure to implement the serviced as planned.Conversely, the referral to a clinic was insufficiently utilised, but this service was sufficiently resourced.
Client perceptions of the programme services were mostly positive (Table 3).The initial services were especially well received by clients, with three of the four services having approval rates above 90%.The core services all had approval ratings of 70% or above, which suggests that the NGO offered these services in way which was easy for clients to access or in a way which clients found helpful.However, it should be noted that the NGO did not offer two of these core services (building a shelter and seeing a social worker).The only referral service to receive less than 50% approval was that of the referral to alcohol rehabilitation.It would be unwise to read too much into this particular result, given that only six respondents indicated that they had been referred for alcohol rehabilitation.
In summary, it is unclear whether the actual audience of the programme conformed to the guidelines for the intended audience, mainly because it could not be ascertained whether all participants were the street homeless.Clients utilised most of the initial services and specifically the provision of meals.Two of the most crucial core services, building a shelter and seeing a social worker, were not utilised as these were not offered because of a lack of funding.The NGO had mixed success as regards the utilisation of its referral services.Client perceptions of the programme services were mostly positive.

Success Cases
During the interviews with the success cases, it often seemed as if clients did not always understand what was being asked of them and this was especially true of the question which asked them to identify factors which helped them obtain employment as street sweepers.
The question that elicited the most useful data was the one that asked respondents if their life was different once they had begun working for the CID.The opportunity to work as a street sweeper has seemingly had a positive effect on these clients.
Clients were unable to identify any hindrances to becoming employed as a street sweeper.Respondents were also asked how the NGO could help other clients obtain employment as street sweepers.These answers are of limited value and there is little data which the NGO can use to improve its programme.

Service Delivery
The NGO did not carry out two of the planned core activities (Table 5), namely building a shelter and seeing a social worker.Both of these activities were reported to be significantly under-resourced.The absence and resourcing of these activities are likely to hinder the NGO's attempts to help clients leave a life on the streets.It is of some concern that those services which are fundamental to the programme theory are not delivered.
The exact number of clients using the NGO's services is unknown.The results of the research showed that there was a difference between estimated usage and recorded usage (Table 5), but this difference was probably due to different time frames.What is clear is that the programme manager was correct in his estimation that initial services would be the most utilised.The fact that the services which offer immediate and tangible benefits are the most utilised suggests that clients have their focus on present concerns and are less concerned with long-term goals.
Given that so little is known of South Africa's homeless population, it is unclear if the NGO's services are under-or over-utilised when compared to the other homeless organisations in South Africa.According to Sanchez (2010), faith-based organisations which help the homeless have dramatic differences in the number of people they help.For example, he mentions that one organisation helps 20 people per week, while another organisation in the same metropolitan area helps 600-700 people per week.
As regards effective service delivery, the NGO has provided employment to a limited number of clients in an effective manner (Table 5).However, it is not known how successful the organisation is in helping clients find employment outside of the CID employment track.
The NGO did not offer clients the option to build a shelter.However, the NGO did help clients find accommodation through other means.Lack of monitoring data precludes a firm interpretation regarding the success of the latter activity.
The effectiveness of the psychosocial support was not known.When asked about counselling, respondents gave the impression that the counselling was fairly informal and that it was not a regular occurrence.
The programme manager indicated that all of the referral services were delivered effectively (Table 5).This seemed to be confirmed by the fact that all referral activities, apart from the referral for alcohol addiction, were viewed in a positive light by the clients (Table 3).
The programme manager stated that the programme was delivered according to the legal requirements of the Department of Social Development.A brief search of the Department of Social Development's (2011) database confirmed the NGO's registration with the department and submission of annual financial reports.
The CID requires a weekly report from the NGO.The programme manager confirmed that such a report was delivered every week.
In summary, the results obtained for service utilisation are reinforced by the results of service delivery.Again, it was unclear whether the actual recipients were the street homeless.Two of the intended core services, building a shelter and seeing a social worker, were not implemented because of lack of funding.There was a discrepancy between participants' reported usage of services and the programme manager's estimation of such usage.CID employment was deemed to be provided in an effective manner, but it is unclear whether other employment was provided effectively.The service relating to building a shelter was absent.Psychosocial support seemed to have been provided in an informal manner.All referral activities seemed to be provided effectively, with the exception of referral for alcohol addiction.The programme was delivered according to the requirements of the Department of Social Development and conformed to the reporting requirements of the CID.

Organisational Support
The programme manager stated that there were only 6 out of 16 activities which had sufficient staff, budget and infrastructure to implement as planned (Table 7).This would suggest that the NGO is significantly under-resourced and that it may be prudent to focus on fundraising in order to obtain the necessary resources to implement the programme as planned.Despite this lack of resources, a minimum 11 of the 16 activities were considered to be delivered effectively (Table 7).This would suggest that the NGO is achieving successes in certain activities (for example, the provision of meals) despite its limited resources.
According the programme manager, the staff spent most of their time on the most timeconsuming and most utilised service, namely food management and preparation.The collection and distribution of donations is the second most time-consuming activity.The third most time-consuming activity is the counselling of clients in one-on-one settings.
In summary, the NGO is not implementing its programme as planned.The lack of sufficient resources has prevented the implementation of key activities and this is likely to prevent the NGO from achieving its goal of helping the homeless reintegrate into mainstream society.There is, however, reason for the NGO to be optimistic.The lack of resources has only prevented a minority of services from being delivered and the clients have a generally favourable perception of the services offered to them.Should the NGO have sufficient resources, it seems likely that the programme could be implemented as intended.
Suggestions for improving the programme Ferguson and Heidemann (2009) observed that NGOs operate in resource-deprived environments which pose many challenges to service provision.This was also the case with the programme under evaluation.However, despite operating in a resourcedeprived environment, the NGO provided some services effectively and to the satisfaction of its clients.
At this stage it is not known whether more funding for the NGO's homeless programme would be forthcoming in the future.Based on this uncertain environment, the evaluators would like to provide the programme staff with two future scenarios for the programme.
The first scenario focuses on a programme which provides short-term, initial services, limited referral services and can operate on limited funding.The second scenario sketches a multi-dimensional intervention in which a social worker will play a pivotal role.This latter scenario requires substantial funding.
The limited programme will focus on those initial activities which the NGO does well, namely provide food and a place to shower, store and wash clothes.Table 2 indicated that referrals for social grants, clinics and alcohol counselling were under-utilised.The evaluators suggest that all under-utilised referral services be discontinued, but that referrals for drug counselling and IDs should be retained.The main aim of this limited programme will be to provide temporary relief and survival services to the street homeless.
For the multi-dimensional programme, the evaluators reviewed literature on existing, successful programmes for the street homeless.It should be noted that while there are many publications which describe the lot of the homeless, very few evaluations of successful programmes are available.This lack of evaluations is not surprising as O'Connell, Blanchfield, Judge, Koh, Oppenhiemer, Swain and Taube (2010:147) note that "the paucity of evidence-based studies, inconsistent definitions of homelessness, a lack of consistent and incomparable data collection, and difficulties with long-term follow-up are major challenges to the evaluation of effective models of care for homeless populations."The exception to the lack of evidence is the four evaluations of complex, successful programmes for the homeless which are discussed below.
Busch-Geertsema's (2002) evaluation focuses on two projects for the provision of housing to end homelessness in Germany.Underlying these projects was the idea that homeless people should be provided with affordable formal housing which is adequately constructed.The participants were also provided with the support of a social worker and other services if so desired.
The first project was located in Bielefeld.The majority of participants were housed in single-dwelling units and provided with intensive support, with the aim that this support would be reduced over a five year period.The aim of the support was independence so that the participants were able to function without any additional help.It should be noted that participants were men who had significant problems and were considered unlikely to integrate into permanent housing with ease.The second project Busch-Geertsema (2002) reported on was located in Hannover.
Unlike the project in Bielefeld, this project housed both men (n=9) and women (n=3) in single-dwelling unit.All the participants had social problems.The participants were able to access support services offered by different agencies.A social worker met with the clients at least once a month and was contactable if a crisis arose.The results showed that over 90% of the participants managed to remain in the accommodation provided.However, the majority of the residents remained in poverty and were unemployed.As a result, some of the participants were unable to pay rent and might have been forced to return to the street, if social services had not intervened.The sustainability of this approach was in doubt unless the participants had access to funding or employment.
In another evaluation, focusing on housing in New York City, Tsemberis, Gulcur and Nakae ( 2004) assessed the longitudinal effects of a Housing First programme.This programme was aimed at homeless, mentally ill individuals.The evaluation examined the preferences, housing stability, substance use, treatment utilisation and psychiatric symptoms of the participants.Participants were assigned randomly to an experimental group (the Housing First group) or a control group (the Continuum of Care group).The Housing First initiative involved the provision of housing to mentally ill homeless people who were given access to social support through a modified form of Aggressive Community Treatment (ACT).The control group's treatment required participants to receive treatment while staying in transitional housing and then moving into permanent housing at a later stage.Support services were offered to the control group throughout the programme.This approach was based on the belief that homeless people were not capable of moving directly from the street into permanent accommodation.
The results showed that approximately 80% of participants in the experimental group were still living in mainstream society and the initiative was considered a success.This would suggest that providing immediate access to permanent accommodation would be of greater benefit than providing transitional accommodation.
Both of the above evaluations focused on housing provision as way to end homelessness.They suggest that homeless people are able to move directly into, and will benefit from, permanent residential accommodation.In both evaluations the accommodation provided to participants could be considered formal, habitable housing (Hall, 1999).There is evidence (Dworsky & Piliavin, 2000) to suggest that the type of residence into which the homeless move has an impact on their likelihood of returning to homelessness.
However, South Africa's housing situation is complex (Hall, 1999) and it is not clear what effect the different types of housing available will have on re-entry to homelessness.It may be that within a South African context, a shack is a practical option for housing the homeless.It has been argued that: "Shacks may often meet some if not all of the requirements for catalysing selfsaving behaviour as a route out of poverty.If so, then existing shack housing is a rung on the accumulation ladder for those who have it, and may be an option for anyonehomeless or work-seekingwho needs accommodation with location advantage."(Cross & Seager, 2010a:18) A third evaluation by Rog (1999) focused on services-enriched housing.The Homeless Families Programme (HFP) aims to improve or create systems of care for the homeless and to grant access to services-enriched housing for homeless families experiencing difficulties.The programme was implemented in nine cities throughout the United States of America (USA).Homeless families who participated were given access to housing subsidies and support via a case management system.The programme was implemented by different organisations in each of the cities.The results of the evaluation suggest that there were multiple gaps in service delivery to the homeless and that where service delivery occurred, it was not uniform.However, the provision of a housing subsidy could help end homelessness.The results showed that 85% of participants in six of the nine cities were able to live in permanent accommodation for at least 18 months.There were insufficient data on the other three cities.
This evaluation is helpful in that it not only reinforces the role that permanent accommodation plays in helping end homelessness, but it also draws attention to case management.Rog (1999) states that the effectiveness of case management and other support services in contributing to housing stability are unclear.However, the fact that such a high rate of housing stability did occur is a significant achievement.In this evaluation, case management occurred over at least one year and other authors have also called for long-term social support (Drury 2008;Ji, 2006).Both of the evaluations which focused on housing (Busch-Geertsema, 2002;Tsemberis et al., 2004) encouraged the use of case management.For example, Busch-Geertsema (2002:14) states that: "The dwelling alone is often not sufficient.Integrative support has the prior aim of maintaining dwellings, but a further integration into society cannot be achieved by maintenance of tenancies alone ... Case management and the use of regular social support (like advice centres for different problems such as indebtedness, addiction, health problems, etc.) are relevant in this context."Goetz and Schmiege (1996) conducted an evaluation which focused on employment as a pathway out of homelessness.They evaluated the Homeless Employment and Related Training (HEART) project which was based in Oregon, USA.The authors sought to evaluate the role of job skills training, job search assistance and intensive case management in helping the homeless move into mainstream society.
Participants were trained by professionals in the construction industry and local contractors were contacted and encouraged to hire participants who completed the training.Throughout the training, participants were housed in local shelters and they were provided with transportation, clothing, food, medical care and work-appropriate footwear.They were also given psycho-social support in many forms.For example, participants were referred to outpatient rehabilitation centres, and given access to anger management classes and trained counsellors (Goetz & Schmiege, 1996).
This programme was successful as over 60% of participants had found employment at the end of the programme and 65% had moved into permanent housing (Goetz & Schmiege, 1996).It should be noted that no long-term follow up was conducted and it is not known how effective the programme is over time.
Based on the above evaluation, it would seem that employment can be used to help homeless people leave a life on the streets.However, there appear to be certain conditions which the employment opportunities must meet in order to help clients get off the street.Firstly, there must be long-term support once the client is employed (Chamberlain & Johnson, 2008;Goetz & Schmiege, 1996;Hursch & Schutt, 2009).Secondly, these employment opportunities must be close to available and accessible accommodation (Cross & Seager, 2010a, 2010b) or the participants run the risk of losing their jobs due to poor performance.Goetz and Schmiege (1996) make it explicit that their participants were given accommodation because they did not want participants to have to worry about daily struggles while learning a new skill.They also argue for a third condition, which is that job training needs to be done in collaboration with industry stakeholders.
Thus far, the literature discussed has called for a multi-dimensional approach to solving homelessness.This is to be expected given that there is almost universal recognition that the solution to homelessness is certain to be complex (O'Connell et al., 2010;Shier, Jones & Graham, 2010).The above evaluations and supporting literature reflect this complexity by showing that shelter, employment and long-term support are foundational to helping the homeless live off the street.Based on the literature presented here, a revised programme theory for an effective multi-dimensional programme has been designed by the evaluators and is depicted in Figure 2.
It should be noted that this programme theory focuses on three main areas, namely housing, employment and case management.All of these areas require substantial funding and a monitoring system which can track implementation fidelity and outcome progress.Should the NGO adopt the latter scenario, a concerted effort to raise funds is required.
Tailor the question to reflect the achieved outcome.For example, if the highest success outcome was that he/she worked as a street sweeper, then question 1 will be: what has helped you become a street sweeper for the CID?

TABLE 3 CLIENT PERCEPTIONS OF PROGRAMME'S SERVICES
Table 4 presents the questions asked of four success cases.Their answers are given under each question.

TABLE 4 SUCCESS CASE RESPONSES Question
1: What has helped you to become a street sweeper?Respondent 1: Persistence in arriving at [the NGO] Respondent 2: [The NGO]and volunteering Respondent 3: It is the only thing can keep me going.Volunteering.How can [the NGO] help other people become street sweepers?Respondent 1: I can't say.Respondent 2: [The NGO] can be honest and caring.You must believe in yourself.You must make your mind open using God's Word.
Respondent 3: Become helpers and get [the NGO] to work with you.Respondent 4: By volunteering.