Programme Evaluation

Summary of 2013 Sexwise Online Surveys

Internal Monitoring of 2012 Sexwise

The Sexwise Evaluation Plan contained four outcomes and these are listed below along with the findings. Each outcome had a standards table and the sources of data include client surveys, THETA recorded observations and health sector feedback. In 2012 THETA trialled the use of an online survey form, and this worked exceptionally well for Teachers. Students are difficult to access, owing for the need to maintain privacy but THETA is considering ways to increase ethical participation in the survey in 2013. Despite the small sample size of the student evaluation, their comments are supported by observations; incidental discussion with students; teachers; service providers; focus groups; and Actor/facilitator journals.

To what extent do participants have an effective understanding of safer sexual practices?

Results from online survey for students: High standard

As a result in participating in Sexwise most (90% or more) students can identify 3 or more practices that one might undertake to practice safer sex.

Over 80% listed 3 safer sex practices
At least 50% listed 5 safer sex practices
Use of condoms listed by all
Other listed practices include: taking responsibility for yourself, saying no, knowing your partner first, an awareness of the issues with sex and alcohol, having someone to talk to about these things, delaying or waiting for the right time to have sex, communicating with prospective partners, and taking responsibility in supporting friends.
Practices mentioned only a small number of times: personal cleanliness, contraception and abstinence.
In responding to what might be done differently by contributors the same things were identified as above, and in the responses given in advice to characters in Sexwise these things are also common.

To what extent do participants have a raised awareness of contributing factors to building positive healthy relationships?

Results from online survey for students: Medium standard

70% - 90% of student participants can give at least two features of a healthy relationship.

Honesty and trust mentioned more often by a considerable margin than other attributes.
Communication, respect and self worth were mentioned frequently.
Friendship, faithfulness and supportiveness were mentioned several times as were safe sex practices.
Love, equality, commitment and decision-making were mentioned but not often.
Actor/Facilitator Journal entries on advice given is in line with those things listed in the online evaluations but also recorded many observations relating to diversity.

What do participants know about sexual health services available to them generally, and locally?

Results from online survey for students: Low standard

50% listed two, three or more.
50% listed one or fewer.

A wide variety of services were mentioned and include those specific to local areas e.g. Choices, Café and Directions, and nationwide services. Some are help phone lines or websites e.g. Help Line, Kidsline, THETA, Rainbow Youth.
A small number were mentioned many times and the majority only a few times.
Refer to Appendix C Summary of Sexwise Evaluation 2012 for the full list.

To what extent is there motivation to engage in meaningful communication within Whånau about sexual health?

Results from survey of Preview attendees: High standard

As a result of attending this parent preview do you think it is likely you will discuss the programme and issues it presented with the young people in you family?
Not likely – 1 Maybe - 6 Very Likely – 58

The Not Likely response was explained as a person having adult children.
The majority attending the parent preview agree this is a programme that will result in discussion withing whanau. A small number were not so sure. No-one felt it was not likely.
The large majority agreed the programme would likely get discussion with students.
Some Parent responses:

‘A great and simple presentation that should promote dialogue/discussion. Very innovative.’
‘I feel more prepared for those questions now.

Although Parent Preview surveys showed parents thought students would be more likely to discuss sexual health with them as a result of seeing the programme, the majority of students did not feel this way at all.
Students believed an increased confidence in discussion with friends and health professionals was most likely.
However, when students were involved in Sexwise they often gave advice that characters should talk to their parents.

Process-based evaluation

Results from online survey for Teachers: High standard

Which of the following best describes the engagement levels of students participating in Sexwise?

84% of Teachers responded: ‘Over 90% of students engaged with programme for over 90% of the time.

Results from online survey for sutdents

In what ways did you engage with the programme? You can check more than one box.

100% of students enjoyed Sexwise and found it engaging
100% engaged passively – watching and listening
Active participation in questioning characters was around 70%
Positive advice to characters was given by between 50-60%
High engagement levels meant little issue with management of students
Motivated interest

Passive engagement (watching and listening) levels were very high.
Active engagement (questioning and giving advice) levels are still high but it would be more conducive to learning if the percentage of actively engaged students were lifted.

Work and Education Development Services (WERDS) evaluation of 2011 Sexwise

An external evaluation of the Sexwise programme was commissioned by the Ministry of Health in April 2011.

Executive Summary

The New Zealand Health Curriculum states that the purpose of sexuality education is to provide students with the knowledge, understanding and skills to develop positive attitudes towards sexuality, to take care of their sexual health and to enhance interpersonal relationships (Ministry of Education, 1999). According to the Educational Review office (2007), these outcomes have not so far been effectively achieved in the majority of health classrooms, in particular for Maori, Pacific Islands students and students of diverse sexual orientations.

This evaluation, using qualitative and quantitative research techniques, explored how well THETA's Sexwise programme meets the specifications of its funder, the Ministry of Health. These specifications are to

build students' knowledge, understanding and attitudes
fit in with and add value to the school curriculum, to engage with national and local agencies
impact positively on teacher and parent knowledge
demonstrate cultural competency.

The Theatre in Education Trust (THETA) draws on an Applied Theatre approach to deliver sexuality education programmes to secondary students in New Zealand. It is this approach which is seen as the 'value added' component and the point of difference between it and other programmes in health education offered to schools.

The results showed that students engaged enthusiastically in the programme and reported changes in thinking about their behaviour and future actions in each of the areas specified. Students felt they had learned from the programme. It is clear that students brought their own knowledge, understanding and skills to the programme and that skilful performances and well planned workshops provided a safe forum in which students could ask difficult questions and offer advice to the characters (and therefore to each other). The limitations of these findings are that since sexual health was being taught in some schools prior to or at the time of the Sexwise programme, gains in knowledge, understanding and/or attitudes during the period of the evaluation may not be attributable soley to the Sexwise programme.

The research found that sexuality education was a difficult area for teachers and that this to some extent explained their willingness to support the external programme. Teachers viewed the Sexwise programme as a complement to and reinforcement of the sexuality themes taught in classrooms as part of the health curriculum. Some teachers recognised that the use of drama allowed their students to identify with the characters and reflect on the scenarios while providing the emotional distance to discuss sensitive issues of relationships and sexuality.

For most teachers, arranging and supporting it in the school proved problematic despite their willingness to accommodate the programme. Costs were minimised for THETA if the company could work with a school over two or more consecutive days in order to reach all targeted students. Yet this disrupted school timetables. An issue for actors was that some teachers had difficulty meeting their obligation to stay with the students during the performance (possibly due to in-school pressures).

Stakeholders included local and national agencies concerned with sexuality education and support. They felt that Sexwise offered a valuable contribution to sexuality education and that it was a powerful vehicle for student engagement. They believed Sexwise should stay as it is: fun, humourous and interactive and did not want the programme to change to incorporate too much factual information. They did, however, believe the workshops could become even stronger by going into more depth with the issues. The main drawback stakeholders saw was the need for follow up after Sexwise visits a school.

Maori and Pacific students account for nearly three quarters of THETA Sexwise programme participants in the research. All teachers and stakeholders agreed that the programme was culturally “very appropriate” for these students. They appeared to be fully engaged with the programme. Their responses were closely in line with those of non-Maori and Pacific students. High numbers of Maori and Pacific students reported knowing more about sexual and relationship topics after the THETA Sexwise programme than they had done before the programme. In terms of accessing information on sexual and relationship topics, the preferences of Maori and Pacific students shifted focus after the Sexwise programme from traditional sources like friends and whanau to organised clinical services like Sexual Health Clinic (SHC), School Nurse, GP, School Counselor and Family Planning. Some of these students are accessing information via these new avenues.

As THETA had advised, there was evidence that the programme was in a transitional phase, part way through a significant shift in approach from a simple entertainment and information focus, to an Applied Theatre approach. The deeper approach focuses more on engaging the active and affective participation and contribution of students in an environment of emotional safety. For actors used to theatre as performance only, this new approach brings great challenges. Already a respected programme in schools, the programme has the potential to become a powerful vehicle for change in students' knowledge, attitudes and skills in relation to sexual health.