Monitoring and Evaluation

Jump to section:

For many schools and delivery organisations, the benefits of participation in Arts and cultural activities for children and young people with special educational needs are already well valued, based on first hand experience of the impact on individual children. However, robust monitoring and evaluation can provide valuable evidence to demonstrate the impact of arts and culture for children and young people individually as well as in group settings, and provide the rationale for investment in more opportunities so that greater numbers can benefit. Effective evaluation should not only showcases the outcomes but also spark learning and organisational development, guiding future activities and foster best practices.

When surveyed, 83% of delivery organisations and 96% of schools expressed confidence in monitoring and evaluating the impact activities have for children and young people with special educational needs. Despite this, over 30% of schools still noted that evaluation requirements sometimes hindered their engagement with arts and culture organisations.

While organisations and schools reported confidence in monitoring and evaluation, this doesn’t always equate to effectiveness. Organisations use a variety of different methods to demonstrate the impact of their work which vary in quality and robustness. Key components of effective monitoring and evaluation included:

Establishing robust data collection and reporting processes within organisations.
Adopting a person-centred approach to evaluation but also addressing impacts beyond the individual, considering the entire family unit.
Emphasising qualitative data collection, such as hosting informal gatherings for discussions rather than relying solely on formal forms.
Leveraging teachers’ expertise to ensure meaningful and appropriate evaluation, but remaining mindful of their capacity and workload challenges.

Employing creative methods to engage young people and amplify their voices, such as video evaluations and using sensory and tactile evaluation techniques, which can include using ‘traffic light paddles’ to convey feelings and emotions, drawing, and using Makaton or Widgit symbols.

“We have worked very hard to improve monitoring and evaluation, but occasionally funders request quantitative / statistical measures for pupil development that are extremely difficult to quantify.”
Akademi

There was a general confidence in monitoring and evaluation, however, notable challenges were highlighted including the self-perception accuracy of children and young people, parental concerns about case study/photo permissions, and the incidental bias caused by traditional evaluation methods such as focus groups, which can exclude non-speaking young people. Less traditional and more innovative evaluation methods need to be used when working with children and young people with special educational needs. It was noted that activities do not always have significant, long-term outcomes, but may simply provide the children the chance to have positive experiences in the moment or the short-term, and that sometimes this is enough. Additionally, organisations and schools reinforced the time-consuming nature of data collection.

While 79% of organisations surveyed receive some contribution for planning and evaluation costs through grants, there remains a consensus among delivery organisations that funding bodies should provide clearer guidelines for evaluation requirements and avoid unnecessary data collection. Most funding bodies have different forms asking for different data which often requires an organisation to have a dedicated person for reporting. Some organisations mentioned needing to ‘jump through hoops at the application stage’ but then a funder not following up at the end of the project.

“Successful delivery requires a dedicated investment of time, effort, and resources to ensure that all individual needs are met. It is crucial to recognize that providing inclusive support goes beyond initial implementation. Proper evaluation is a critical aspect that is often overlooked. Without inclusive evaluation techniques, not all voices and perspectives will be effectively captured, potentially leaving some individuals unheard (particularly those with communication challenges). Moreover, if evaluation is not accurate and comprehensive, the project as a whole cannot properly evolve and adapt to meet the changing needs of those it serves. Therefore, it is imperative to allocate sufficient resources and attention to both the delivery and evaluation phases, ensuring that all individuals with SEND receive the necessary support and that their experiences contribute to ongoing improvement and inclusivity.”
Flamingo Chicks

Examples of Evaluation Techniques

Using Makaton or Widgit symbols across data collection methods
Surveys and questionnaires that include sensory materials and sounds
The use of peer evaluators
Gamification of data collection
Observations and individual case studies
Parent/carer and teacher testimonies
Using colours or faces to show feedback

Considerations

Robust monitoring and evaluation is needed to share the impact of arts and culture for D/deaf, disabled and neurodiverse young people.

There is no one-size fits all approach to monitoring and evaluation so schools and organisations are using different creative and sensory methods to evaluation.

Organisations and schools agree that there needs to be proportionate and reasonable evaluation criteria set by funders, and that it should be made clear how this information is to be used.

Recommendations

It is important to ensure evaluation remains meaningful and proportional to the project’s scale and funding
Evaluations techniques should be solution-based, person-centred, and reflective of the needs of children and young people
Funding organisations should maintain realistic expectations regarding data collection within given timeframes
Funding organisations should clarify the purpose and use of data collected by funding organisations
With the capacity required to gather data it is important that funders, schools and delivery organisations work together to avoid duplicating data collection efforts

Contact

Savoy Strand, 105 Strand, London, WC2R 0AA
T: 020 7259 1700