Paper
Research informed practice? Making a difference?
- issue: Issue 1 / 2009
- authors: Margaret McKenzie
- keywords: New Zealand, family, interventions, evaluation, research
- views: 3165
- downloaded: 0
- download pdf (63 Kb)
abstract
Background. The call for and emphasis upon rigorous evidence based evaluation to inform both practice service delivery and programme policy planning rests upon the notion that such research both can and will make a difference. The emphasis in this paper is on the developing of appropriate research methodologies.
How will research make a difference? How do we ensure research informs practice?
This paper critically reflects on the possibilities and potentials of these dual challenges; can make a difference (evidenced) and will make a difference (responsive), that emerge when conducting research to measure the effectiveness of social work intervention with troubled families.
A key focus is getting the balance in research design to ask useful questions and provide useful answers for all partners in the research. How do we address these to ensure competing demands around who will benefit from research are met? First, will it make a difference to the life of the participants? How will telling their individual stories benefit them? And secondly will it make a difference to the lives of other families? Will their stories be listened to? How can these stories of lived realities generate information to shape governmental policy? How can stories of individual realities be translated into the funder/policy development requirements for evidence-based, outcome orientated policy development information and yet retain the local and grounded elements of individual family needs.
Aim and method. This paper explores these issues by describing a developmental process of working through the methodological challenges of collecting such appropriate and useful data and its consequent impact on service delivery and policy provision.
A research programme encompassing three evaluation projects each with providers and service users of programmes providing social work services to troubled families is traced and discussed. Each study is of a small-scale population-specific programme and is linked through a cumulative process of addressing the dilemma of responsiveness to all interested parties: recipients, providers, funders and policy-planners of programmes. Each project is delivered within a strengths-based social work philosophy.
The early project involved the collection of rich experiential retrospective qualitative data from both service users and providers, the second project added the collection of both pre and post intervention outcome measures and the third incorporated the development of further multi layered data collection, adding both standardised well-being indicator outcome measures and the tracking of external contextual factors, such as service and policy drivers again form both users and providers.
Key findings. Project One findings were of significant impact in furthering understanding of individual experiences of involvement in a new programme delivery and highlighted a wide difference in experiences between programme recipient and programme provider but an overall emphasis on the experiential aspects of the programme did not allow sufficient ability to evaluate outcomes.
Project Two incorporated retrospectively reported outcomes, findings of positive family outcomes despite ongoing concerns about the programme processes while of key value at single programme level both for feedback and improvement in delivery remained of limited value at policy and funder level as the absence of pre and post measures limited the cross comparability value of these findings. The impact of procedural, administration, funding and policy alterations was also widely reported by both providers and recipients but had not been specifically measured.
Project Three design development incorporated a mix of the elements cumulatively found to be needed for programme evaluation of strengths - based social work interventions, to address the multi layered information expectations made of social work programme evaluation research. These include, pre and post test of standardised well-being indicators for children and young people incorporated to address policy and funder delivery requirements, regular tracking of the impact of key policy and procedural changes and developments and their impact on agency ability to deliver programmes to address programme provider concerns and retaining the collection of rich experiential data via regular interval interviews to ensure the voice of programme recipients remains.
Recommendations. Extensive stakeholder consultation is essential to ensure all parties will be in a position to utilise the research results.
Understanding what comprises "beneficial" research must acknowledge that the needs of stakeholders are varied and thus complex evaluation design strategies are required.
Localised contexts of delivery are crucial to understanding why or why not service provision is considered beneficial/successful.
Conclusion. The paper describes the development of a multilayered research projects derived from engaging with the competing demands of balancing the intrusion into both families lives and agency workplaces in order to provide evidence based findings which can contribute to making a difference to both individual families realities but also to policy development within a strengths based context. The paper argues for a joint emphasis balancing meaningful in-depth involvement and contextual understanding with standardised outcome indicators for measuring effectiveness and concludes with a discussion on the emerging complexity of managing the reporting of unpalatable effectiveness results.
Key references
Darlington, Y. & Scott, D. (2002). Qualitative Research in practice. Australia: Allen and Unwin.
Dolan, P., Canavan, J., & Pinkerton, J. (Eds.). (2006). Family support as Reflective Practice. London: Jessica Kingsley.
Worrall, J. & McKenzie, M. (2003). Researching with Vulnerable families. In R. Munford & J. Sanders (Eds.), Making a difference in families: Research that creates change. Australia: Allen and Unwin.
Contacts: Margaret McKenzie, Senior Lecturer, Department of Social Work and Community Development, University of Otago, P.O. Box 56, Dunedin, New Zealand. E-mail: margaret.mckenzie@otago.ac.nz.