York Report....Part One

The Integrated Children’s System

An Evaluation of the Practice, Process and Consequences of the ICS in Councils with Social Services Responsibilities

Margaret Bell

Ian Shaw

Ian Sinclair

Patricia Sloper

(University of York)

Jackie Rafferty

(University of Southampton)

Report to

Department for Education and Skills

Welsh Assembly Government

May 2007

The Integrated Children’s System:

An Evaluation of the Practice, Process and Consequences of

the ICS in CSSRs

Contents Page

Executive Summary and Recommendations 3

Section A The ICS Evaluation

Chapter One Introduction 17 Chapter Two The ICS Pilot Sites 22

Chapter Three Design, Fieldwork and Analysis 32

Section B Implementing the ICS

Chapter Four Realizing the Integrated Children’s System 41

The Case Studies

Section C Practitioners and Managers Views of the ICS

Chapter Five Practice Participants Assess the Integrated Children’s

System 52

Chapter Six The Social Workers Experience of using the ICS 68

Section D How the ICS is used in CSSRs

Chapter Seven The ICS and Aggregate Statistics 83

Chapter Eight Using the Exemplars 97

Chapter Nine The Time Study 114

Chapter Ten ICS and Questions for the Delivery of

Social Work 122

Section E Users, Carers and the ICS

Chapter Eleven Children, Young People, Carers and the ICS 137

Chapter Twelve Children and Young People with Disabilities

and their Families 151

Chapter Thirteen Conclusions and Recommendations 170

Appendices 183

References 225

Notes 227

The Integrated Children’s System

An Evaluation of the Practice, Process and Consequences of

the ICS in CSSRs

Executive Summary

This summary and report give an account of the evaluation of the Integrated Children’s System implementation in two sites in England and two in Wales, conducted on behalf of the Department for Education and Skills and the Welsh Assembly Government by a research team based at the University of York. The evaluation took place from 2004 to 2006.

A THE EVALUATION

Design and delivery

A1 The evaluation had the following aims:

To explore, from the perspective of service providers and service users:

• How well the different parts of the ICS fit together and promote best practice in direct work with children of all ages and abilities and their families or carers
• The ways in which the ICS helps children, young people and their carers from different subgroups, in particular children and young people with disabilities, to better understand the social work process and to make more sense of what is happening in their lives
• Whether the system is acceptable to all service users and to practitioners and their managers and does it help to promote partnership working and information gathering?
• What difficulties have been experienced in implementing the ICS and how could these be overcome?

A2 From the perspective of the service providers:

• What additional time will staff need to complete the records and what are the costs?
• What training needs are identified, including in the use of technology
• What additional technology will be required and what are the implications for councils IT systems?

A3 The design for this evaluation involved three linked substudies.

1. The audit study:

This assessed the system’s records (the download study), the time it required (the time study), the coherence and quality of recording (the record study) and the response of social workers and team leaders to it (the questionnaire survey and associated focus groups).

2. The process study:

The process study comprises three parts:

1. It examines the system from the perspectives of the children, their carers and their parents
2. It explores the views of practitioners and members of the pilot site local implementation management groups on the suitability of the system for children of different ages and at different stages of the system.
3. A technology case study evaluates the technical aspects of implementing the ICS in CSSRs.

3) The disability substudy:

This substudy explores both social workers’ experiences of using ICS, in particular the exemplars, and their perceived relevance and appropriateness for disabled children, and parents' and disabled children’s experiences of how information had been collected under ICS.

A development worker supported the authorities in developing the system and learning from each other and the research, and provided a bridge between the pilot work and the evaluation team

A4 The DfES and WAG selected five sites for the evaluation; three in England and two in Wales. One of the English sites withdrew before our study started, leaving four pilot sites for us to study.

A5 As the study progressed, significant modifications were made to the planned design due to the difficulties of implementation and delays at the pilot sites. These caused delays in fieldwork; some loss of data quality; and loss of some planned datasets. In addition, the brief was premised on the mistaken assumption that service users would possess first-hand experience and knowledge of the ICS.

B Implementation

B1 The process of implementation was characterized by delay due to:

a) Variations in the extent to which the ICS was known about, ‘owned’ and prepared for within each pilot site

b) Fragmented technology package development due to different contractual arrangements with software providers

c) Underestimating the demands that a complex system such as ICS would make upon the organization, the individuals within it and resources

d) Underestimating the significance of ensuring compatibility of ICS with existing databases within councils and with other agency databases

e) Input problems, associated in particular with the increased time demands of the ICS and the inflexibility of the electronic file structures developed for the ICS

f) Difficulties in changing file records and managing the assimilation of paper documentation

h) Information retrieval and security problems associated with restrictions on particular teams or from particular places, and difficulties in collating material on different screens.

B2 Two IT case studies suggested that successful IT implementation will need to resolve questions relating to:

1. Choice between in-house and externally commissioned information system development. In-house is likely to be preferred by local staff. This may not be realistic given the need for inter-operability between agencies and government, and to achieve updating. However, in-house support will be necessary with the ability to tailor the system to local needs.
2. Identifying transition issues, and distinguishing IT transition issues from broader ICS ‘technology’ issues related to the Exemplars.
3. Ensuring that ‘micro-level’ problems are not neglected while resolving ‘high level’ problems.
4. Managing variations in practitioner’s levels of IT skill and the acceptance of IT in the workplace.
5. Ensuring up to date and appropriate technology within CSSRs.

f) Providing ongoing additional training to ensure full use of the ICS capabilities, including comprehensive joint training for practitioners working with children with disabilities.

B3 Site implementation case study: the ‘success’ story

The experience from an examination of the pilot site that successfully implemented the ICS suggests that the key implementation issues involve:

a) Managing innovation, change and uncertainty;

b) Addressing tensions between the national ICS drivers and local practice;

c) Managing the interplay between ICS and social work practice

Shared ownership was important and influenced by active and ongoing ‘education’ about the ICS and the valuing of local expertise.

Problems that presented as intractable during the period of our evaluation included the role of external agencies and problems connected with the referral process.

C PRACITIONERS’ AND MANAGERS’ VIEWS OF THE EXEMPLARS

C1 In general, there was considerable similarity in many of the views expressed by social workers and team leaders in our pilot sites over the two years of our evaluation. There were some differences in attitude across sites and teams. Qualified social workers remained more negative about it than other workers.

C2 Overall, the system was seen by two thirds of the respondents as an advance on the paper systems that preceded it and that it had the potential to bring major benefits. The aims, to communicate a detailed picture of the child and family across and within agencies electronically, were strongly endorsed. The need for practice to be accountable and for this accountability to be made manifest through documentation was also widely accepted and understood as a general principle.

Positives

• The highest rating was given for the aim of ‘a convenient way of keeping practical details’.
• The system was given quite high ratings for its security and for it’s capacity to keep all key information electronically.
• Preparedness to adapt the system was experienced positively.

C3 However, the ICS was also subject to criticism and the belief that the overall aims were not being realized. While optimism about the system’s potential prevailed over time and some early sources of negativity were resolved, other negative attitudes crystallized and hardened. Social workers with longer experience of ICS were not more positive towards it than those who had relatively recent contact with it. The changes are therefore not in any straightforward direction

Criticisms

• The most common criticisms were that the ICS was too ‘prescriptive’ and uniform, too long, too repetitive and divided information unhelpfully into chunks. Concern was expressed that since the exemplars insisted on similarity, they failed to ask questions that were relevant to some children while asking of others questions that were irrelevant.
• The tick boxes were criticised. Concerns were expressed that lack of precision could lead to inaccuracy and that in some cases a box had to be ticked before they could move on. It was felt that, although the exemplars were used to describe individuals, they symbolised the view that the business of assessment was one of collecting unrelated pieces of information.
• Exemplars were experienced as producing potentially bland and generalised family information, with the specific care needs of a child and caring strategies of each family not at the fore. The view was that the exemplars failed to capture key information and the documentation obscured the family context, resulting in the loss of narrative. Separating information under numerous headings and sections fragmented reports and was thought to inhibit a holistic overview of family life. This meant it was difficult to convey the essence of a case quickly.
• The uniform approach was seen as disadvantaging certain groups. Social workers in the disability teams thought the multiple and complex needs of disabled children could not easily be slotted into a specific area of the exemplars and they were unclear where to record specific disability relevant information.
• Similar concerns were raised in the way ethnicity was addressed. It was unclear where to insert specific cultural information.
• The segmentation of the social work task was felt to diminish analysis and risk assessment. There were particular concerns about risk because it was unclear where this information would be located.
• At the same time, social workers in the disability teams expressed concern at the child protection bias of the system, which they saw as being a carryover from the LAC forms.
• The level of detail required to make ICS forms a comprehensive means of evidencing practice was thought to potentially sacrifice the conciseness and clarity that is needed to make documentation useful to service users.
• There was agreement with the principle that each child should be separately considered, but many felt the system was taken too far. The result of opening a separate file for each child in a family was that the documentation was voluminous, cumbersome and unwieldy.

C4 Role and skills

Striking aspects that emerged were the need for the acquisition of new expertise and some role changes, and the need for a high level of analytical skill to make appropriate decisions about which parts of ICS exemplars to complete. Knowledge about the research base of the ICS was also important.

Concerns were raised about the re-definition of social workers’ role, as more time in the office form–filling was viewed as a potentially negative re-direction into a more administrative role, with less time available for family visits.

C5 Practice

There was agreement that it was the quality of the individual practitioner that was key, and that prescriptive exemplars could not ensure a uniform standard of practice.

The respondents thought the exemplars demanded a changed culture of working by determining what information is sought at what stage and within specified time limits, and by highlighting issues regarding the relationship between administrative and professional roles in social work.

There were particular difficulties for disabled practitioners, for example font size.

The disability substudy suggested that, while the language of the exemplars probably did lead to a more ‘up front’ focus on aims and objectives, the language of outcomes was not very explicit.

C6 Impact on services users

The lowest achieved aim was for ‘user friendliness’. Half those replying considered the system ‘worse than useless’ in this respect. In the disability substudy three quarters of the social workers had not considered parental participation to be an aim of ICS, and some thought the ICS could lead to a decrease in parental participation.

Generally, the exemplars were thought to be cumbersome and unwieldy and risked alienating parents as a result of unduly technical and complex forms. In the disability substudy it was suggested that such a complex system could lead to less information being shared.

Some disability social workers felt that they were now more reliant than previously on parents for children’s views and that the exemplars were meaningless for children and young people with disabilities. They raised issues relating to jargon and inaccessible language, uninteresting formats/layout and also depth and degree of information recorded.

C7 Inter agency work

While the principle of sharing information with other agencies was widely welcomed, there was uncertainty that inter-professional difficulties would be resolved or addressed by the ICS.

Inter agency information sharing continued to be limited due to incompatible information systems. In the disability substudy, other agencies were still completing and submitting their own assessment formats and it was considered unlikely that this practice will change.

D How the ICS was used in CSSRs

D1 The quality of the data recorded by the system was variable. Some data (eg. date of referral) might be reliably provided, while other data might often be missing (eg. reason for referral). Questions were only routinely answered when the computer system demanded that this was done, and such prescriptions could also be resented.

D2 On many of the exemplars, sections were blank or incomplete (eg. interviewing dates). Recording practice was variable as there were differences between social worker and work groups in the ways in which they entered the data. On a crude rating of the completion and quality of the information recorded, one third (34%) were rated as good, just under a third (30%) as intermediate and the remainder (36%) as poor.

D3 Recording was more commonly descriptive than analytical. 25% evidenced clear analysis, 28% limited and the reminder (47%) none at all.

D4 It was difficult to get a holistic picture of the case and its progress because of the segmental nature of the exemplars. They did not ‘tell the story’.

D5 For children with disabilities, the inappropriateness of the Exemplars fostered a perceived risk of information overload and inadequate fitness for purpose.

D6 Some exemplars were rated highly in some agencies, and after amendment- the Initial Assessment and the Children and Young Persons Care Plans. Some were not used at all, eg the Assessment and Progress Record, or were not accepted by other agencies, such as the Chronology

D7 There was a high-level of duplicate entries in that two or more lines referred to the same process starting on the same date for the same child. Many initial and core assessments were neither completed nor cancelled. A high proportion of referrals are closed on the day of referral or the day after.

D8 None of the exemplars contained any first hand comments from service users. Where they were included they comprised summaries by the social worker.

D9 The problem of paper records and documentation existing alongside electronic records was not resolved

D11 Time commitments

The time required for completing the exemplars is considerable and varies between cases and between authorities. Completed initial assessments took a mean time of 10.19 hours, completed core assessments took 48.14 hours; completed child care plans took an average of 27.4 hours and completed reviews took an average of 36.81 hours.

As some of these tasks appeared not to be completed in some cases, the average of time spent on the related activities would be much larger

A small group of cases (eg. young children at risk) make extreme demands on time and have a considerable effect on the average.

In general, the evidence suggested that the ICS had increased the time spent on recording (at least in the eyes of the social workers) but other pressures were more influential in determining the overall amount of time spent on initial assessments, core assessments etc in the different authorities despite their common use of the ICS

D12 Workload

While there were differences by role and by the area considered, almost everyone agreed that the ICS demanded more time, the additional time being spent on the actual business of recording.

E SERVICE USERS AND CARERS

Carers

E1 Generally, the carers knew about the ICS only because they had been briefed about the evaluation. Otherwise they said that either they knew nothing about it or had not understood. The only change mentioned was the social workers greater preoccupation with accurate information and longer meetings.

E2 The increased time demands of ICS were seen as associated with ‘paperwork’ rather than any significant shift in orientation to social work.

E3 Carers emphasized the relational aspects of work with young people and social workers; continuity of social work links; being listened to by social workers; being given appropriate information about children; and evidence that they had achieved something for the lives of the young people in their care.

Children and young people

E4 Seven of the ten young people interviewed had no knowledge of the ICS and did not think they had seen ICS documents,

E5 All were aware that children’s social services held records about them, and wanted their records to be secure and only shared with relevant people.

E6 Nine out of ten used computers for themselves.

E7 As with the carers, it was their relationship with their social worker which was important and this featured more than the information they were recording about them. None were aware of significant changes in approaches to social work practice as a result of ICS.

E8 They expressed concern that social workers do not always communicate fully and clearly with them.

Disabled children and their families

E18 Although families were interviewed after their assessment/review under ICS most had no knowledge of the ICS system, or that their social service department was piloting new information and recording system. Despite this lack of information, less than half of the parents wanted more information about the ICS.

E19 None could recollect their children being informed. Although it was apparent that most social workers did not explain or mention ICS to the children this was not viewed as problematic. As both a concept and system of information collecting and recording it was regarded as too abstract for children, especially those with communication difficulties.

E20 While many parents had not considered issues around electronic exchange of information, some expressed concern about confidentiality issues and some raised issues of accessibility to their records. Others were pragmatic in saying, like the social workers, that security could not be guaranteed on paper files, either.

E21 The potential advantages they foresaw from electronic records were to enable social workers (and also professionals from other agencies) to transfer and share information more effectively – especially in the hope that sharing may reduce the need for repetition of information

E22 Parents viewed written reports in a practical manner - as a bureaucratic form to be kept but sometimes of very little interest. Paper records were seen as enabling parents to contribute more easily, whereas with electronic records there is greater dependency on social workers/ administrators inputting their comments or making amendments.

E23 Parents’ evaluations were that social work practice had not changed with the introduction of ICS. Three factors were important to them: the format assessments take – especially having relatively informal meetings, being kept informed, and social workers’ personal approach. They stressed the importance of being respected as a ‘knowing parent’ with social workers asking them for information about their child’s medical and care needs and also wider family needs, such as partners and siblings. Being consulted and listened to in a non-patronizing manner was paramount, as was to meet their needs.

E24 Parents noted no discernable difference between pre and post ICS assessments/reviews, including that social workers had not altered their practice. Only a small number could remember signing a copy of their recently conducted ICS assessment, although it emerged that signing is currently not routine practice; neither is it viewed as a matter of concern

Recommendations

Preliminary comments.

We believe that many aspects of the intentions of the ICS policy are greatly to be welcomed, and hope that our report will contribute to facilitating the appropriate delivery of those policies. We also believe that there are significant examples of good practice, and that a move to electronic records may eventually be welcomed by social work practitioners and managers. We have gained great respect for the commitment of the majority of staff at the pilot ICS sites. However, our evaluation raises serious reservations about the design and use of ICS in its present form and we believe that the ICS has yet to demonstrate the degree to which and how it is fit for purpose. Certainly the task of rolling it out nationally in its present form is huge and should be incremental.

Our evaluation reflects the profound problems all of our sites experienced in using the ICS. Two never really implemented in the timespan of the project, one did so in one area and decided not to do so in others, one had workers that made many criticisms of the system, changed their forms and are changing their supplier. This evidence, in itself, provides useful information both about the difficulties of implementing the system as well as suggesting some of the ways in which the difficulties might be addressed. The report on which our recommendations are based should therefore be of use to the local authorities who are currently setting up their systems, as well as suggesting changes that might helpfully be made to the system and the support of it at Government level.

1. Practitioners and managers felt that the prescriptive nature of the system and the splitting into sections of the exemplars meant that it was difficult to grasp the key features of a case or to track its coherence. In many cases the ‘family aspect’ of the case got lost. Also, the same information was repeated across the exemplars.

We recommend a review of the design of the exemplars, so that:

• The exemplars are simplified, shortened and made easier to complete, for example by reducing the amount of information requested and the number of sections and tick boxes.
• The facility is provided to give a pen picture of the family and a key case summary of the work being undertaken on each exemplar.
• Pathways are created to enable irrelevant questions to be bypassed or a short version completed, while at the same time some fields, essential to identifying the case, are mandatory.

2. Practitioners and managers felt that the inflexibility and standardisation within the exemplars meant that they were not appropriate for all cases. There were particular difficulties in their use with children with disabilities.

We recommend that:

• Consideration be given to designing, enabling and endorsing a more flexible use of the system.
• Exemplars are .developed that are user-group specific, in particular the core exemplars for children and young people with disabilities. This would include providing opportunity for information to be included in different formats, for providing designated space to record specific care needs and caring strategies and for recording alternative milestones and achievements for disabled children

3. Practitioners were concerned that the length of the records, their format, the language and the underlying assumptions made it difficult to promote user involvement. It is possible that the ICS could lessen user’s involvement in decisions that affect them because of its inflexible packaging and a probable marked increase in time demands on practitioners.

We recommend that:

• The implementation of the policy aims of the ICS that relate to the involvement of service users is reviewed to ensure that the system can deliver the aspirations.
• Consideration is given to the language used so that families can understand them.
• Authorities should, as a matter of course, inform families about ICS, specifically that their records will be stored electronically and who will have access.

4. The evaluation suggests that one reason completion rates go unrecorded is that the same time scales are unrealistic or inappropriate for all cases.

We recommend that consideration is given to also making possible more flexible time scales for their completion, or alternately that the time scales are kept but the criteria for completion are relaxed. (In some cases the full information required to complete the task is simply not available within the time limits.)

5. On our sites there were serious difficulties in inter-agency communication and information sharing, in developing interfaces with other agency databases, and consequently of ‘early-warning’ of risk to children.

We recommend that:

• The implementation of the policy aims of the ICS that relate to inter-agency information sharing and the systems put in place to enable it are carefully reviewed.
• There should be clarity as to how information sharing will be promoted, funded and prioritised across health, education and CSSRs.

6. A number of difficulties were encountered with the operational features of the ICS. Continuing dual paper and electronic record systems are problematic.

We recommend that urgent attention is paid to the operational features of the ICS to enable these problems to be addressed.

• It should be possible to email records, sign them electronically and access them remotely.
• Provision of lap tops is indicated.
• Inter agency and service user input should be enabled, consistent with data protection and maintaining a secure system.
• Consideration should be given to enabling outputs to be created from inputs to enable exemplars to be printed out to share with agencies, such as courts, with children and families and for particular events and procedures, such as initial child protection conferences.

7 Problems arose on our sites from the unclear balance between central government lead and local autonomy. These included a lack of ownership of the ICS, uncertainty about local modifications of the exemplars and exacerbated the technical difficulties in using the system.

We recommend that a process of consultation within the social work and social care community is set up to promote ‘ownership’ of the ICS and consideration of its fitness for purpose. The terms of reference would cover a review of:

• The model and conceptualization of social work practice within the ICS.
• The relationship between the core electronic record and the exemplars, and the intended relationship between electronic and hard copy files.

8 The present arrangement for commissioning ICS package providers is the cause of serious difficulties. On our sites the Local Authorities became bound into expensive systems defined by their suppliers which did not always meet their needs, but which they were unable to alter themselves. While we do not recommend a single national provider, nor centrally agreed contracts, we believe that the emphasis on local autonomy may have contributed to

1. an inadequate sharing of delivery expertise;
2. a risk that future national communication and exchange of information between children’s agencies will be fragmented;
3. difficulties in monitoring any unforeseen delivery problems; and
4. lack of transparent means of estimating true overall national costs, brought to our attention by the coverage of difficulties in the delivery of NHS IT contracts.

We recommend a review of the commissioning policies regarding ICS package providers.

9 The need for social work staff training in IT skills and in the conceptual underpinning of the ICS was consistently revealed in our evaluation.

We recommend that training and support for all staff using the ICS is embedded in the CSSR and continually updated, and that:

• A focus on the use of analysis and evidence in recording practice is promoted.
• The particular needs arising from the more diverse and special situations in the disability teams, and in work with children and families from ethnic minorities are addressed.
• The training needs of administrative staff are addressed, including addressing responsibility for inputting data.

10 In conclusion, the evidence from the evaluation suggests that the ICS has yet to demonstrate the degree to which and how it is fit for purpose. Implementing the system in its present form is a massive task, and progress toward a satisfactory system is likely to be slow and incremental.

We recommend, therefore, that further progress takes into account our recommendations for change, and that the ICS is not extended to other services for children before monitoring of the issues identified above has been undertaken.

SECTION A

THE ICS EVALUATION

Chapter One

Introduction

The Integrated Children’s System (ICS) is “a conceptual framework, practice discipline and business process, to help frontline social care staff undertake work with children and families in a systematic manner, and to enable practitioners and their managers to collect and use information systematically, efficiently and effectively” ( DfES, 2005). It has been developed by the Department of Health and the Welsh Assembly Government (WAG) and is now the responsibility of the DfES and WAG. The intention is to bring the ICS into general use in England and Wales by January 2007.

The ICS involves a single approach to the key processes of assessment through to review based on children's developmental needs, parenting capacity and family and environmental factors (p2, DfES, the Integrated Children System, 2004). It applies to all children in need. It provides a common language to describe children’s need and circumstances, to be used by all professions and to facilitate information sharing within and between agencies concerned with children’s welfare. It provides a standard set of tools for gathering the information necessary to plan for children at individual and strategic levels. It is designed to be implemented electronically as the core of the Electronic Social Care Record (ESCR) for children, and will form part of the overall performance assessment of CSSRs.

The ICS is designed to help social services managers and practitioners working with colleagues from other agencies to improve outcomes for children in need and their families. Key aspects are that it should be understandable to the children and families and informed by them, that it should improve multi-agency working and that it should be implemented in electronic format.

It comprises three elements:

- the already developed assessment records and scales of the Assessment Framework (AF) and Looking After Children (LAC) materials for assessment through to review

-a set of data requirements for children’s social services, derived from individual children’s records, to provide information common to all agencies involved with the child

-records or exemplars for practitioners to use to gather and record information in a structured way and that can be transferred to other records/reports.

The exemplars should support management in monitoring children’s progress over time and provide data for corporate planning locally and national statistical returns. Information derived from the ICS is defined in the Children’s Social Services Core Information Requirements for the purposes of supporting the development of electronic systems. The exemplars form the basis for designing front end software to assist social workers to collect, organize, retrieve and analyze information about cases, and they will be part of the e-social care record. The hope is that they will capture and encourage best practice, ensure that appropriate information is collected and analyzed and provide information for managers at local level and Government at national level. Features of the ICS include the following:

• A focus on outcomes for children in relation to achieving their potential and monitoring their progress
• A coherent set of processes, from referral through assessment to review
• Processes which are applicable to all children in need and are rooted in an understanding of children development
• A common language which can be used by all professional processes which facilitate information recording , storage, retrieval, analysis, updating,
• Aggregation and sharing

Policy Context

The development of the ICS is taking place alongside a number of other projects and policy initiatives to re-shape local services for children and families, including the National Service Frameworks for Children in England and Wales, the Common Assessment Framework and the Information Sharing Index. These initiatives highlight the importance of joined up working and systems which can aid appropriate information sharing. The scope of the ICS is intended to take these developments into account.

Background

The Integrated Children’s System marks an important stage in Government policy to improve outcomes for children in need and forms part of the Every Child Matters: Change for Children agenda in England and the Children and Young people: Rights to Action in Wales.

The ICS was first signaled in the Government’s response to the Waterhouse Enquiry, Learning the Lessons (DOH, 2000). In part it is a response to research suggesting that too much social work attention was focused on risk and too little on a child’s needs (Messages from Research, 1995). Equally it is a response to inspections and enquiries that have found deficiencies in social workers’ assessments and in the way they record and communicate information, (eg.Laming Enquiry, 2003). Its development brings together the Assessment Framework and the Looking After Children System (LAC) The LAC materials (DOH, 1995) aimed to set an agenda for good parental care, to promote partnership planning by health and welfare agencies and to introduce the concept of outcomes into everyday social work practice. Additionally they provided the basis of a system by which the information on individual children could be aggregated and used strategically as management information to assess outcomes (Ward, 2001).

The subsequent development of Working Together (2000), and the Assessment Framework (AF, 2001), took into account research on the LAC materials and the need to further their usage for all children in need. The resultant AF records, for assessment through to review, build upon the need for a child centred, inter agency approach to assessments and intervention taking into account the wishes and feelings of the child. They are in wide use across England and Wales.

The first study of the implementation of the Assessment Framework in twenty four English councils (Cleaver and Walker, 2003), reported a number of factors which affected its implementation. These included the existence of collaborative arrangements across agencies, the level of theoretical and technical knowledge of the staff and the availability of resources, such as computers and software. The study identified the importance of building blocks, of senior management commitment and of strong working relations inter and intra departmentally.

The findings from this study echoed a number of those reporting on the LAC system – that practitioners were uncertain about the contribution of the records to best practice, that training and resources were insufficiently developed and that managers lacked the IT facility to aggregate the recorded information for data analysis purposes (Bell, 1999 Wise, 2003). The research was common in finding that the key elements in their implementation were an understanding of the background to the development of the materials, the role of training, the commitment in agencies of key managerial staff and the need for more resources such as information technology and additional clerical help. More recently, SSI inspections of children services have reported continuing concern about how well the AF has been introduced and about the poor quality and reliability of assessments of some cases in all LAs . A recent study examining information outputs (Gatehouse et al, 2004) confirmed the importance of good quality information for effective assessment and decision making. However, the research into the core reporting requirements for social services found that most LA information systems were designed for recording rather than retrieving information.

Early findings from the pilot study of the ICS funded by the Treasury (EBPF), set up to assess the development of a multi agency approach to assessment, found that implementation across social service departments in England and Wales was proving to be more complex and difficult than anticipated, partly because the new system needs to cover all areas of children’s services provision, and because it requires changes in practice and in the use of electronic information systems. However, a strong sense of inter agency cooperation emerged in the three pilot authorities researched.

Evaluating the ICS

The ICS is a highly complex system. Problems in evaluating it and its use arise in part from this complexity. The central concerns of our report are to understand why the system works well or badly as seen from the perspective of social workers and managers, and to assess the relationship between good information collection, the use of the ICS and best practice with children and families. Further questions arise about the use of ICS with different groups of children and young people – especially those who are socially excluded, those with disabilities and those from ethnic minority groups – and the degree to which it can enhance the social work process and the involvement of families in that.

Structure and authorship of the report

The report is built around the three sub studies that comprised the evaluation, and also draws upon the information acquired at site meetings and site visits by members of the research team. Two of the three substudies are themselves further divided, as can be seen from Table 1.1. The aims and methodology of the substudies are described in Chapter Three and the researchers with primary responsibility for conducting those, and writing the substudy reports, are outlined in the Table below.

Implementation and the 3 substudies	Principal researchers
Implementation	Bell, Dyson
Audit study: focus groups, questionnaire survey download study, record study, time study	Sinclair, Clayden Dyson, Bell
Process study: focus groups, technology and case studies service user interviews	Shaw, Barrett, Walker, Ashworth, Rafferty
Disability substudy	Sloper, Mitchell

The report is the product of extensive collaborative work over the period of the evaluation from the above team over the duration of the research contract, from April 2004 to August 2006. The fieldwork was conducted largely in the latter stages of the project owing to the delays in implementation on the four sites selected by the DfES and WAG. The situation of the sites presented here with regard to their implementation of the ICS is based upon the information we were given at the completion of this project at the end of August, 2006.

For this report we have chosen not to submit substudy reports separately (with the exception of the Disability Substudy which was completed earlier this year and has already been submitted to the DfES and WAG). In order to make clear the specific nature of the individual substudies and their relationship to each other, the substudies are presented in separate Chapters, with the connections between them drawn out at each stage and in the conclusion.

Chapters One and Two draw on the progress reports and site visits carried out during the evaluation. They set the scene, by describing the background to and policy informing the development of the ICS on the four sites which piloted the ICS and which we evaluated over a thirty month period from 2004-6. Profiles of the four sites- two in England and two in Wales are then outlined, with a pen picture of the process of implementation that they separately underwent. An account of the state of the ICS in each of the sites on completion of our research, August 2006, is provided, followed by a brief summary of the characteristics of the evaluation common to them all.

Chapter Three then outlines the aims of our research, the limitations imposed by the implementation problems and the research methodology. The next Chapter explores in detail the implementation process on two of our sites, focusing on a case study of the authority which was the most successful in implementing the ICS and describing the detailed technology study which explores technological issues in two other sites.

We then move on, in Chapters Five and Six, to look in detail at the views of the social workers and managers on the ICS, and the issues they identified. The study of the records is taken forward in the download study (Chapter Seven) which analyses the aggregate data from the ICS exemplars which the sites provided us with. This is followed, in Chapter Eight, with an analysis of how the exemplars were used by the practitioners on our sites, and with a study of the time taken to complete them (Chapter Nine). The data reporting on the use of the ICS with children and families from BME groups is in the appendix.

Chapter Ten discusses in depth the evidence regarding the relationships between the ICS and social work practice. The last Chapters, Eleven and Twelve, bring together what we learned from the voices of service users and carers from the Process Study and the Disability study. Chapter Eleven reports on the interviews with the young people and their carers carried out as part of the Process Study, and Chapter Twelve does the same in relation to the understanding of the ICS of parents and children with disabilities.

The conclusion pulls together the main themes and discusses the issues we think are raised by the experiences on our pilot sites during our evaluation, and presents our recommendations. Our main findings are summarized in the Executive Summary at the beginning, where the recommendations are also located.

Chapter Two

The Pilot Sites – an Overview, Progress and Common Characteristics in Implementation

The Pilot Sites

The four pilot sites we evaluated were selected by Government before our evaluation started: three English sites by the DfES and two Welsh sites by WAG. One English site withdrew at an early stage and was not replaced.

In this chapter we provide an overview of each authority involved, followed by a brief account of their progress at the end of our study in August, 2006 and the common characteristics of their implementation of the ICS.

1. Authority A – a southern English authority

Authority A is a large, mainly rural area in southern England with a densely populated coastal strip and some urban concentration around towns in the north-east of the County. These areas are adjacent to and feed the employment needs of an international airport.

The population comprises about 750,000, 95% of whom are White British with some intense urban concentrations of population. Although the total population from Black and Minority Ethnic (BME) groups is relatively small (25,602), below the national average, some wards have some of the highest BME populations in England.

Children’s support services (formerly social services) are delivered from 7 locality offices and a number of specialist teams, including 3 Leaving Care Teams, 2 disability teams, 3 Family Placement teams and an Adoption and post adoption support team. A Child Asylum Team deals with unaccompanied minors arriving at the Airport.

There is a team of Independent Reviewing officers and one of Child Protection Conference Chairs. Teams are mainly divided into Assessment teams and then generic teams carrying responsibility for child protection and Looked After Children.

Table 2.1

The number of children and young people registered with Children’s Services on 30 June 2006

Children on the Child Protection Register	337
Looked After Children	762*
Total receiving services	3426

*The numbers of LAC fluctuates as there are a percentage of children arriving at Gatwick airport who are not seeking asylum, but are not eligible to remain, and are accommodated pending being returned to their country of origin.

The authority uses a small to medium sized software provider, well established in the field of social care. It had started the implementation of ICS before this evaluation commenced, introducing ICS in one of the disability teams in December 2003, and in one of the care management teams, a small locality office for children and families, in April 2004. All exemplars were built in to the system excepting the Care Leavers Pathway Plan and the Adoption Plan. At this stage (2004) there was a clear plan for the roll-out of the ICS simultaneously into the rest of Children’s Services across the County. However this never happened for a combination of practice, change management and logistical reasons. By March 2006 the Local Authority had implemented all exemplars, with the exception of the adoption plan and closure record, in one of seven care management sites and one of two Disability Teams. The system remains up and working systematically in those teams. There are still insufficient resources in the implementation team to extend the roll out. Every social worker has a PC, and was given basic training

To date (August 2006), ICS rollout has not progressed although there is now a plan in place to do so. One of the main sticking points has been the child protection module which was not deemed ‘fit for purpose’. This has been considerably re-worked and re-specified with the supplier. The format but not the content of the core assessments and the APR’s have been amended by taking the questions out of the document and putting them as an appendix. This is said to be much more useable. It is expected to go live in February 2007 and the roll out will then proceed.

2. Authority B – a Welsh authority.

This authority consists of both urban and rural communities, with a population of 130,000. The children’s profile comprises 18,000 aged 0-11yrs, 7,000 aged 12-15yrs, 4,700 aged 16-18yrs. Birthrate is similar to Wales overall but with birth rates falling slightly less than the national average.

The authority has increasing diversity in its culture, language and ethnicity profile, not as yet reflected significantly in the needs of its Social Services service users, but is a factor in providing for learning & general social care needs, particularly Welsh language-based provision.

As from April the delivery of Social Services has been provided through a new organizational arrangement. The authorities Children and Young People Service provide for education, social care and social welfare services through its three interconnected departments: Learning & Achievement; Prevention & Inclusion; Safeguarding & Support.

The latter department primarily provides the statutory services to children in need as such, and is responsible for implementing ICS. This is organised under a Chief Officer and Service Managers into 6 social work teams; 1 for assessment; 1 for children with disabilities; 1 for LAC; 2 for family support; plus a Family Placement team (including Adoption); Child Protection Coordination & IRO service; and the Children’s Home provision.

Table 2.2

The number of children and young people registered with Children’s Services on 30 July 2006

Children on the Child Protection Register	41
Looked After Children	115*
Children in need	373

* excluding short breaks

This authority uses a medium–sized software company and implemented the ICS in August 2004 in the Assessment, Family Support and Disability teams. From the outset this authority has experienced major problems with the functionality of the system, including lost data, deflective cloning, etc. Throughout the thirty months of our evaluation, only the Referral & Initial Assessment were recorded electronically, the remainder being Word attached documents which are not self populating. They await the fully working version of Software from their provider to be able to input exemplars directly.

Staff have the computer resources and training required to utilise a database system. At the point the evaluation concluded the authority were reviewing the exemplars in order to make them present on their system as advantageously as possible, and modifying content only where this would fit their business processes better. A project consultant was overseeing this work. The proposal was to partially implement use of ICS exemplars pending provision of the computer programme solution. The timetable for full implementation remained open to change, with a working version due in September 2006.

3. Authority C – a Welsh authority

This authority is a relatively small county, covering an area of 49,891 hectare, with five main population centres. Much of the county is rural, containing a number of small towns and villages. The south and west of the county is quite sparsely populated. The economy has changed dramatically over the last twenty years from being industrial, based on coal, steel and textiles to basing the economy on the manufacturing of products within the aerospace, food, paper, chemical and automotive sectors. Agriculture still features significantly. The tourism sector also makes a significant contribution to the economic wealth and employment of the county.

The population, 148,594, has grown markedly over the last 10 years, making the county the sixth largest in Wales in terms of population with a population density of 342 persons per square kilometre. This is more than double the average population densities for both North Wales and for Wales as a whole. 21% of residents use the welsh language. The ethnic population comprises 0.8% of the total, the main ethnic minority group being Chinese.

The Children’s Services Department is located within the Education and Recreation Directorate, the largest and most diverse of the eight Directorates within the Council. The Directorate is structured into four Divisions with the Strategy and Performance Improvement Unit working alongside the Divisions driving whole–Directorate activities and supporting Heads of Service and Service Managers. The Department either directly provides services for children and young people or work with partners and stakeholders to commission services for clients. However, the services which are provided fall into three main areas of activity - child in need, child protection or looked after children.

The Duty and Assessment Team is responsible for undertaking Initial Assessments on referrals at first point of contact or re-referrals of previously closed cases, after a period of time. The team conducts all Section 47 investigations on new cases, Initial Assessments for child protection referrals and all requests for Section 7 and 37 reports (unless the case is already open to one of the Locality Teams). Locality Teams provide services for the North and South of the County respectively. The primary role of these teams is to complete Core Assessments and provide longer term support for all children on the Child Protection Register, those children who are looked after by the Local Authority and children in need and their families. Locality Teams also complete Section 47 investigations and Section 7 and 37 reports for existing cases.

The Children’s Integrated Disability Service (CIDS) provides a single simple route for families to access information assessment, assistance and services. Referrals are made directly to CIDS (excluding child protection). A multi-agency referral panel ensures the child meets the criteria and identifies the most appropriate worker. The service seeks to provide a holistic approach to the needs of disabled children and their families. Care co-ordinators and social workers aim to address all family issues. As part of its aim to open direct communication channels, a Senior Management Project Board (multi-agency) and a Disability Project Group have been established, both of which feed into the Children and Young People’s Partnership.

Table 2.3

The number of children and young people registered with Children’s Services on 31 March 2006

Children on the Child Protection Register	86
Looked After Children	187
Children with Disabilities	124
Children in Need	739
Young People on the Leaving Care Register	78

A number of the children and young people registered with the Department will appear in more than one of the above categories.

This Authority currently have an in-house software system which was built upon systems already in place. Implementation of the ICS begun in August 2004 across all teams. Initially the implementation was phased, but from March 2005 all exemplars are being used across all teams. From August 2005 a number of exemplars have been modified. The Authority is currently tendering for a new system.

4. Authority D – a northern English authority

This large urban city is a major port and an industrial centre in the centre of a metropolitan county including other towns. From a population of 269,500, 90% are white British and 18,865 BME. The largest ethnic groups are Pakistani (1.9%) , Indian (1.2%) and Bangladeshi (1%). The main employment now is in the manufacturing industry and the wholesale and retail trade. Forty five percent of the population live in rented accommodation. The population includes 64,000 children and young people aged under 19.

Table 2.4

The number of children and young people registered with Children’s Services on July 31^st 2006

Children on the child protection register	258
Looked after children	454
Children in need	2600

The area children’s teams are based around 5 sites. Three are area teams, with each site having a duty team and two long term teams. There is also a children with disabilities team and a family support team.

This authority uses one of two major social care software providers who currently serve nationally approximately 35% of the Local Authority Social Care software market. Their intention from the outset was to ensure that the software provider delivered a product that met the needs of ICS and of other SSD requirements, such as linking ICS with the ISA and the core data base. Additionally, the authority is attempting the complex task of integrating ICS with FAME, ISA and the authority’s educational database. The authority is a pilot Children’s Trust.

The initial plan was for a gradual small scale rollout, with training running alongside. The council have therefore approached the process of implementation incrementally. However delays in implementing the ICS at all were substantial because of continuing problems with the software provider, and they have continued through the time of our evaluation.

Eventually, implementation of the Referral, Initial and Core Assessment exemplars begun in the Disability Team in July 2005. In the last few months, ICS has been rolled out into 2 Customer Service Duty Teams in the East and West of the City, and the plan is to start roll out to another Customer Service Duty Team in the week commencing 21/08/06. The following 8 exemplars are in use: Contact Record, Referral and Information Record, Initial Assessment Record, Section 47 Outcome, Initial Conference Report, Strategy Discussion, Core Assessment (Pre-Birth to 12 Months) and Closure Record. The remainder of the exemplars are available but so far have not been utilised.

The authority has substantially modified the shape and the order of the DfES exemplars while ensuring all essential data is collected. The software provider is also developing a toolkit that can be used for all applications of their system (Adults and Children) in which the user could define questions and fields.

Currently 75 of the 400 Children Services users are trained in using ICS exemplars. Approx. 98% have a computer on their desk. When all Duty Teams are trained in the use of the new system the plan is to roll out ICS into the long-term teams and to use the majority of the exemplars. The biggest hurdle currently is the resources available. Teams are stretched to full capacity and are finding it increasingly hard to meet the timescales set out by DfES. The process change has been hard to handle, as the implementation team has been unable to support all area teams in an appropriate way. However, they are confident of meeting the January 2007 deadline, as Senior Management are becoming increasingly aware of the situation.

Overall Progress of Implementation in the Pilot sites.

The research project began in April 2004. Three English sites had been selected by the DfES and two Welsh sites by WAG. One English site withdrew at an early stage and was not replaced.

The process of implementation, during the thirty month timescale of this evaluation, has been characterised by delay and patchy implementation. Our project plan assumed a six month period (April to October 2004, see Memorandum to stakeholders, March 2004) for the pilot sites to implement their systems. However, by April 2005 no site had the system fully up and running. All were still in the process of implementing the ICS in varying degrees and at different rates. One authority (C) developed its own in-house system (which will in time be replaced by a software provider). The other three sites purchased different IT systems to provide a base for running the ICS system. All the systems were designed to provide the core database for the authority, the ICS being populated from that data base. All hold information about all service users on the data base for both children and adult services.

The initial phase on the sites was dominated by the tasks of installing and testing the software and preparing staff to use the system. All used a form of project methodology, identified a lead manager and involved their IT staff. However, the variation in implementation was considerable. This appears to be linked to the sophistication and degree resources were available to implement the system, to difficulties and time delays in the procurement and installation process and to features of the software providers.

Another factor in delay is that, while all of our pilot sites were developing ICS systems integral to the databases used by social services, they were at the same time considering compatibility with existing databases within their councils as well as addressing the restructuring of Children’s Services.

Management and technical issues have also arisen recently from the introduction of the other systems, such as the Common Assessment Framework and the compatibility with ICS of those recording systems This impacted upon the implementation and rolling out of the ICS, one authority (D) deliberately delaying until compatibility with other data bases could be established.

At the conclusion of our fieldwork, in May 2006, only one site had full implementation of all exemplars across all teams. The situation in the pilots at the end of our fieldwork was as follows:

• Site A had implemented all exemplars with the exception of the adoption plan and closure record, in one care management site and one Disability Team
• Site B had not received a working application from its software provider and was using the system as word attachments, thus losing the electronic integration.
• Site C had implemented all exemplars right across Children’s Social Services through its in-house system.
• Site D had implemented the Referral and Information record and Initial and Core Assessment solely in its Disability Team. Having experienced many problems with its software provider at this point it had just received a working ICS application from its suppliers.

Common characteristics in implementation:

The process of rolling out and managing the ICS in all four pilot sites has had consistent characteristics.

Delays

In all sites the experience and process of implementation was closely related to the development of an IT system, and all but one of our authorities experienced major delays linked to two aspects of the implementation process- issues with software and scarce resources.

Relationships with providers were important and negotiations with suppliers time-consuming. In working with external software houses, there were major difficulties in securing a working system. Basic problems included self-populating documents, saving documents, producing copies of the records that are useable by all stakeholders, lost data, defective cloning, defective date notification, security errors and faulty audit practice functions. The implications were a loss of confidence in the system.

Management

The management of the ICS projects by those involved has been committed and methodical. In all four sites the scale of the task of introducing ICS was recognized early in the process. All authorities identified a lead manager for ICS, although none of them were solely devoted to ICS. Not one maintained this responsibility (for a variety of reasons) during the two years of our evaluation, so that, by the time we had completed our fieldwork in May 2006, all the sites had a different manager in post. This meant the loss of experience and consistency while new managers took on the complex developmental tasks.

All sites approached the process systematically using a Project Methodology (for example, Prince 2) which helped, through the process of exception reporting, to identify difficulties. All established the importance of making sure that the system they had procured could support the ICS and set up ‘test beds’. These regularly checked the system and identified key issues.

Each site has used some form of project planning methodology and convened a project board to monitor progress. All set up an implementation group which met regularly, comprising the lead manager for ICS, the lead IT manager for ICS and some key practitioners. All involved their IT staff (both within the social services and the corporate IT departments) in the process. This group became the core management of the implementation project and was a crucial driver. The employment of external project management consultants played a lead role in the core group advising and progress chasing in two sites

Ownership

The need for ownership of the ICS project at all levels was also a common feature. In particular the commitment of senior management to understanding the value and concepts of the system and seeing the process through to completion was key, as was the provision of resources. All of the sites reported that implementation involve a delicate managerial balancing act in which preparing and warning staff of impending changes had to be balanced against losing their commitment and goodwill if the subsequent implementation then had problems.

All sites encouraged ‘product champions’ in the teams implementing ICS - staff who could use the system and support their less skilled colleagues. As an example, Site C gave their champions baseball caps, signifying when that member of staff was available for consultation on ICS matters. All sites developed a means of enabling practitioners to report back issues, for example by postcards sent from team members to the implementation group. As described by those responsible, the system support from IT colleagues, who were also ‘social care literate’, was important in redressing the balance between IT development, which social workers needed to be cognizant with, and the demands on social workers which IT staff needed to be alert to.

Access, security and data protection issues.

All sites experienced a number of issues relating to the transfer and receipt of web based information, including plans for inter-agency sharing and definition of levels of access within the authority. Data protection issues arise because single data entry items are used in multiple contexts. All sites recognized that a hierarchy of access based around teams and roles would emerge which could be problematic in managing data protection, for example.

Amending the exemplars

A common theme in all sites was ongoing discussion and consideration of change to the exemplars, varying from simply adding the specific authority logo to substantially changing and or re-ordering the content.

During the course of the fieldwork all four pilot sites modified and adapted the official DfES exemplars. Adaptations occurred at different points in time and in different degrees, some modifying them before staff started using them, whereas others adapted them at a later stage and in a staged manned. In the light of this diversity and potential for ongoing change it was not always clear when social workers were referring to the format of official exemplars or to the authorities modified format and staff were not always aware if or what modifications had been made.

Practice

The emergence of a substantial IT component of social work practice, highlighted by the implementation process, was a recurring theme, and that it felt like more work. One exemplar in particular raised issues- the Report of the Child Protection Conferences, in part because of the difficulty of sharing information electronically in agencies based on paper. The focus of the implementation was on getting the system working for staff; no work was undertaken with children, their families and carers and there was concern that the exemplars are not user friendly. In all sites an acquisition of new expertise marked the work at local level. The roles of both support staff and practitioners were seen as changing, and this led in some instances to openness to experiment.

Training

All four sites agreed that the training should involve two aspects: the conceptual framework underpinning the system, and the IT skills required to use it, reflecting the wide variation of computer literacy in the workforce. All sites developed plans for training practitioners to use ICS, but the amount and timing of training has been variable. Some used the DfES training material (Walker et.al, 2004), while others provided all the training in-house. Staff in specialist teams, such as disability, felt what training they had did not focus sufficiently on their specialist needs.

There was agreement that the timing of training was important and should only take place once the authority had a working system that had been thoroughly tested. However, because of the delays in implementation, and the ongoing changes in staff as a result of recruitment and career progression, there was also recognition that training needed to be ongoing.

Main Points

• The system took far longer to install than has been expected, with long lead in times and patchy implementation.

• The implementation process has reflected a combination of serious problems with software providers and difficulties in integrating the systems with other.

• All authorities modified the original exemplars to fit local circumstances.

Chapter Three

Overview of Evaluation Design

The evaluation was planned and carried forward with the aim of answering a cluster of questions from the perspectives of service providers and users.

From the perspective of both service providers and service users:

• How well do the different parts of the ICS, from referral to assessment, planning intervention and review fit together and promote best practice in direct work with children of all ages and abilities and their families or carers
• Does the ICS help children, young people and their carers from different subgroups, in particular children and young people with disabilities, to better understand the social work process and to make more sense of what is happening in their lives?
• Is the system acceptable to all children, young people, practitioners and their managers and does it help to promote partnership working and information gathering?
• What difficulties have been experienced in implementing the ICS and how could these be overcome?

From the perspective of the service providers:

• What additional time will staff need to complete the records and what are the costs?
• What training needs are identified, including in the use of technology
• What additional technology will be required and what are the implications for councils IT systems?

In this chapter we provide a general overview of the evaluation design. The specific methods for each cluster of research activities are set out in conjunction with the different datasets. The design for this formative evaluation involved three closely intertwined clusters of research:

3. An audit study to assess the system’s records, the time it requires and the response of social workers and team leaders to it.
4. A detailed examination of the central processes of the system from the viewpoint of the key actors involved (children and their carers, parents, and local implementation management groups).
5. A specific study of the system’s impact on disabled children and their carers.

These activities were supported by a development worker who worked with the pilot sites in developing the system and learning from each other and the research, and provided a bridge between the pilot work and the evaluation team.

Audit Study

The central purposes of the quantitative audit studies as originally conceived were:

• To assess how far the system achieves its purpose of generating accurate, reliable data in a framework which is useful to practitioners, enables monitoring and provides management information
• Assess some of the quantitative demands of the system in terms of time
• Identify the resource and other constraints that make it difficult for the system to be fully effective.

The main intended outputs of the audit were:

• An assessment of the degree to which the system passes or fails some simple tests of accuracy and reliability and of the apparent quality of the assessment
• An assessment of the social work time required by at least one stage in the system and hence a discussion of its implications for cost and for the number of social workers required if, for example, many more cases were to receive a full assessment
• A description of social work and team leaders reaction to their experience with the system

Process Study

The focus of the process study was to understand how the ICS system works within a CSSR context. The central purposes of the process evaluation in part overlap those for the Audit studies, and were to:

1. Complete an evaluation of the coherence of different parts of the ICS system; the extent to which the core ICS processes assist information gathering and recording; the role demands on staff implementing the system.
2. Gain clear evidence regarding the suitability of the system for children of different ages and at different stages of the system; and the extent to which the system helps birth parents, carers, and young people come to a better understanding of social work processes and the rationale for intervention.
3. Complete a process evaluation of technological aspects of implementing the ICS, including staff training and support needs; additional technology required to implement the system; and the implications for CSSRs' present ICT systems.

The main intended outputs of the process studies were:

• An evaluation of the coherence of different parts of the ICS system.
• An understanding of the suitability of the system for children of different ages and at different stages of the system.
• Different perspectives on the extent to which the system helps birth parents, carers, and young people come to a better understanding of social work processes and the rationale for intervention.
• An information base to inform decisions regarding future training requirements and also technological resources required by authorities.

Disability study

The main additional intended outputs of the disability specific study were:

• An understanding of the acceptability of the content and application of the ICS for disabled children and for their parents/carers.
• A range of evidence from different stakeholder perspectives about the suitability of the ICS for children with a disability at each stage of the process.

Our approach to evaluation

The broad range of data collection methods included auditing electronic databases and files; structured telephone interviews; semi-structured interviews; postal questionnaires; audio-diaries; documentary analyses; and field logs. The approach we took to the evaluation project has several general distinctive characteristics.

First, it was developed as a stakeholder/responsive evaluation proposal. A tradition of stakeholder evaluation has been very strong in the USA through people like Robert Stake, and involves both an inbuilt consultative dimension throughout the project, and also a responsiveness that allows key aspects of the evaluation to be fashioned and finalised during the evaluation. This gave us sufficient flexibility that, when we encountered serious challenges to the research stemming from delays in the implementation of the ICS we were able to withdraw from some parts of the study and commit freed resources to strengthening other areas of the study. This would not have been feasible if the terms of the design had been pre-shaped and contracted in great detail.

The term formative evaluation is used in different ways, but we use it to distinguish between evaluations that are geared to understanding the processes by which a programme is developed, set up and run, in partial contrast to summative, accountability evaluations where the primary purpose of the research is to assess the extent to which a programme, service or project achieves planed outcomes. Hence, the term ‘evaluation’ is used in this report in two senses - generating or enhancing theory and knowledge about social policy; and instrumentally improving practice and organizational learning.

Third, this is study conventionally evaluative rather than strongly Research and Development in focus. As a consequence, for example, we tried not to offer advice on best practice during the fieldwork. We wanted to know, for good or ill, how a relatively ‘unattended’ initiative would work out in a typical CSSR.

Fourth, it combined both a local and cross-site focus. We were interested in both local distinctive conclusions about ICS and also conclusions that can be generalised across sites. We believe the evaluation has proved able to deliver this aim with some fullness.

Fifth, and moving on to methodology, the evaluation is a mixed method package. This means, of course, that we are using both qualitative and quantitative methods, and also that we are using a diversity of methods within those broad traditions. The broad range of data collection methods included auditing electronic databases and files; structured telephone interviews; semi-structured interviews; postal questionnaires; key informant interviews; documentary analyses; and field logs. This means that we have been challenged by the different kinds of knowledge claims that emerge from different methods of inquiry. We have not assumed that the conclusions that seem to follow from, for example, a focus group, will be 100% consistent with what comes out of, eg, a file audit. This has made the management of the data analysis less straightforward than it might have been. For example, the ‘results’ from some of the focus groups did seem to run partially counter to data from a social worker survey. We have tried to address and interpret these differences in the report.

Sixth, we aimed to achieve coherence to the design of the evaluation by building it around the ‘spine’ of the audit and linked studies. The more detailed qualitative studies were planned as link-able to the spine. This was one area of the research that suffered as a result of the implementation problems at local level. However, by extending our time scale by five months, we were able to undertake the planned analysis of exemplars within the record study, albeit for a smaller number of cases than originally hoped for. The exemplars analysed enabled us to connect up the qualitative data across the three sub-studies as they comprised the records of the children and parents/carers we interviewed in the process and disability studies, and the social workers interviewed in the disability study. The analysis of the exemplars undertaken in the record study was also augmented by the data on the use of particular exemplars from the audit and disability studies, as well as being informed by the perspectives of the managers and practitioners involved in the focus groups and the questionnaire survey. This meant that data from all three sub-studies could be brought together to achieve a multi-perspective understanding.

Modifications to the evaluation design

The planned evaluation design is set out schematically in Figure 3.1a. The actual evaluation as delivered is set out in Figure 3.1b. The evaluation encountered major sampling difficulties associated with implementation delays at site level, which jeopardized this planned coherence. The nature and causes of these local difficulties are multi-layered and complex. However, they bore on our ability to conduct the evaluation in tune with the original plan. The evaluation team faced a major unanticipated challenge where pilot sites needed much longer familiarization with the ICS than had been anticipated. This is apparent by comparing the time line in each figure. The study was extended by five months and all the major fieldwork activities were extended to approximately double the planned time. The implementation phase stretched throughout the project.

The modifications to the evaluation were fourfold. First, there were very large delays in fieldwork. By the conclusion of the evaluation the ICS still had not been fully implemented in any of the four pilot sites. Two of the sites (one in Wales, C, and one in England, A,) were able to deliver the majority though not all of the intended research data. In the other England CSSR, site D, the launch of the ICS was delayed to such an extent that we were only able to collect case-based data for the disability substudy. However, the technology case study which was carried out at that site provided us with valuable data about implementation processes which we were able to link to and compare with the more successful implementation on Site C. Data from the second Wales site, site B, was incomplete or absent at all levels of the study, and it was agreed to abort most of the fieldwork from this site at a late stage in the evaluation.

Second, there was some loss of data quality. We are not able, for example, to advise on the extent to which social workers’ assessment of the ICS is reflected in their completion of exemplars. The extent to which we could undertake standard reliability checks on data has been restricted by the less full links between different datasets.

Third, there was a loss of some planned datasets. This primarily affected the process study, where there was a late aborting of fieldwork at one of the pilot sites and local sites encountered serious difficulties in identifying and then resolving consent for interviews with children and carers. In the event, we were only able to recruit just under a half of the intended sample from sites A and C. In total eighteen interviews were completed. Of these, 10 were with children and young people and 8 were with carers. Two of these were birth parents. Six-paired interviews were achieved. There was also a loss of intended data regarding the specific implications of the ICS for black and minority ethnic service users. This was due in part to the partial local implementation of the ICS in one pilot site (A), where the planned rolling out of the ICS to an area where migrant and refugee service users would have been likely, never took place. The demographic balance of the pilot areas made a significant presence of black and minority ethnic service users unlikely.

Fourth, the project was premised on the assumption that service users would possess first-hand experience and knowledge of the ICS. This was not how children and young people perceived it. The reasons for this are doubtless complex.

• We encountered occasional snippets of evidence that social workers may have sought to ‘protect’ service users from the ‘burden’ of the workings of ICS.
• Implementation management teams may have been preoccupied with service-development level issues and negotiations with commercial providers, and given little attention to the direct consequences for and interests of service users and carers.
• Given the selective rolling out of the ICS, there were inevitably large swathes of agency operations where knowledge and experience of the ICS were absent. Service users would discover nothing about the ICS in those areas of work.
• It is likely that social service authorities saw some element of burden in involving users and carers in the evaluation.

Figure 3.1a Planned Evaluation of ICS Pilot Sites

Month 1 M4 M7 M10 M13 M16 M17 M20

Figure 3.1b Actual Evaluation of ICS Pilot Sites

Month 1 M7 M13 M19 M25 M29

Our presentation of fieldwork delivery issues has stressed the deficit issues. This is not the whole picture. We utilized our resources to compensate and augment the evaluation to exploit those opportunities that arose during the study. This was true in five areas.

1. The sub-study on ICS and children with a disability was augmented, by agreement with DfES and WAG, by adding telephone interviews with social workers to the original plan for only questionnaires for social workers.
2. The work on the technological dimensions of ICS was also augmented.
3. The team carried out an additional data collection exercise regarding black and minority ethnic service users and the ICS.
4. Additional study of the implementation process was carried out in one site, based on access to all local documentation and minutes.
5. Additional key informant interviews took place linked to the technology case studies and the two more active evaluation sites.

These changes to the evaluation have impinged on the planned outputs from the study. The major shortfall is that we have more limited data than hoped for regarding the suitability of the system for children of different ages and at different stages of the system. The major gain is that we have fuller evidence and understanding of the implementation process.

Data has been stored securely. We will liaise with the DfES/WAG regarding the ‘shelf life’ and archiving of the data, for example through the ESRC archives for quantitative and qualitative data, although documentary data studied in the study will not be available for archival purposes.

SECTION B

IMPLEMENTING THE ICS

Chapter Four

Realizing the Integrated Children’s System: the case studies

A central task of the evaluation was to explore the experience and process of our pilot sites in implementing the ICS. In this Chapter we present the analysis from two data sets: the technology case study, which analysed data from two sites with contrasting experiences of implementation – one successful and one unsuccessful; and the ‘success’ case study, which analyses in greater depth the implementation process on the one site which achieved full implementation of all exemplars across all teams in the time of our study.

The technology case studies

The technology case studies provide an evaluation of the technological aspects of implementing the ICS in two pilot sites, Site D in England from an urban setting, and Site C in Wales from a rural setting. The fieldwork for Site C took place in January 2006 and for Site D in February 2006.

Methodology

The methodology comprised documentary analysis (eg. Briefing Paper 8: English Pilots ICS IT Systems – Technical Report by Cleaver, Children’s Social Services Core Information Requirements), observation and semi-structured interviews with key informants. Interviews were conducted on Site C with a member of the business systems team, a planning manager, a freelance IT consultant, four team managers and four social work practitioners; and on Site D with the business systems analyst, a team manager and five social work practitioners from the Children with Disabilities team. A ‘constraints and culture check list’, based on Gatehouse, Statham & Ward’s report (2004), formed the interview guide (see Appendix). The interviews were tape recorded and the transcriptions analysed using Microsoft Word.

The project leaders

The ICS implementation on both sites was led by senior staff. On Site C the project lead was the Chief Planning Officer; on Site D a Senior Business Systems Analyst. Responsibility for ICS comprised only part of their workload, but because it took the majority of their time, other areas of their work sometimes took a back seat. It became apparent that the project lead needed to be conversant with practice issues, while having good IT and business management support.

In-house or external commissioning of information system development

Site C’s implementation was driven by the Chief Planning Officer supported by an independent consultant and an in-house IT team. ICS forms were incorporated into the in-house system. The team instituted minor changes to it, relying on the Council’s IT unit for more complex support. The key reasons why the in-house nature of this system was successful included practitioners’ familiarity with the existing Information System and therefore less need to make adjustment when the ICS templates were added, the availability of IT support and that they were close personally and geographically to the business systems team.

Site D experienced complex problems because their commercial software provider failed to produce a workable system during the lifetime of our study. Their situation is unusual in that the software provider approached them, offering to develop the system for a fixed sum. Site D did not, therefore, investigate other opportunities. Two issues were relevant to the difficulties encountered:

1. Technically it proved more difficult than envisaged to design an IT system on top of an existing database.
2. The informal and somewhat ambiguous nature of the contract and business agreements compounded the difficulties and relationships between the two parties became strained. This was exacerbated when the initial ICS leader had extended sick leave.

At the time of this case study (Feb. 2006), the Referral, Initial and Core Assessment exemplars had been in use in the Disability Team for eight months, and roll out into two duty teams had just started.

Transition issues

Site D, for the reasons given, carried on using paper files. And although Site C were using electronic records for auditing files, by the end of the evaluation both sites still held dual paper and ‘e’ systems.

On Site C, the managers and practitioners on the implementation committee and the independent consultant focused on process mapping and changes to the ICS exemplars. Information was disseminated by management and, initially, supplemented by a newsletter. In the event, the amount of work was more than originally envisaged and only two newsletters were printed.

This Site also operated a system whereby practitioners could send their problem on a postcard to the relevant manager or business systems worker. Both of these staff thought this system worked well. However, the senior manager also felt that establishing the implementation groups at an earlier stage and producing the newsletter regularly would have helped. For their part, the practitioners, while welcoming the system in principle, did not feel involved and reported that nothing was done in response to their concerns.

Site C manager believed that practitioner’s transition to ICS was curtailed by their slowness in adapting to the increased levels of IT, and that social work now involves a cultural change to outcome driven practice. While some of the problems were ascribed to IT issues, the manager also felt that a lot of people were not happy with the layout of the exemplars. Further, IT issues in relation to interface and usage were raised in the weekly implementation groups in the early months of implementation. The biggest gap perceived by the team managers was in relation to the language used on the ICS forms rather than the IT skills required.

Particularly new social workers don’t know how to express what a child’s needs are in words so that it works in ICS. It is like speaking a particular language and practitioners have to understand how to phrase what they are trying to express, as ICS is so …pedantic. For instance they do not understand what is being asked in relation to corporate parenting, parental capacity and environmental factors.

IT skills, training and support

There was wide variation between practitioners from both sites in the level of IT skill and the acceptance of IT in the workplace. Practitioners with more advanced IT skills found the transition to ICS more comfortable. One manager noted a relationship between administration and IT competence, while reporting that effective practitioners had poorer admin skills. Younger social workers were commonly more IT literate.

I think if they [practitioners] are using the system for direct input it’s a way of processing their thinking…..ICS may not simply be a means of ordering practice but of actually changing it - ‘processing their thinking’ –some people do find it hard and they are disadvantaged and if they are sitting wondering what key to press…….

Site C had used technology since 1997 and every team member has a computer linked to internal and external networks. Clear policies were in place and it was seen to be the practitioners’ responsibility to move things through the system. Both team managers and practitioners experienced IT support as good. An IT specialist devoted half a day a week to support and ICS champions went on extended training. Care plan training was not linked to the IT system training and this needed integrating.

A number of practitioners from both sites felt that, although the information system they used was generally ‘fit for purpose’ as a professional tool, they needed additional training to make fuller use of its capabilities, and to address data protection issues.

Impact on recording

Among the most problematic aspects of ICS implementation on both sites was that paper ICS records continued to be used. They had to be transferred to an electronic file following visits. A solution could be to equip practitioners with a laptop or tablet computers so that they could record electronically when receiving the information. However, practitioners in Site C feared that taking laptops on visits would be impractical: they could be stolen and it would be more time consuming.

If you do the core assessment for example, you would just put your notes, if you were there, each section is 10, 15, 20 minutes, just on one question, where if you just scribble little notes you do that back at the office, the family don’t want you there, a visit of an hour, with a computer you end up there for two hours.

Practitioners on Site D were positive about the opportunity laptops would present to work from home. However, of significance is the fact that neither site has a fully functioning web based electronic information system with which to input data remotely. Currently, the electronic information systems used are accessed through a local networked PC. However, on Site D the implementation plan would allow access to ICS and shared folders from any internet enabled computer.

Interagency / interprofessional communication and sharing of information

Although on Site C Education and Children and Families Services are in the same directorate and building, they use two different information systems that are currently incompatible. There is no electronic sharing of information. The multi agency referral forms, for example, are sent electronically to schools and other agencies but they can not be returned electronically for security reasons.

I think we’re bogged down in data protection issues….. we are part of the education directorate and our colleagues who work in special education often work in the same place but they don’t have access to [our system] and we are trying to disentangle that because we think they should have.

Site C also raised the difficulty that ICS court reports were not accepted by the court. Since the Assessment exemplars, Care Plans and Chronologies developed by the ICS are not accepted by courts, they need to resort to paper reports. The problem is compounded by existing guidance for Care Plans (DOH, 1999) which sets out clear guidelines on the content and structure of plans to be submitted to courts in care proceedings, the structure of which does not reflect that of the ICS Care Plan. For similar reasons the ICS chronology is not accepted by courts.

Site D has been involved in Trailblazer pilot projects. They were working to enable a referral entered into the information system to be linked through to Education and Health as part of the FAME project. One manager felt different agendas were operating, and that “nobody knows exactly which way we’re supposed to be heading”. Another team manager supported the idea underlying ICS – “of telling the story once and that should remain confidential but open to certain professionals” – but her experience was that social workers experienced difficulty in balancing the demands of practice and inputting data. She identified problems with resources and felt that, while her team were open minded and willing to try new things, the ICS documentation faced them with repetition and difficulty in recording different levels of assessment, especially for disabled children. However, she also felt that the ICS system had produced a better standard of documentation.

Impact on day to day working

• Parallel paper records

Team managers in Site C welcomed the policy of replacing paper records, but said they were a long way from achieving that. Scanning was not being used in practice. They felt that unless the electronic system reflected the whole case it was pointless having an electronic system at all.

• Time consuming

While the team manager at Site D felt that familiarity with the forms would reduce the time taken to complete them, concern about this featured on both sites. On Site D there was debate about the role of admin in inputting data, but concerns were expressed about their training and familiarity with the system. On Site C practitioners reported difficulties in amending documents, in copying and recopying documents, and in getting logged out of the system to answer the telephone, for instance. Mention was made of the time needed to complete a care plan or a child protection plan. The security measures in place also took time, as they had to enter three passwords to get the system to start. Printing also created problems because this required the practitioners to come out of the council system and go back into the ICS database.

• Design

Issues of design were also raised on both sites. Some practitioners commented that the ICS was devised without practitioner input: “it’s incredibly repetitive, bureaucratic and user unfriendly”. Some found the forms very prescriptive. They said it was not always clear where to put information, and that the forms did not lend themselves to the level of analysis required – especially on complex cases.

Disabled practitioners experienced difficulties with the inflexibility of the ICS because of the 10 point font and because the spell checks didn’t work. Further, while the screen could be magnified, when printed the small font was difficult to share with families.

Despite the problems experienced in usage, the managers agreed that ICS had the potential to enhance social work practice, and that it was less woolly then the previous system. One practitioner said the core assessment gave a focal point which improved on previous systems.

Some of the core assessments I have seen from other teams have been absolutely one liners and it tells me nothing….Whereas this at least ….gives people a focal point of what information is required. ….It does repeat itself though.

• Use with families

The reports were seen as “incredibly user unfriendly”. At Site C the IT support unit was designing a service user friendly output using colours, though colour printers were not available to practitioners. That problem is currently being addressed.

• Surveillance and time limits

Two particular and linked concerns were raised by practitioners - that the system might guide the way they organised their work and that it might be used as an instrument of surveillance by managers and central government. For example, the deadlines for completing assessments could mean both that more urgent tasks were sidelined and that practitioners might complete sub-standard records to meet deadlines. Practitioners at Site D felt that the PIs did not take into account that they were a specialist team dealing with long term cases.

…in some ways the computer and the statistics are controlling us, it’s not the other way around, because the government look at a tick in a box, don’t look at anything else, and say ‘they’ve only done 4% work in that team, everybody else is doing 13%’. We’re probably doing 24% because of our specialised team and our work is long term, it’s not just in and out quick, it’s long term work.

The ‘success’ case study

As Chapter 2 and the previous section have identified, the only site of our four to successfully implement all the ICS exemplars across the authority in the time of our study was Site C. This is a small Welsh county, predominantly rural and quite sparsely populated. The Children’s Services Department is located within the Education and Recreation Directorate, comprising a Duty and Assessment Team, two Locality Teams and an Integrated Disability Service (CIDS) . The Authority developed an in-house software system in August 2004, phasing it in across all teams by March 2005.

This analysis is based on an analysis of agency documents, minutes of nineteen ICS implementation management group meetings held between summer 2004 and September 2005, and an interview with a key figure in that group. The minutes comprised summaries of issues raised, discussion, lists of action plans, targets set, targets met since the previous meeting, and so on. Other group meetings reported on in the documents included a multi-agency group, the department’s management team, and workshop and training groups.

The implementation management group

In total, twenty-nine people attended the group during the study period, representing all the ICS interests in the authority. Membership comprised ICS specialists, service managers and practitioners, the chair being an ICS specialist who acted to bridge the differing specialisms. The group initially had sixteen members. While lead figures remained constant throughout, a statistical analysis of membership and attendance suggested the group had less success in embedding new members. Initially they met weekly; towards the end of the study, monthly

The authority rolled the ICS out in three phases, implementing selected exemplars across the whole authority. The work appeared well managed. There was some slippage – for example Phase II roll-out was first scheduled for late November, 2004 but took place in March 2005. But there is a sense of the group being in reasonable control of the process.

The Story Line

The tone of the early minutes is positive. The focus is on information seeking, changes to working practices, developing a communications strategy, sharing protocols, and the role of other agencies. An early decision was made to set up a problem-reporting mechanism so that staff could report to one of the group members, thus aiming to log all issues and ensure they gained attention. The language of service outcomes is present in early minutes:

The ‘standard’ performance key indicators have now been identified and base information will be captured over the next four weeks. A to review with B¹ to identify any further requirements for base information capture. This information will be used to evaluate the improvements/impact of the ICS implementation. (090704)

• Communication issues

An early concern was the relationship with other agencies. This often focused on the referral process where there was resistance from other agencies to the Referral Form. Initially, this was seen as a communication issue - a lack of understanding of what each agency needed to complete. As time passed its links with the operation of the duty and initial assessment work took the fore. Worries about initial information recording recurred.

• Training

Training was another early and recurring concern. Although ICS training was part of the standard induction, a number of staff had missed the training sessions through sickness and maternity leave or were new recruits. The need to offer sessions on an ongoing basis became apparent.

• Modifications and consolidations

The need to make modifications to exemplars emerged early, for example, the Referral Form. As modifications were made, the group was unsure which versions of forms were being used. Numerous minutes express concern that ‘There is some confusion on which referral form version is the correct version’ (081004). The decision to record the removal of redundant forms and previous versions was made, a check which was welcomed by staff as a sign of the close and efficient management of the local ICS project. The complexity of the relationship between the ICS records and paper-based records was also a part of this discussion.

The question as to whether forms could be consolidated or new forms added was clearly taxing. There was an agreement to amalgamate the child protection conference initial report and child protection review into one document, and a decision to review whether the court care plan and the looked after plan could be combined (261104). At the same time there was discussion about whether to create a new contact record.

Another concern was illustrated later, expressing the tension between a national ICS and local practice

“if a [third party] package solution was adopted then the department would have to revert to the base exemplars as defined by the Department of Health. This would be a huge step back for the department and would de-motivate staff as a result of all the lost effort and would remove all of the efficiency gains implemented in the C solution.” (280105)

The following extract illustrates a more subtle process of adaptation. The group members were reflecting on how they both drew on ICS training but also hear, assimilate information and go on to record in their own way:

1^st Participant …the crux of this is that people will hear so much, take on so much but a couple of days later they will kind of, you know, put their own angle on it [sounds of agreement]. Do you know what I mean? [Sounds of agreement] The next thing you have is someone saying, ‘Well I wasn’t told this thing’ or ‘that thing’ or… Then you have ten people operating in very different ways and with nothing, really, to turn to… Except one person saying, ‘well they did say this’ or another person saying, ‘No, they actually said this’.

2^nd Participant: And there is a consistency issue as well [sounds of agreement]. We’re maybe all using it a bit differently to get around problems…

• Reference to service users

Reference in the minutes to service users was rare, only being raised eight months into the project in the context of continued efforts to consolidate forms:

It was agreed that the child protection plan should be used to hold all siblings details and that if a child asks for their specific information this would have to be extracted upon request. (140105)

This minute suggests that user’s access to ICS records sits uncomfortably with the operation of the system. Reference to service users occurs in a long minute reviewing the relationship between the ICS and practice, an extract from which reads:

It was generally agreed that the exemplars in their current form are not usable for service users. The documents are too lengthy and not presented in an acceptable manner. It was agreed that a set of customer facing exemplar versions should be developed which would draw from the standard exemplar being entered into [the local IT system] and could be easily printed and presented to clients. (220405)

This recommendation was accepted in June 2005. In July it was recorded that ‘the definition of the family extracts has now been completed which will provide the social workers with a set of easily readable and appropriate exemplars for parents, carers and children’ (010705).

Themes from the ‘success’ case study

• Innovation and change

It seems likely that it will take time to manage the transition to ICS. However, it will not always be clear what is ‘old’ and can be discarded. This analysis suggests that redundancy is a dimension of practice which requires sensitive and ongoing negotiation. This can be seen in both managing the transition to electronic records and in the modification of the ICS exemplars.

The management of uncertainty is an important feature of the process, and the case study illustrates some ways in which innovation takes place. We have described how the pilot site adapted technology to practice and adapted practice to technology. Such adaptations could be seen as introducing a formulaic approach to practice and playing down the role of professional discretion. At the same time, this case study has demonstrated opportunities for the exercise of discretion in fashioning the ICS to suit local needs.

• External agencies

The role of external agencies in the use of the ICS recurred throughout the case study and was seen as raising relatively intractable problems in relation to electronic information sharing. This raises the question whether the problems encountered by ICS are implementation issues that with good management and sound resourcing will disappear, or whether they are inherent in the design?

ICS and social work practice

The issue of the impact of ICS on social work practice proved to be one of the weightiest issues in the work of this group, surfacing in different ways. For example, the discussion about how to record ‘significant’ and ‘non significant’ events. What counts as significant is a fundamental practice issue, and one on which the options within the exemplars steer decisions. Discussions regarding the relationship between electronic and paper records were also part of this underlying issue of the IT/Practice relationship.

• Training

The importance of rolling programmes to training is highlighted. Ways of embedding the training in the agency were not fully resolved during the period of study.

• Semi-absent themes

Some themes were notable for the thinness of their presence. IT issues were not much discussed, excepting when the agency began the move to an external software provider at the end of the study. The policy aim of the ICS, to drive through an outcomes focus, rarely featured in discussions. The question of how well the ICS worked for children with disabilities and their families was never raised in the group. The interests of service users were also mostly invisible, although when the issue was raised it was given dedicated time and a solution developed and accepted, albeit without service user or carer representation.

Conclusion

Chapter Two has described the process for rolling out and managing the ICS on the four pilot sites, and this chapter has provided an analysis of a successful and an unsuccessful process of implementation.

The differences between the sites in implementing ICS are as striking as the similarities. While implementation raised difficulties on all of our sites, Site C made less heavy weather of it. There were various reasons for this. The project was widely understood. The in house computer system did not raise serious problems. Both the computer company and the IT staff were perceived as responsive and efficient, and there was clear direction by a responsive management. The project manager ensured that decisions were made, clearly recorded, implemented to time and ensured that staff were involved, their concerns addressed and that necessary changes were made.

The delays in implementing the ICS in Site D were substantial because of the continuing problems with the software provider. These have already been described. There were also great difficulties in Sites A. and B. In Site B there were major problems with the IT and a lack of direction for the project. In Site A the key problem seemed to be the perceived prescriptive nature of the exemplars. Necessary changes to the system were slow and costly, leaving workers feeling that they had been dumped with a burdensome and unwieldy system and that no one listed to their concerns.

Our analysis suggests that eight inter-related factors affect the process of implementation.

1. Variations in the extent to which the pilot launches of the ICS were known about and ‘owned’ within each pilot site. This was associated with uncertainty about direction and committee-level informed support for and prioritising of the ICS pilot work.
2. Lack of recognition of the differences for small and large CSSRs in initiating the ICS
3. The fragmentation of ICS technology package development due to the diversity of local contractual arrangements with commercial developers.
4. Variations in the extent to which the technology had been embedded in routine agency work.
5. Dependence of local implementation on a relatively small number of local ICS ‘champions’ and individual leaders. Staff retirement, sickness and post-changes all impacted on the ability to deliver the ICS.
6. Underestimating the demands that a complex system such as the ICS would make upon individuals and the organisation
7. The co-terminus task of considering compatibility with existing databases within their councils. The restructuring that was occurring across education and child care social services increased at a pace during the evaluation, and consumed senior manager energy and commitment. Management and technical issues have also arisen from the introduction of the Common Assessment Framework and the ISA, and the compatibility of ICS with those recording systems. This had an impact upon the implementation of the ICS, with one authority (D) deliberately delaying until compatibility with other data bases could be established.

Our study of implementation in four pilot sites has unpicked some of the factors that contribute to a smooth and effective implementation of the ICS, and these have been described above. Two general points can be made. First, the management requirements of introducing ICS are probably similar to those of introducing any large project. They include clarity in explaining the rationale, good IT training and support for staff, and a businesslike and responsive management. Second, the differing success of the implementation was not associated with differences on the sites in their critique of ICS, nor in the time the system had been running. Indeed the similarities in the views of the participants in Site C, which had fully implemented and Site D, which had only partially implemented, are striking. In both cases there was optimism about the potential of the system but specific concerns about its usage and design and about its effects on the social work role. These issues are further explored through the data collected in the other sub studies and reported in later chapters.

In considering the factors that have led to the delays and problems identified we need to ask whether the problems encountered can be seen as implementation issues that, with good management and sound resourcing, will disappear, or as problems inherent in the ICS, or as both. The data presented here suggests that there clearly are shifts over time. One person expressed the view that ‘in the early days if people have not used systems before, they take a real dip in their belief’. ‘Once its routine’, he suggested, ‘they might be able to see some of the light beyond…the current darkness of, and frustration of, the learning curve’.

This would support the view that, with good management, the problems encountered are transitional and will evaporate with time. Our analysis suggests that some of the problems are technical, and some are resource driven. However, at the same time some of the difficulties described by our respondents and revealed by our analysis may be inherent in the system, for example, in the design or amount of the exemplars. On the basis of the data presented here, we speculate that there are three kinds of implementation² problems.

1. Implementation problems that with sound management and adequate resources can be resolved.
2. Problems that may prove to be implementation problems, but which nonetheless are likely to prove enduring and possibly intractable.
3. New challenges and problems that are inherent to the ICS.

The experiences of two sites in implementing the ICS have been described by our case studies. The problems encountered there were, in more or lesser degrees, also encountered on our other two sites. All four sites had different systems and software providers. While four sites can not be taken to be representative of all authorities, the similarities in their experience suggest some communality. In the following Chapters we will continue to hear the voices of all stakeholders, to examine in detail the use made of the ICS Exemplars and to explore the practice of social work within the ICS.

SECTION C

PRACTITIONERS’ AND MANAGERS’ VIEWS OF

THE ICS

Chapter Five

Practice Participants Assess the Integrated Children’s System

The following two chapters present the standpoints and judgments of managers, practitioners and ICT managers who have been directly involved in delivering services through the Integrated Children’s System. The basis for the first chapter is twelve focus groups conducted at the beginning (ten groups) and at the end (two groups) of the evaluation, and for Chapter Six we seek to capture the changes in experience and usage over the two years of our fieldwork, through a presentation of the views of 52 social workers and team leaders, collected at the end of our study from the Questionnaire Survey part of the Audit Study. This survey follows on from the final part of the present chapter by reporting on experience at a later date.

The focus groups were conducted to achieve three primary and one secondary purpose. The primary purposes were:

1. To understand the initial perspectives and judgments of practitioners and team managers
2. To identify how perspectives varied or were consistent between sites.
3. To discover if and how judgments shifted from the start of the local implementation to the close of the evaluation.

The groups fall into three clusters. Six groups were conducted early in the evaluation, consisting of practitioners and team managers. In addition to seeking an understanding of initial perspectives these groups had an additional secondary purpose of pretesting ideas for the cluster of Audit sub-studies. Four focus groups took place in the early months – one per pilot site - with members drawn from a wider representation of stakeholder interests, and also included ICT staff and managers. Finally two groups took place fifteen months later. The members were drawn from the same individuals who were in the second cluster of groups, but this time each group had a mix of representatives from different sites. They were held in neutral sites (Birmingham and Manchester).

The groups differed in several respects. The first cluster was run as focused discussion groups around a series of provided topics. We aimed to run eight groups in total two in each of the four participating authorities. In practice one did not implement the system early enough for us to do this. This report is therefore based on six groups.

Practical experience of focus groups suggest that these work best when the participants share a basic similarity but do not work or live together (Krueger, R. (1994) Focus Groups: A Practical Guide for Applied Research, Sage, Thousand Oaks). The similarity allows them to check and build on each other’s comments. The requirement that the groups are not part of the same living or working group reduces the risk that members will agree with each other in order to avoid conflict.

For these reasons we asked for one group of team leaders and one group of social workers in each authority. We aimed for groups of between four and six members excluding ourselves and provided two ‘facilitators’, one to attend to the process of the meeting and one to take notes and make sure that the practical side of the meeting worked. These aims were achieved in terms of numbers. A small number of participants had job titles other than ‘social worker’ or ‘team leader’. Undoubtedly, however, the great majority had the roles we expected. Our topic guide was as follows:

1. How would you describe the Integrated Children’s System?

• How did you find out about ICS?

• Explore training (who from, was it offered to everyone, when was it offered i.e. how close to implementation, has it helped?)
• What is its purpose? What would you see as being its key parts (e.g. common assessment framework, management purposes, professional purposes)

2. What are the main potential advantages of using the system?

• Have they been realised? (Why? Why not?)

• Might they be in future?
• Who has been benefiting (you, management, clients, others)

3. What the main potential disadvantages of using the system?

• Have they been realised (Why? Why not?)

• Explore issues around literacy both computer and general
• I.T. problems
• Structure of exemplars
• General philosophy behind it

4. What has been helpful in the way it has been introduced?

• Explore support I.T. and ICS

• Product champions? (helpful, accessible)
• Training

5. What has been unhelpful in the way it has been introduced?

• Explore support I.T and ICS

• Time requirements
• Training

6. Can you see or have you experienced ways of overcoming any disadvantages of using the system or improving it?

• Explore – how problems with the system are identified, what action is taken by whom, and how long. Feedback

7. What if anything is the impact on clients?

• Are they aware of it? Safer? Better recorded? Offered better practice (in what way)
  

As can be seen the guide had subsections in italics. These were ‘prompts’ that we only intended to use if the topic was not fully explored or if the conversation flagged. As it happened, the prompts were almost never necessary. The participants were fully engaged in the process, felt that ICS was important and had a great deal to say about it. Importantly from our point of view their comments were often ‘analytical’. They had much to contribute on why the system was or was not working. These comments seemed to us acute and perceptive.

We had the focus groups transcribed so that we could analyse their content. In keeping with usual practice we did this by ‘familiarising’ ourselves with the data and then identifying ‘key descriptive themes’ which we coded on the transcripts themselves (cf Ritchie and Lewis, 2003; Ritchie and Spencer, 1994). This allowed us to group quotations by theme but also by authority and type of group. We could thus explore how far a given concern was general and shared by both social workers and team leaders or, alternatively, specific to particular authorities or worker groups.

Again in keeping with usual practice we have tried to identify over-arching explanatory themes that underlie the common surface concerns. Rather less usually we have found that these themes were supplied by the participants themselves. Most of the discussion in the group obviously focused on every day matters – for example, the characteristics of particular forms. However, in all the groups thoughtful social workers and team leaders commented on why they saw things the way they did. We found that these analytical comments provided our over-arching themes.

The six focus groups linked to the process study were conducted with a series of piloted group tasks – ranking exercises, vignettes, and the like (Appendix D). The varying of same-site and mixed-site memberships was planned to allow a provisional judgment regarding the extent to which views were shared or differed between sites. In that respect this offers something different from other data, and enables us to make modest comparisons of the standpoints of participants in the early weeks of the local launches of the ICS with their more considered views with the advantage of more than a year’s experience and hindsight.

The groups were audiotaped, transcribed and analyzed through a combination of CAQDAS (Atlas.Ti) and manual analysis.

There were relatively minor divergences between the sites, and for this reason we have opted to present the themes of the groups in a consolidated form, pointing out site-specificities only when they seem important. We start the chapter with a fairly concise discussion of positive views regarding the ICS. The criticisms of the ICS outweighed the appreciative evaluations, and we break down these concerns under five broad sections.

1. The size and cumbersome nature of the ICS³
2. The assumptions about good social work that were thought to lie in the Exemplars.
3. The general characteristics of the Exemplars.
4. The implications of the ICS for work with service users and carers.
5. Ways in which the ICS shaped and even hindered social work.

We close the chapter with a review of the main ways in which these views developed and changed between the start and end of the evaluation.

Welcoming the Principle

Overall, both positive and negative comments were made about the ICS. It was supported in principle and the view was widely expressed that ICS had, potentially, a lot to offer. ‘The principle of it is very helpful’ was a typical way of expressing this position. One practitioner (Site B) concluded,

I think may be where it is useful is it highlights what you haven't got, so if you're doing a core assessment and you're going through that, you think “Ooh, I haven't really looked at that area” or something like that. It's a memory sort of a jogger tool rather than the be all and end all.

I do actually think that the way you describe it as a professional tool is absolutely right, you can’t conceptually argue with the various dimensions…

Managers were, on the whole, rather more likely to welcome the ICS than practitioners.

We are going in the right direction. The work, I think has improved, the standard of work has improved no end, we are moving in the right direction Site B Team manager

Practitioners were more likely to include significant caveats with their positive remarks. For example, the social worker who said ‘The principle of it is very helpful’ ended her sentence with ‘…but as a total it’s confusing’.

Over time there were, as may be expected, some references to growing familiarity with and confidence in using the ICS. More importantly, there was awareness of how working with the ICS led to the acquisition of new kinds of expertise – although perhaps not always of the kind that a rational system use model would anticipate. Without doubt, one of the most striking aspects of the ICS pilot experiences was the indubitable fact that a substantial acquisition of new expertise marked the work at local level. Roles changed. In Site D, for example, the ICS was seen as offering new opportunities – both for enhanced status, mentoring roles, and for new forms of working relationships. It is possible that these benefits will not be fully appreciated until a later stage. The roles of both support staff and practitioners were seen as changing, and this led in some instances to openness to experiment and organizational risk-taking (Site D). We have emphasized the implications for organizational learning in our discussion of the implementation process in Chapter Four.

This was marked by selective use of appropriate (in our view) discretion. One manager explained how, as part of ICS training, she had tried to refine the analytical skills of her team in order to make them better equipped to make these kinds of decisions.

One of the things that our practitioners have found is that they’ve used this word ‘prescriptive’ about the forms and what we're trying to get them to do is focus away from seeing it all as a (form?) filling exercise into an exercise of the fact that they've already done the assessment in their head and what they're doing is they're really putting all that information and just organizing it on a piece of paper.

However, the kind of skill and expertise that we encountered rested at least as much in informal as in formal processes. We pressed for discussion of this issue in Round Two of the groups by asking participants to ‘think of anybody in your local authority who possesses enough knowledge of I.C.S. to make them an expert’. The textured exchanges in one group are summarized in Box 5.1.

Box 5.1 Informal Expertise in Realizing the ICS

C. began by saying that there are certain people in Site A, mainly in management roles, for whom expert is an accurate description. He named N as an expert whose role was critical in overseeing the implementation of ICS in Site A. He also said that he saw A. and B (two members of the group). as ICS experts within their respective teams. A. and B. were unsure as to whether they were worthy of the label of expert. The interviewer asked A. and B. to assume they had reluctantly accepted the role of expert, and asked them to say by virtue of what qualities could they be seen as experts in ICS. A. said that a capacity to provide support to staff members on specific issues of ICS was an important part of this. A. also said that her managerial expertise was useful in both maintaining high standards of use of ICS and in sharing her knowledge and experience of ICS with her team. She went on to add that managers have sufficient information about both the technical aspects and the aims of ICS to provide effective support for practitioners with the problems they might encounter. She said that mangers have the expertise to help practitioners with practice aspects of ICS as well as the recording aspects. A. then went on to mention some specific examples of how her expertise might be called upon. She said that she tries to encourage practitioners to think creatively when making use of ICS forms. She said that social workers often feel compelled to fill in every part of the form and said that sometimes she advises social workers to leave some parts blank if the box is not relevant. She said that she often tries to encourage social workers to have the confidence to make decisions about which sections to leave blank.

C. went on to argue that social workers themselves can become experts. He said that during the process of rollout into teams other than [District X], the role of social workers who have experience of the system is critical. He explained how these practitioners can use their experience and knowledge to provide support to workers in other teams in a way that managers cannot. He argued that social workers have a certain credibility when they impart knowledge to other social workers that managers do not. He said that social workers are often more likely to take seriously advice from other social workers with direct experience of using the system, rather than from managers who might be seen as having a more detached relationship to I.C.S.

This account illustrates what we know from other research that the exercise of professional and clinical judgement – while skilled – is often distinct from formal models of how theory should be sued. We witnessed various instances of the emergence of local expertise. This was sometimes capitalized on by agencies, for example, by naming people as ICS ‘Champions’ or ‘super-users’.

We suspect that the initiation of any substantial organizational innovation will fall foul if it fails to recognize and ‘honour’ emergent domains of informal expertise. The point can be expressed more formally, and in doing so connect the argument to the literature on programme evaluation. Stake and Schwandt, in their illuminating discussion of the quality of ‘evaluands’, distinguish between ‘quality-as-measured’ and ‘quality-as-experienced’. In the first case quality is regarded as measurable and judging quality takes on the characteristic of ‘thinking criterially’, ie through explicit comparison of the research in question to a set of standards for it. Judging quality criterially is more or less an ‘experience-distant’ undertaking. ‘Quality-as-experienced’ starts from the view that quality is a phenomenon that we personally experience and only later make technical, if need be.

This view emphasizes grasping quality in experience-near understandings, that is, in the language and embodied action of those who actually are undergoing the experience of a program or policy. Criterial thinking is important but it is rooted in interpretation of personal experience (Stake and Schwandt, 2006: 408).

Consistent with this analysis, practitioners were not averse to formal training and indeed in some cases lamented the inadequacy of training provision. Participants in Round Two groups concluded,

I think we need refresher courses built in at regular periods of time, as people have picked up bits and feel more confident in some areas and not others.

It’s a real process and because there are different people working on different parts with different levels of understanding ... I think the expectation that you can have two weeks training then everybody can, sort of, go with it is totally unrealistic.

We have not exhausted the ways participants saw positive aspects of the ICS, and will return to this in the closing section of the chapter. However, even the kindest reading of the evaluation leads to the conclusion that criticisms heavily outweighed praise.

Criticisms of the ICS

The size and cumbersome nature of the ICS

A much remarked feature of the ICS was its unwieldy size as a practice system. The force of this criticism was in part directed against the exemplars and in part against the electronic recording aspects. Hence, one of the people who welcomed the principle went on to say,

The structure of the form and that could just be a function of where I'm at in my relationship with it, and getting used to working within that system, but at the moment I'm still finding it quite unwieldy and obstructive to the social work task because … it feels so form driven at the moment, paper driven.

This is seen as linked to matched resources and consequences for workloads, eg the general lack of availability of support systems such as laptop computers. Existing paper based forms were seen as relatively easy to co-construct with children and families. An exchange in an early focus group ended with,

The child and family could be part of it. It sort of depersonalises it when you have to go back and put it on a machine and then send it back to them for their comments and then got back on the machine. You lose that, sort of, working together bit really. It does not seem as much a partnership as it did.

We discuss the results of the time audit in Chapter Nine, but its pervasive presence crops up here too, when participants reflected on the implications for the relative weighting of time spent on direct practice and time spent recording that practice. The system was partly ‘sold’ to the social workers as something that would save them time. According to the participants this was not their experience.

You can sit and transfer [the information from one exemplar to another] which would cut down work. But I think initially putting the work on, like for example the care plan it takes, Oh ten times longer than it used to take before… It's horrendous…It is so long, it takes so much of your time. (Focus group: various social workers)

There were two implications. First, social workers had less time to spend on what they saw as ‘real social work’ - to wit seeing their clients. Second, in order to see their clients a minimum amount they had to limit the number of cases they could take on.

The fact that we're working with a system that's very complicated and has slowed all the social workers down, limits the amount of work they can carry really on their caseload.

In part the demands on time may have reflected inexperience with the system. Hence,

Thus far people are commenting on the amount of paperwork. Now whether that's a true reflection or not I don't know because [with a] new system people always have a tendency to sort of think, wow, yes…A bit overwhelming but maybe the previous paperwork was just as much I don't know. I think we need to re-look at that I think.

Asked if notwithstanding the time demands it was any better for the service user one social worker responded, ‘Well I wouldn't say that. It stops us going out, because we're spending all the time on the computer doing the forms’.

The time and resources issues link to questions of assumptions about appropriate social work skills. Arguments about resources and skills are not freestanding. Lack of resources leads to lack of skills, and lack of skills may result in less pressure to improve resources. A final stage mixed site focus group seemed to corroborate this line of thinking.

You need to make sure you’ve got enough IT staff to actually support your staff while they are coming up to speed with their IT skills.

One particular manager explained how, as part of ICS training, she had tried to refine the analytical skills of her team in order to better equip them to make these kinds of decisions.

One of the things that our practitioners have found is that they've used this word ‘prescriptive’ about the forms and what we're trying to get them to do is focus away from seeing it all as a (form?) filling exercise into an exercise of the fact that they've already done the assessment in their head and what they're doing is they're really putting all that information and just organising it on a piece of paper.

Yet despite these efforts, the general tone was downbeat, even pessimistic. The unwieldy nature of the system meant that for some effective service provision for children and young people is hindered by the format and complexity of the exemplars.

...what is lost in that is the child. You don’t get a picture of the child and their needs very succinctly. It is all lost in these questions and jargon. It is very difficult for another professional to read it and get a picture of the child. (Site A).

The impact of this should not be underestimated. For some, “When you go through some of those documents, they’re all very, very similar documents, LAC review, review this, review that, and it’s just so boring”. More than boredom, for some the effect on morale was noticeable.

I think I can speak for all of us really that since [the IT system] and ICS we have been more stressed, tired, moan more. Team spirit has dropped, we don't feel as professional. Site A Social worker

There were some compensations, albeit perhaps somewhat left-field.

Working with families - if that wasn't important to me and spending little bits of very time with a child and take them out for ... Macdonald’s or something or do some direct work - if that wasn't important to me, I wouldn't be here any more. But there's a reason why I'm still here, but the balance is beginning to tip, and … they're going to lose really good social workers. Site A Social Worker

Team leaders likewise were concerned that turnover would increase.

Social workers have got to handle these cases, you know. At the end of the day you’ve got to manage teams and the aim is to retain social workers. If you haven’t got social workers then it’s no good, you know, having all these systems and this wonderful idea because you’ve got no-one to work the cases. Site B Team manager

Social Work Practice

The exemplars appear to embody a conceptualization of social work as a set of discrete tasks that can be measured, monitored and timed. We return to this theme in Chapter Ten.

I think there are tensions because it has not really sorted out, what it wants. We're told that social workers …need to get to grips with the child's world, you know, provide an account of their story to bring the reader in touch with who they are and what they're about. They need to process and synthesise information and they're putting it all together into a coherent whole and then analysing from that ... But, what some of the system tends to do is to … break it all down into boxes.

This partializing tendency was one of the most common comments made from all sites and at all stages of the evaluation. A feared consequence was that there would be a loss of the central story line. For some, the documentation obscured the family context, and resulted in the loss of narrative.

They're a family unit and because [of] that there's an interaction of different personalities there and their place in the family's very important that they're first born, last born, whatever. And therefore there's …. a lot of research which shows that you should take a holistic view of the family, and not treat everybody as kind of individuals who don't have links to each other, whereas this particular lot of paperwork doesn't recognise those links or that holistic kind of view of the family.

In summary the social workers accepted the need to focus separately on each child but did not accept that this meant all documents had to reflect this principle. Firstly, this was not necessary as they routinely considered the separate needs of each child. Second, it took up a great deal of time. Third, it was not user friendly and could not be easily grasped by professionals or the families themselves. Fourth, the practice obscured rather than clarified family dynamics. Responses to these problems varied from a refusal to use some exemplars to a reluctance to open more than one file for children in certain families. This clearly raises – as an anonymous reviewer of the draft evaluation report pointed out – major issues.

The charge that targets and milestones are not suitable for all kinds of work with children and families is not new and should not be seen as special pleading. Martin Davies has repeatedly made the case that important aspects of social work practice are better seen as associated with a model of ‘maintenance’ rather than change (eg Davies, 1994).

A further concern raised by participants was that the ICS system led to a practical conception of social work as resting less on individual skill and more on observance of procedures. This more fundamental concern was in most cases implicit. It lay behind the discussion of the impact of the system on work with families, the complaints about the time spent in front of a computer and the dislike of a method of analysis that was perceived as form driven, mechanical and ill adapted to individual clients. In essence this was that the system challenged the role and values of social work. One social worker put the point clearly.

It’s not us getting used to a radical new system. I think it's over and above that. It’s challenging our role and asking us to redefine our role essentially to become, ultimately, at the end of the day, it only really matters if we fill the boxes in.

Site A Social Worker

Somewhat similar points were made elsewhere:

I just think [the previous approach] was a much more creative way of working. I just think that you involved yourself and your brain with it far more than clicking loads and loads and loads of boxes … it seems to me that the purpose of social work now is to create information rather than to actually go out with [the clients]

Site C Social worker

We’re making it an admin typing job rather than a job that has interaction with families and children. Site B Team manager

In case we treat this as knee-jerk reactions to innovation, some participants made efforts to work through the issue. For example, someone in Site A reflected:

What it does to practice is it stops you thinking for yourself ‘cos you’re so busy trying to obey. ..That whole bit about how you make decisions and how you prioritise risks and actions is not so amenable to thinking.

So fundamentally the dilemma remains. Social workers recognise the potential of ICS. In principle it is an aid to the process of social work. In practice many of our participants perceived it as a potential threat to its values and its practice.

The general characteristics of the Exemplars.

The picture of the exemplars was not entirely negative. Some they liked, some they did not like For example, one team leader spoke approvingly of the adoption forms and saw them as an improvement. But again the criticisms, perhaps inevitably, outweighed the commendations. The issues are implicit in what we have said in the previous two chapter sections but bear spelling out.

First, the length of exemplars was criticised. While acknowledging that they were seeking new information it was argued that some of the information was not necessary and that the sheer weight of assessment complicated the social work task. Social workers and team leaders in one authority, which had stopped using the child in need plans, were combined in their wish for something simpler and shorter.

we just thought that the forms were, you know, fifty-four page care plans and things like that ….You need good analytical skills but does it take thirty pages to tell you what you need to know?

Second, participants often detected a tendency to repetitiveness, both within and between forms. The social worker who lamented boredom, quoted above, did so on the grounds that, ‘They are all very, very similar documents…and you think, “How many times do I have to read the same thing?”’

Third, there were complaints of inappropriate uniformity, whereby exactly the same questions are asked of different children, and do not allow for diversity within families, hence failing to ask questions that are relevant to some children while asking of others questions that were irrelevant.

You’ve got to have the information and the information’s got to be useful information and targeted information, but information for information’s sake I can’t see the point of that.

Fourth, paradoxically, the exemplar format was associated in the minds of some with a belief that the ICS, despite its overwhelming detail, lacked precision and led to bland analyses. The concern was that although tick boxes were used to describe individuals, they symbolised the view that the business of assessment was one of collecting unrelated pieces of information.

[I am afraid] that people see it as being very much individual tasks rather than process [of understanding a history], …and it doesn’t, doesn’t flow easily…it’s the way that they’re presented…

This imprecision could lead to actual inaccuracies. Someone picked the example of being asked if there were any alcohol or domestic violence issues.

“(Are) there are any alcohol issues?" Yes, No. "Is there any domestic violence? Yes, No.” With no, discussion. So you may say, “Domestic [violence], substance misuse …yes, there was historically”. [Say] he's got a history of alcohol misuse. [Do] you tick ‘Yes’ but actually you're saying he hasn’t been drinking for the last two years? Or do you say ‘No’, because actually he's not drinking at the moment….So I just think you can't use ticky boxes in stuff like that.

Fifth and finally, there was a sense that the ICS had an implicit agenda to deliver an evidence-based practice model that was not always appropriate. This occurred at both the beginning and the close of the evaluation. There were two major reasons why focus group members saw the emphasis on evidencing practice as potentially problematic within the ICS context. Firstly, there may be a tension between ICS forms as an accounting tool on one hand and as a document that is clear and understandable to service users on the other. The level of detail required to make ICS forms a comprehensive means of evidencing practice potentially sacrifices the conciseness and clarity that is needed to make documentation useful to service users. Secondly, it was argued that the heavy emphasis within ICS documentation on having to evidence so many parts of practice unnecessarily increases workload and makes practitioners feel as though their work is subject to unfair and unhelpful levels of surveillance. A manager in our second round of focus groups commented that the emphasis on evidencing practice has made practitioners in her team work “defensively”.

Distancing the service user

Among the oft-heard allegations about the ICS was the belief that the exemplars and the electronic system was not ‘user-friendly’. This came to the surface in the mixed stakeholder focus groups when they were asked to rank who they saw as the beneficiaries of the ICS. While the rhetoric that was seen as stemming from central government referred to the likely benefits to service users and carers, it was these groups that were often seen by a majority of group participants as least likely to benefit. This led to a form of defensive practice whereby service users were deliberately not informed about the ICS, forms were adapted ‘on the hoof’, and where the ICS was seen as requiring unrealistic expectations it was soft-peddled. Three reasons were given for this response. First, forms were too lengthy and overwhelming for service users to cope with.

I mean, you’re talking about a 25-page document for a review of a service, with client/family groups that are in many case in impoverished circumstances and to be presented with a document of that type is simply overwhelming… And for myself I find it quite overwhelming. And that’s on a professional level. And I know which bits to skip, … I know where to look. I’m not a client or service user literally … having to go through every section in order to try and find out what I want to. I think they are very, very un-user friendly.

Second, the language used in the forms was criticised as difficult to understand (by service providers as well as service users).

ICS is not easy to understand. I mean, we struggle with some of the questions, in terms of some of the phrasing in terms of, ‘what on earth does it mean by this?’ You know, come on ‘Plain English’ please!

Finally, the volume and density of information required from the exemplars was seen as intrusive.

I think we have to be careful really not to dehumanise in the process, ‘cos this is people that we are working with, and when you get into using language like that and 20, you know, page long documents for a review of services you’ve been providing, you kind of risk, I think, alienating people and leaving people feeling like they have been picked to pieces.

It's the layout. It's the way information's presented. It's the assumption that a service user knows what this system is and why are these questions being addressed and asked. It's the density of information. It's the inability [on the written form] to distinguish between what's being asked and what [the replies are]…And the jargon … and the length of it… I've got nothing good to say about them to be honest.

Site A (various social workers)

The general response to how well the ICS would prioritize the interests of service users and carers was on balance pessimistic. Central to the rationale behind ICS is the aspiration that it should make the social work process more understandable to children, parents, carers and other agencies. This, it is hoped, will bring about a greater degree of co-operation between these parties in order to provide more effective support for vulnerable young people. According to the documentation on http://www.dfes.gov.uk/integratedchildrenssystem/ the ICS will “…benefit children and families by enabling them to understand what information agencies are seeking and why, and will help them to judge whether they are getting the help they need”. The evaluation gives very limited support for these hopes. One social worker reported that a mother had valued the attention resulting from going through the form. However, the general reaction to the form itself varied from the bemused to the negative:

I bring them out and they can sit there and they roll their eyes, as soon as they see the size of that paper, they glaze over. Social worker Site B

I've done one and had negative feedback just because we had to sit down together and go "Look, ignore all of that preamble there. Read it if you need to but the issues that we're thinking about is ‘what are your child's needs at the moment?’ Are they being met?" So you have to sort of ignore great wads. Social worker, Site A

A team leader recalled her first reactions on seeing the ICS. “I remember thinking to myself, how on earth is a mother, a lot of the mums we deal with, going to make head or tail of it”. Potentially these problems have a negative effect on the relationship between worker and client.

It's all there but in terms of sitting down with a client and going through the issues, very sticky. I mean it's sticky enough doing that, sitting with the client and saying "I don't like the way you talk to your child" … but actually saying that and, and kind of coming out with this rather sort of dry document

Ways in which the ICS shaped and sometimes hindered social work.

The perspective of the ICS as promoting ‘form-driven’ social work has been mentioned several times, and also the response of one team leader who had sought to counter images of the ICS as prescriptive. The social workers resented having to provide information that was – as they perceived it - either irrelevant to the task in hand or served organisational and management purposes rather than their own.

You’re asking social workers to sit there and ask not particularly meaningful questions because some statistician somewhere wants that information.

This is not a new argument. One participant placed it in a wider time context.

So you're driven by the system and I'm not saying that's good or bad … but it is a very prescriptive way of working and … the nature of social work in the last ten years has changed enormously … When I started doing social work twelve/fifteen years ago you had a huge amount of discretion about the sort of work you could do and how you did it and how you ordered it and people carried child care plans around in their heads. It was all a terrible way of working and now it's very much, you know, filling the forms.

Social workers sometimes referred to implicit modes of analysis and understanding that undergird good practice, and whether the ICS supported or undermined them. Key information ‘you're holding ... in your head or on the system, it’s not being crossed over onto the form’. Good practice is seen to entail elements of expertise that are not easily, if at all, susceptible to being inscribed in CS formats. The gap between the system’s aspiration and achievement may not be bridged.

The assessments could be much less prescriptive because ultimately at the end of the day it’s about professional practice, it’s about training and development and it’s about making sure that that equips us to do the job … unless you've got that level of professional expertise, it doesn't matter how many questions a piece of paper asks you to complete, you're never going to get the quality of work.

We have hinted already at ways in which these difficulties led to opportunistic formal and informal ‘rule-breaking’. This criticism did not gainsay the principle that data protection issues are relevant to recording requirements, nor that each child should be separately considered, a principle which was seen as a strength of the system but was taken too far.

Of course I recognise the need for every child to have their own records, and I recognise that that’s critical. [However] the production of the child protection forms from our computer system means that the child protection form has to be (a) written for each child which is time consuming if you’re talking about five, six, seven children in a family, and then printed out for each child.

The perceived loss of the story-line and narrative referred to earlier also led social workers adopted a variety of ‘rule-breaking’ recording devices to circumvent these difficulties, such as to put some information on the record of one of the children in a family but not the others, or to open only one record for a large family thus essentially creating a family file. Likewise, social workers and team leaders in one authority, which had stopped using the child in need plans, were combined in their wish for something simpler and shorter.

we just thought that the forms were, you know, fifty-four page care plans and things like that ….You need good analytical skills but does it take thirty pages to tell you what you need to know?

Finally, implicit in much of the criticism of rule-driven practice was the conviction that it stifled originality. “Filling what the boxes are asking you - it prevents you (from) thinking outside of that. And anyway if you do think outside that, where are you going to put it?”

Developing Appraisals of the ICS

The final focus groups of mixed stakeholders from more than one pilot site allowed us to begin opening up the extent and character of any changes in perspective over the period of the evaluation. This final section of the chapter should also be read in conjunction with the data from the questionnaire survey of social workers reported in the next chapter. Structured data often yields a somewhat different angle on data compared with qualitative data, but we believe the two datasets bear comparison.

We should stress from the start that agency stakeholders’ views will develop and change over the first year or so of local ICS development. There was, as we noted earlier, a growth in participants’ confidence in their ability to comment in an informed way on the operation of the ICS. But apart from this, the changes were not in any straightforward direction. For example, it would be of interest to know if participants grew more positive towards the ICS with time and first hand experience. In some cases yes, but in other cases no. Some early sources of negativity seem to be resolved, whereas others crystallized and hardened. In one authority (Site A), where the general ICS infrastructure appeared to be some years ahead of other sites at the beginning of the evaluation, progress appeared to become moribund, and negative views become consolidated. Comments were made by team managers from different teams in this authority.

During focus group discussion one said that IT support had been poor in her site. The group agreed that there had been and continues to be a problem in Site A in that I.C.S. is regarded too much as simply an IT system. This person and another member both expressed a deep sense of disappointment in their teams regarding the I.T. elements of I.C.S. Practitioners had been given the impression, they argued, that the new I.T. system would make their jobs easier and this had proved not to be the case.

Selective hardening of opinion sometimes posed new perceived problems. In the same focus group, one member said that engagement with ICS documentation had been an overwhelmingly negative experience from the perspective of disabled service users. She argued that the focus within ICS on “milestones and achievements”, which are unrealistic for disabled children, meant that reviews and assessments had become a much more unpleasant experience for her service users.

Earlier views are inevitably centred on hypothetical ideas, for example, about how ICS could work if its potential is realised. We noted this in the opening section of this chapter.

I do actually find the possibilities in ICS quite exciting in terms of social work practice and outcomes for children and families. Whether it … actually turns out to be that way is another matter, but the stuff I’ve read and the stuff I’ve seen has still left me feeling quite excited about the possibilities that there are.

Site B yielded the highest proportion of positive expectations for the ICS, despite the fact that this was the site that eventually became largely passive in its relation to the evaluation, and where local problems meant that the ICS was least progressed here than in any site. The strength of numerous positive hopes has a sad ring in the light of subsequent experience and the note of conditionality that runs through these comments. Extracts from different stages of the focus group indicated as much.

I feel more optimistic about the future...I think just - the staff having been through what they’ve been through - just seeing how patient they’ve been, what a lot of good will there has been, you know, I think there is a lot of promise for the future…So, if it works it will be marvellous…

I can’t get over how motivated the staff still are. Quite frankly I’m so impressed with them. Despite, you know, losing hours and hours of work and so on they still are willing to give us a go and haven’t yet lost interest in it. So we are very, very fortunate and what we don’t want to do is lose that because umm I don’t think they could bear it again. If we go live with the system again and it all crashes down we couldn’t do that again because that would be abusing them really. So we’ve got to get it right next time. They have been excellent our staff group. So good.

It is likely that emphases brought to the table at earlier stages are more likely to be management driven, and that practitioners gain in vocality as time passes. Insofar as that is the case, the outcome measurement aims of the initiative will probably gain more attention in the early stages and may be treated more sceptically as the system develops. There was a disagreement in Site C between a management member of the group and a practitioner. The manager’s view was that, by virtue of its outcome focussed approach to service provision, ICS ought to benefit service users more than any other single group.

…if we hold our hands up we’re not that good at outcome focussed planning. ….and I see this taking us a lot closer to doing that. If we’re better at that then it’s got to be better for them. For kids and families. So you’re wrong I’m afraid!

The ways in which ICS encourages practitioners to evidence their practice was an aspect of the system that generated particularly impassioned responses during the focus groups. The need for practice to be accountable and for this accountability to be made manifest through documentation was widely accepted and understood as a general principle by the focus group members. This comment was made by a manager in the first round of focus groups.

…sometimes social workers do lose sight of the fact that we are accountable for everything, for every single thing we do. We’re public servants, paid by the public to provide a service …And, maybe, by having a system in place that helps us record all our interventions in a very appropriate way it will drive home the message of accountability.

In a Round Two group the workload issue was raised. When discussing a vignette that asked them to discuss whether their daily work had altered as a consequence of the ICS, there was strong agreement between two team leaders that their daily work and that of the practitioners in their respective teams had been altered considerably as a consequence of ICS. One said that this was due, to a large extent, to increased amounts of paperwork. The other concurred with this view and said that social workers are now forced to spend more time in the office rather than with service users. The group as a whole did not demur. While it is not certain how much of this, if true, can be explained as part of the induction period, the nature of the complaints may be more related to the inherent demands of the ICS rather than a phasing in period, given that this data was collected towards the close of the study.

However, it is possible that the more subtle, less detectable benefits of the ICS may not be noticed immediately. For example, we invited Round Two participants to comment on the child-focused orientation of the ICS. While there was some ambivalence, on balance it received a welcome. The capacity for the ICS to allow practitioners to convey the ways in which different children in the same family can be affected by a certain risk factor in very different ways was seen by a small number of practitioners as a very significant benefit of the system. It was argued that different children can be affected in different ways by a certain risk factor depending on their age and levels of resilience and ICS, by focussing on the needs of an individual child, has the capacity to reflect this. It was also argued, however, that many practitioners fail to recognise the importance of a needs led approach to reporting and that this is connected with a general lack of understanding of the research base and rationale on which ICS is founded, in terms of child development.

Conclusion

The picture is not entirely bleak. The basic thinking behind the ICS - at least as perceived by those to whom we spoke – is generally welcomed. In addition, some of the exemplars were seen as an improvement on previous arrangements. When we spoke to some of the same people fifteen months after the first round of focus groups, there was evidence of some shifts and clarifications. Also, the introduction of the ICS led to the emergence of new expertise.

Yet the balance of the picture is still in the negative. The most generous gloss on the data cannot cloak that at the very least there is a major education task ahead. More likely, the issues go beyond ‘education’ and enshrine core debates in social work about the nature of the professional task and the ways to develop and deliver services to service users and carers. We pick up the evidence from the process study once more in Chapter Ten where we aim to elucidate the agenda of issues and themes that form part of the future agenda for children’s services.