The Swedish Agency for Public Management

Evidence-based practice in social services. Evaluation of the agreement between the Government and Swedish Association of Local Authorities and Regions (2014:18)

Statskontoret (the Swedish Agency for Public Management) has been commissioned by the Government to follow up the agreement between the Government and the Swedish Association of Local Authorities and Regions (SALAR) concerning evidence-based practice in social services. The aim of the agreement is to improve the full range of social services and to improve social services' use of information technology using various initiatives that go by the name eHälsa (eHealth). The Government has allocated a total of SEK 682 million from 2011 to 2014 for this intervention. The Government and SALAR have announced their intention to continue the cooperation through to 2015.

Statskontoret has submitted three interim reports on the work. This report is Statskontoret's final report. Statskontoret has evaluated whether the agreement has contributed to improving conditions for social services' evidence-based practice and use of information technology. Statskontoret has also evaluated whether the agreement has contributed to improving the responsible authorities' work with this.

Conclusions on the agreement on evidence-based practice

Statskontoret's overall assessment is that the agreement has contributed to the development of activities within social services. The step to a more systemised knowledge development, however, is still far. Statskontoret has also determined that the agreement has improved the conditions for social services' evidence-based practice and use of information technology, but further efforts are required before the support structures for knowledge development developed in the counties can be firmly established. The reasoning behind Statskontoret's assessments is described below.

The municipalities are developing their activities

In Statskontoret's evaluation, it has been found that the agreement contributes to the development of activities. Heads of social services believe that the support structures have contributed to the development of activities within a number of areas of social services. This is partly a matter of method support and training provided to the municipalities by the support structures.

Knowledge development not yet systematised

Statskontoret feels that knowledge development within the area is not yet systemised. There is a lack of clear processes for user collaboration and the links to research need to be strengthened. Furthermore, the potential in the IT development has not yet been harnessed. The work with eHealth is not sufficiently integrated with other efforts in social services.

Efforts are required for the support structures to be firmly established

In the counties, support structures for knowledge development have been built up. These support structures consist of various strategic functions such as political committees, steering groups and R&D activities. It is therefore a matter of a network of people who are active in e.g., research environments, R&D units, welfare centres or healthcare centres.

Statskontoret has determined that the organisation of the support structures has improved conditions for evidence-based practice. All of 93 per cent of the municipalities have taken part of the activities conducted in the support structures. It can often be difficult for the work being carried out in temporary efforts to continue once the project has finished. The previous year's agreement, however, imposed organisational and financial requirements on the responsible authorities. According to Statskontoret, the concrete plan to phase out central government support which was presented at the time served to intensify the work to obtain regional funding.

The follow-up for the year reveals that the responsible authorities in many counties do indeed finance parts of the support structure today, but that the long-term financing is not yet guaranteed. Statskontoret is of the opinion that the Government and SALAR should demand that the counties submit an up-to-date progress report and report on which measures have been taken in order to ensure long term sustainability after 2013, when they last submitted plans for financing and organisation to SALAR.

Inadequate evaluations decrease opportunities for evidence-based practice

If the users are to benefit from initiatives which are built on the best available knowledge, the methods and approach must be systematically evaluated. There is however a low level of knowledge within social services regarding how effective different forms of intervention are. It has proven difficult to get researchers interested in this area. Statskontoret believes that one key factor is that people within support structures have knowledge of the research community and understand in what way the area can be empirically interesting. Efforts are also required from actors not covered by the agreement, such as the Swedish Research Council for Health, Working Life and Welfare (FORTE).

Activities run the risk of missing out on support

Statskontoret has determined that private practitioners are often excluded from initiatives. The support structures require more systematic channels of contact with the private practitioners. The Government and SALAR need to impose requirements on support structures to establish such channels of contact. The municipalities also have a responsibility as client to impose requirements on their practitioners to take advantage of the support for knowledge development. According to Statskontoret, the municipalities need more developed support in the work to impose requirements and perform follow-ups to ensure the practitioners do this.

Not even activities within the affiliated healthcare are fully covered by the support. The lack of collaboration with healthcare poses a long-term risk of leading to people covered by health and social care in the interface between social services and healthcare been unable to take advantage of initiatives that are based on the best possible knowledge.

Weak links in the knowledge management chain

So that users can benefit from initiatives which are based on the best available knowledge, there must be a sound interaction between the national, regional and local level. Affected authorities can use the support structures in the work to generate new knowledge and gather experiences from municipalities and county councils, though Statskontoret has established that they do not yet do this to any great extent. Even if the authorities may have other channels to reach the activities, Statskontoret feels that it is a weak link in the knowledge management chain that the authorities do not use the support structures to a greater extent.

Investments in knowledge development need to be coordinated

The responsible authorities receive funding from several agreements which influence knowledge development within social services in different ways. Statskontoret has established that the responsible authorities have sometimes had difficulties seeing the connection between SALAR's various initiatives in the area of social services. There are also "national knowledge centres" which have tasks that partially overlap with the goals for the support structures. Statskontoret believes it is important that investments in knowledge development are coordinated where possible and that overlapping tasks are avoided so as to make it easier for the responsible authorities to make prioritisations within their activities.

In the report (chapter 5), Statskontoret provides proposals to the Government for further collaboration on the agreement.

Conclusions on agreements as policy instruments

According to the assignment, Statskontoret shall also draw general conclusions on agreements as policy instruments. At present, there are 14 agreements in the area of healthcare. They comprise a total of SEK 5.7 billion from the state budget, the majority of which is distributed among municipalities and county councils according to performance levels or results.

Statskontoret's overall assessment is that if the Government wishes to prioritise an area and at the same time take advantage of local experiences, agreements can be an effective policy instrument. The Government should however be relatively restrictive with the use of agreements. As far as possible, the traditional policy instruments and the conventional chain of command between the authorities and the Government are used.

When are agreements an effective policy instrument?

Statskontoret's analysis shows that agreements intended to develop knowledge or systematise approaches have the conditions to work well as they are based on the voluntary nature of the agreement. Statskontoret's continuous follow-ups of two such agreements reveal that they have contributed to the development of activities, even if the goals of the agreements have not yet been achieved.

Agreements can also be effective if the Government wishes to place one area in focus, for example in matters which are normally managed and developed by municipalities and county councils. It can be of great symbolic significance that the parties have pointed out the areas as important. Statskontoret has seen examples of this in its follow-ups.

When should agreements not be used?

Statskontoret does not believe that agreements can replace the existing chain of command and the division of responsibility between state, municipality and county council. Nor is it appropriate to use agreements to achieve compliance with laws and regulations. A special reward system for compliance with certain legal requirements entails the risk that other legal requirements will become a lower priority. It is also problematic if compensation is provided for requirements which are lower down the list than those stipulated by law.

Furthermore, agreements are not the most effective way of managing fixed term projects, where the purpose is not that the responsible authority will continue development work once the project is completed. The management of agreements entails a greater workload for the Government Offices, SALAR and the responsible authorities alike. In order to justify the higher workload, the work must be such that the actors can continue to develop once the agreement has come to an end.

How should agreements be formulated?

Like all policy instruments, agreements have their weaknesses. They are for example not formally binding, and the parties to an agreement do not have direct responsibility for its implementation. Furthermore, SALAR has contradictory roles as it acts as a standards authority for its members whilst at the same time being a special interest group. When formulating an agreement and when managing the work, efforts should be made to mitigate or compensate for the weaknesses inherent in agreements. There are also measures which can be taken to strengthen the management effects of agreements. Statskontoret believes that among other things, the Government should observe that:

  • a shared perspective of problems among the Government, SALAR and the responsible authorities strengthens management effects,
  • fewer agreements should be prioritised over many small ones, which would enable the responsible authorities to benefit from the initiatives,
  • the assessment of the responsible authorities' initiatives should not be performed by SALAR as this could conflict with its role as a special interest group,
  • the management of involved authorities should follow the conventional chain of command,
  • the authorities should have the time and mandate to perform independent assessments of objectives achieved if the authorities are to audit the initiatives in municipalities and county councils,
  • SALAR must clearly coordinate the various agreements so that the connections are clear to the responsible authorities,
  • overlaps in the division of roles between SALAR and the National Board of Health and Welfare should be avoided in knowledge-generating assignments,
  • funding should where possible go to the responsible authorities, which would have a more positive influence on the management effects than if the funds were to go to SALAR,
  • external evaluation needs to be ensured when the agreements concern the same sums of money or when performance requirements are formulated,
  • correctly formulated performance requirements can strengthen management effects,
  • the continuity of the improvement work also needs to be secured when the agreement comes to an end by means of e.g., presenting a plan to phase out central government support and imposing financial requirements on the responsible authorities,
  • where information systems and other infrastructure has been built up, responsibility for the ongoing management must be clarified once the agreement ends.