How good is our public library: the framework

The framework measures Scotland’s progress against the National Outcomes.

To do this, it uses ‘National Indicators’. These indicators give a measure of national wellbeing. They include a range of economic, social and environmental indicators.

The framework aims to create a more successful country, give opportunities to all people living in Scotland, increase the wellbeing of people living in Scotland, create sustainable and inclusive growth, reduce inequalities and give equal importance to economic, environmental and social progress.

To achieve the national outcomes, the National Performance Framework aims to get everyone in Scotland to work together. This includes national and local government, businesses, voluntary organisations and people living in Scotland.

Sources

The sources that you consult to provide evidence for your self-evaluation document will overlap across the four key areas of the indicators. Your evaluation should aim to show how your library services can demonstrate the following aspects.

Key themes

The Key Themes in this Quality Indicator are all interconnected and interdependent:

1. The library has high quality physical and digital spaces that respond to evolving needs.

2. The library caters for people with diverse needs in terms of sufficiency, range, and suitability of resources, services and programming.

3. The library encourages active citizenship through access to trusted, accurate information that empowers people to make informed choices.

4. The library has an embedded culture as a principal contributor to developing reading, literacy, numeracy through local and national initiatives.

 

Key things to think about

1. How the diverse needs of users are identified and met, stock selections and acquisition including evidence-based stock management, strategies and approaches for reader development (including, for example, book groups or author events), management and promotion of collections, partnerships with other agencies and organisations.

2. The range and suitability of resources (both print and digital), formats, languages and accessibility, range and suitability of services, specialist information provision ( e.g. in support of education or business), range and suitability of programming and activities, role as a digital enabler, new or innovative services.

3. Branch distribution and availability (locations and opening times), home or mobile provision, digital provision, remote services and access, digital infrastructure and governance.

Sources

The sources that you consult to provide evidence for your self-evaluation document will overlap across the four key areas of the indicators. Your evaluation should aim to show how your library services can demonstrate the following aspects.

Key themes

The Key Themes in this Quality Indicator are all interconnected and interdependent:

1. The library provides a stimulating and motivating learning environment that supports lifelong learning for all.

2. The library supports learning and learning opportunities through its own staff expertise and in partnership with other agencies.

3. The library, through its learning provision, gives individuals opportunities to improve their lives.

4. The library uses physical and digital spaces to encourage, support and deliver learning opportunities.

Key things to think about

1. How the library identifies learning needs and profiles of users, provision of formal and informal learning opportunities (in-person and digitally as appropriate); feedback, evaluation and impact.

2. How the library creates a welcoming learning environment, use of space (physical and digital) to support learning, facilities, staff expertise and training for supporting learning.

3. How the library works collaboratively and partnership with other learning organisations; how lifelong learning is supported and developed; how the library’s learning provision supports wellbeing, employability and other goals.

Sources

The sources that you consult to provide evidence for your self-evaluation document will overlap across the four key areas of the indicators. Your evaluation should aim to show how your library services can demonstrate the following aspects.

Key themes

The Key Themes in this quality indicator are all interconnected and interdependent.

1. Library services are designed to reach the heart of communities and engage with new audiences.

2. Services and programming reflect the rich and diverse cultures and communities of Scotland and provide an offer celebrating local and national heritage.

3. Services encourage economic and cultural growth within communities through stimulating new ideas and co-creation.

4. Services work collaboratively to support the wellbeing and resilience of communities.

Key things to think about

1. Co-creation and working with the community, partnership working, community outreach, stakeholder engagement and consultation, innovation.

2. Programmes, initiatives and partnerships that support and showcase the diverse cultures of communities, resources (print and digital) that enable their story to be told (such as local studies materials, spaces and staffing), engagement with regional or national cultural or heritage initiatives.

3. Programmes, initiatives and partnerships that support health and wellbeing for individuals and for the community, and those that support societal or economic resilience.

How good is our public library service has several important elements:

(i) emphasis on the self-evaluation process, with the main aim being a mechanism for improvement in public library services

(ii) staff work together to plan the process and gather evidence and data, ensuring all staff and stakeholders are successfully engaged

(iii) external peer reviewers, acting as ‘critical friends’, visit to review the self-evaluation report, and discuss it with staff

Reviews using the framework of How good is our public library service ensure consistency in self-evaluation across authorities. This peer review process provides a mechanism for identifying and sharing good practice between libraries, while also maintaining and updating the benchmark statements.

Planning for self evaluation

In planning for the self-evaluation process, library services can choose which of the Quality Indicators to address, fitting in with individual circumstances, plans and priorities within each library service.

Managers should aim for participation by different levels of staff across the service for the self-evaluation process. The team involved with each Quality Indicator should ideally represent a cross section of management and library staff to help to ensure a shared understanding and ownership of strengths, areas for improvement and impact of the library service.

It is useful for the team to ask the “so what?” question regularly as they gather evidence and draft the report. This will strengthen the self-evaluation and assist identifying sources of evidence to demonstrate value and impact. Self-evaluation should be viewed as a positive diagnostic tool for a service.

RADAR© can be particularly helpful here. At its highest level, the RADAR logic states that an organisation needs to:

• Determine the Results it is aiming to achieve as part of its strategy
• Have in place an Approach that will deliver the required result, both now and in the future
• Deploy this approach appropriately
• Assess and Refine the deployed approach to learn and improve

Self evaluation report

The Self-Evaluation Report is the document that library services will use to bring together all the information and outcomes of the self-evaluation process on the quality indicator in question. It is the document which will be sent to the peer reviewers ahead of the visit.

The Self-Evaluation Report should be a reflective and evaluative document. It should be concise and succinct, not being longer than eight pages. It should also be capable of acting as a working document for library services to record planning, progress and outcomes.

The aim being that this allows for a more complete picture of the process undertaken and outcomes achieved when the peer-review panel add their final report to produce the overall assessment. The use of hyperlinks within the text is encouraged to enable other supporting documents to be viewed remotely; these can be appended in an additional list of references, not exceeding two pages in length.

Gathering evidence

Self-evaluation needs to be based on the collection and review of the evidence of outcomes and impacts. The Quality Indicators provide several key evaluation themes, and public library services are asked to identify ways in which they know about and can demonstrate how they are performing. Assessments should be made about the public library service’s performance based on the evidence.

Evidence helps to support self-evaluation processes by demonstrating proof of activity and impact. It can help in establishing how well the service is meeting users’ needs. Evidence can be quantitative or qualitative. Quantitative evidence records what can be measured, for example numbers of registered learners. Qualitative evidence will seek to draw out the value which users may put on services, and this is often unstructured in format, for example feedback from users, or evidence of successful learners moving on to other education or employment opportunities.

These sources of evidence are complementary. For example, a policy statement seeking to “promote learning for all” can be observed across the range of learning opportunities offered, and backed up by feedback from learners. Examples of sources of evidence are suggested in the Quality Indicators for each theme, and some more general examples are included here (they need not all be used and equally the list is not exhaustive):

Consult users, non-users and staff:

• Feedback, oral or written, from discussions with individuals and staff about specific activities
• Formal discussion with individuals, non-users, elected members, other council officers and library staff, possibly as part of community engagement activities forming part of the local Community Planning process
• Surveys and questionnaires run nationally or locally.

Observe activities:

• Examine the use of the library
• Review the range of resources and services available
• Observe individuals and staff across a range of activities
• Review the library environment

Consider measurable outcomes:

• Analyse quantitative evidence from sources such as statutory performance indicators, CIPFA, surveys and the library management system

Examine documentation and resources:

• Local authority strategic/corporate plan
• Community Learning and Development Strategy
• The Community Plan and other Community Planning information relevant to library services
• Public library service objectives and improvement plan
• Event programmes for range of individuals
• Lifelong learning programmes and materials
• Range of resources in relation to community diversity profiles
• Public information documentation
• Marketing and publicity strategies and documentation
• Press coverage
• Staff training programme
• Evidence of library staff involvement in internal and external partnerships
• Evaluation of externally-funded programmes

 

Photographs, audio and video clips of library activities are also useful to demonstrate particular strengths.

In looking at evidence, all of the RADAR© Elements should be considered to understand what aspects of performance are working well and where there are opportunities for improvement.

Demonstrating impact

How will you know if what you are doing, or planning to do, in the public library service will make a difference for individuals and groups in the community? We can describe, using the various ‘inputs’ and ‘outputs’, what the library does: the size of the adult fiction collection, the number of hours open, how many borrowers it has, the number of books issued to children, the various outreach programmes it is involved with and so on, but then we must ask the question ‘so what?’.

How do you demonstrate the value that the various activities and services provided by the library service have for individuals and communities served? This is where impact comes in and involves the planning and gathering of evidence to demonstrate the value and impact of library services.

In gathering evidence, library services need to act strategically, and the evidence may be both qualitative and quantitative. Sophisticated quantitative data and good quality impact information should complement each other as they have different roles and purposes; the quantitative data can reassure stakeholders, but this, presented alongside the qualitative evidence, can presented a broader picture and demonstrate value and impact very effectively. It is recognised that gathering evidence to demonstrate impact is neither an easy nor simple task.

One significant aspect of undertaking a self-evaluation process such as How good is our public library service is to identify areas for improvement, and also to share good practice with colleagues and other library services. This process can also assist staff when considering difficult decisions such as ‘should we really be doing this piece of work with this partner’ or ‘will this activity really make a difference’. Impact and value are not necessarily monetary (indeed they frequently are not) but rather about the difference they make to individuals or to communities.

Determining the level

When the evidence gathering stage is completed (this may however be ongoing if the self-evaluation process is truly embedded in library service planning) the drafting of the Self-Evaluation Report should take place. The Self-Evaluation Report is a free text document of no longer than eight pages per Quality Indicator that seeks to draw everything together.

The report should aim to document current performance, what difference it makes (the impact), and identify where improvements can be made. The report will recognise key strengths and identify ways in which improvements can be made. From this, an overall and realistic grade should be given by the library service.

 

Sharing good practice

One of the key benefits of, and important outcomes from How good is our public library service is the sharing of good practice about what works well in public library services. As part of the self-evaluation process, one way of ensuring good practice is captured is for library services to offer up examples of good practice, a snapshot or small case study, as part of the self-evaluation process. Such examples of good practice can be shared and disseminated later.

Encouragement to also share these experiences at events such as conference and other meetings would take this a step further, giving the teams involved more opportunities to showcase what works well in their library service, while also providing valuable opportunities for staff development.

Peer review process

How good is our public library service is primarily a self-evaluation process supported by a system of peer review. The combination of these two elements is widely considered to be appropriate and robust for an effective quality standards mechanism; the approach is also embedded in a range of other public sector quality improvement mechanisms.

Peer-review is the solid basis for driving continuous improvement and enhancement. Peer-review within the How good is our public library service process exists to ensure consistency, credibility, transparency and externalisation. Those involved in peer reviewing bring with them the benefits of a wider perspective, but with enough pragmatism to understand situations ‘on-the-ground’.

 

Roll of peer reviewers

The role of the peer reviewers is to consider the self-evaluation report and the levels allocated by the library service. Three peer reviewers consider the self-evaluation report, visit the library service to meet with staff and users, examine the evidence, and ensure the levels allocated are an accurate reflection of service delivery in relation to characteristics and benchmarks of other Scottish public library services.  Peer review is an integral aspect of the How good is our public library service process.

By inviting knowledgeable library professionals to review their self-evaluation process, library services benefit from hearing an informed and impartial assessment from an external and knowledgeable perspective.  The peer reviewers may best be described as acting as ‘critical friends’ who examine the library service’s self-evaluation thoroughly, reviews the evidence impartially, engage with staff and stakeholders, and comes to an independent assessment that they share in a frank and friendly manner with the library service.

Peer review panel

The peer review panel consists of three members with one acting as convener; the members will be drawn from staff in other library services across Scotland. The peer review visit will normally be a one day event.

One of the members will act as convener and have responsibility for compiling the summative report of the panel regarding the self-evaluation, the ratings and the experiences on the day of the visit. The final report from the peer review panel will be summative in nature, normally around two or three pages in length and will be appended to the self-evaluation report.

Self-evaluation is an important diagnostic tool for any public service, and it should be embedded in the work of libraries.

Self-evaluation needs to be supported by the ongoing collection of evidence relating to performance, value, and impact. It also supports the process of continuous improvement and enhancement. The evidence collected will vary depending on the quality indicator being considered and the context of individual library services.

The self-evaluation report for each quality indicator should be no longer than eight pages (A4). An additional two pages of references or hyperlinks may be supplied and single sheet answering the questions overleaf may be appended. The final question (any significant characteristics of the services) enables services to note any particular demographic, social or geographic facets that are distinctive.

The self-evaluation report should be succinct and to-the-point. It should open by addressing the overarching questions connected with the strategy and vision for the service and how that informs the particular quality indicator being examined. The report should present evidence to support the statements being made throughout and should seek to demonstrate the outcomes and impact of library service on individuals and communities.

How good is our public library service operates using a six point scale.

Awarding levels requires considerable professional knowledge both on the part of the library service itself as they create the Self-Evaluation Report and on the part of the Peer Assessors during their visit.

The starting point for allocating levels should always be positive, using a combination of the evidence.  There should be sound arguments to support the level, backed up with evidence.

 

6: Excellent

• Clearly excellent and of the highest of standards.
• Very best practice worth disseminating widely beyond the service.
• Users’ experiences and achievements are of a very high quality.
• Outstanding delivery of core functions and range of enhanced services.

5: Very Good

• Major strengths.
• A high standard of provision, examples of good practice to share.
• Any weaknesses do not impact on users’ experience.
• Services will take opportunities to improve and strive to raise performance to excellent.

4: Good

• Important strengths that have a positive impact.
• Few weaknesses that do not have a substantial adverse effect.
• Some examples of good practice.
• Services seeking to raise performance further, and addressing areas of improvement.

3: Satisfactory

• Provision where strengths just outweigh weaknesses.
• Individuals have access to core service provision.
• Strengths have a positive impact on individuals’ experiences, while weaknesses have some adverse affects on user experiences.
• Services will seek to address areas of improvement while building on strengths.

2: Weak

• Weaknesses have a negative impact on the quality of individuals’ experiences.
• Such weaknesses diminish individuals’ experiences in significant ways.
• The service will need to take structured and planned action to address weaknesses.

1: Unsatisfactory

• Major weaknesses in provision.
• These weaknesses will require immediate remedial action.
• Individuals’ experience is at risk in significant aspects.
• Improvement requires strategic action and support from senior managers.
• Remedial action may involve work alongside other staff and agencies.

Your vision

• Vision and objectives
• Strategic / Corporate / Community Plans
• Budget Allocation
• Community and / or library profiles
• Annual Library Plan / Service Improvement Plan / Annual Review
• CIPFA Statistics

Your activities

• Adult learning guides
• Assistive technologies
• Bookbug schemes, events and packs
• Community surveys or public consultations as appropriate
• Customer Care Standards
• Digital platforms (website, social media, e-resources)
• e-books and audio – range and extent of provision
• Equality and diversity training (including DDA)
• Feedback from users and community groups (and responses)
• Events and activities (such as book groups, events, author visits, book festivals, promotional activities)
• Heritage and local studies specific assets
• Range of collections (formats and languages provided)

Your policies

• Acceptable Use policies
• Collection (or Service) Development policies (including stock selection, acquisition, management)
• Communication or Marketing plans or policies
• Community Learning and Development strategy
• Customer service standards/Digital policies
• Equal Opportunities policy
• Funding applications for new projects
• Home delivery / Click and Collect policies
• Lifelong Learning policies
• Literacy policies
• Mobile provision
• Outreach policies
• Partnership agreements
• Reader Development policies
• Staffing policies (structure, handbooks, job descriptions, recruitment and development policies)
• Usage and loan statistics
• Volunteer strategies and policies
• Young people’s policies

Overarching questions

The following questions apply to all the quality indicators and should be addressed at the beginning of the Self-Evaluation Report.  This approach embeds discussion of vision and strategy in each quality indicator (rather than being a separate one as was the case in the past).

These three overarching questions form the starting point of the examination of each of the quality indicators, but evidence supporting these aspects can also be embedded in various other places in the report when addressing specific issues.

How do we plan and manage high-quality people-centred services?

This should consider:

• How the vision and strategy is developed in an inclusive fashion and how it facilitates the delivery of local and national priorities
• How the vision and strategy are articulated to all (both internally and externally)
• How staff contribute to its development and implementation
• How leadership, management and staff development (including how recruitment and retention) are fostered) and ensure appropriate staff for service delivery

How do we work in partnership to create more resilient, fairer and healthier communities?

This should consider:

• How the vision and strategy is developed in an inclusive fashion and how it facilitates the delivery of local and national priorities
• How the vision and strategy are articulated to all (both internally and externally)
• How staff contribute to its development and implementation
• How leadership, management and staff development (including how recruitment and retention) are fostered) and ensure appropriate staff for service delivery

How do we use qualitative and quantitative evidence to support continuous improvement?

This should consider:

• What performance monitoring is in place and how it informs continuous service improvement
• How budget allocations and financial metrics match service priorities
• How the library service delivers Best Value
• How evidence-based decision making is implemented

Process

• Step 1: PLAN: Choose which of the Quality Indicators to address and assemble a team.
• Step 2: GATHER EVIDENCE: Gather both quantitative and qualitative evidence to demonstrate performance.
• Step 3: CONSIDER IMPACT: Ask the ‘so what’ question when describing what the library does.
• Step 4: DETERMINE THE LEVEL: Assign a realistic grade based on the key strengths and areas for improvement.
• Step 5: SELF EVALUATION: Bring all the evidence together to write the self-evaluation report.
• Step 6: PEER REVIEW: Use the panel’s visit as a learning and development opportunity.

Outcomes and impacts

Having considered the evidence in respect of both the overarching questions and the specific quality indicator the final element is to reflect on the outcomes and impacts.  The following questions should be addressed for each quality indicator in the concluding sections of the Self-Evaluation Report:

• What difference does this make to individuals, communities and society?
• How do we communicate to ensure the voice of the library is heard by key stakeholders?
• How do we ensure that our service provision is sustainable?