A Review of the Education External Evaluation Services

Brian Donnelly Education Review Office

APPENDIX 6: LIST OF CONTRACTUAL OBLIGATIONS AND UNDERTAKINGS

Assurance audits
in schools are carried out by the Education Review Office against legislation,
regulations, Gazette statements pursuant to legislation, conditions
of licences, signed Charter undertakings and any other specific agreements
between the Board of Trustees and the Crown.

SOURCES

The 'Handbook
of Contractual Obligations and Undertakings - Schools' comprises,
to date, requirements stated in the following Acts, Regulations and
other specified sources.

  • Education Act
    1964
  • Education Act
    1989
  • Education (School
    Attendance) Regulations 1951
  • Education (Secondary
    Instruction) Regulations 1975
  • Education Staffing
    Orders
  • Code of Ethical
    Conduct for the Care and Use of Animals in School Programmes
  • Collective Employment
    Agreements for all staff
  • Health and Safety
    Code of Practice for State Primary, Composite and Secondary Schools
  • Memorandum of
    Agreement - Teacher Salaries Grant, Direct Resourcing of Teachers Salaries
  • National Education
    Guidelines 1993
  • New Zealand Gazette,
    Issue 190
  • Syllabuses for
    New Zealand Schools
  • Animal Protection
    Act 1960
  • Animals Protection
    (Code of Ethical Conduct) Regulations 1987
  • Children, Young
    Persons and Their Families Act 1989
  • Civil Defence
    Act 1983
  • Disabled Persons
    Community Welfare Act 1975
  • Employment Contracts
    Act 1991
  • Fencing of Swimming
    Pools Act 1987
  • Fire Safety and
    Evacuation of Buildings Regulations 1987
  • Food and Hygiene
    Regulations 1974
  • Health Act 1956
  • Health and Safety
    in Employment Act 1992
  • Human Rights Act
    1993
  • Local Authorities
    (Members' Interests) Act 1968
  • Local Government
    Official Information and Meetings Act 1987
  • New Zealand Bill
    of Rights Act 1990
  • Privacy Act 1993
  • Private Schools
    Conditional Integration Act 1975
  • Public Finance
    Act 1989
  • Residential Tenancies
    Act 1986
  • Resource Management
    Act 1991
  • Smoke-free Environment
    Act 1990
  • State Sector Act
    1988

Requirements
in legislation not specifically related to the education sector cover
all schools.

The Handbook
also lists obligations and undertakings related solely to specific types
of schools - integrated schools (Section I), private schools (Section
J) kura kaupapa Maori (Section K), holders of certificate of exemption
(Section L) and correspondence schools (Section M).

APPENDIX 7: DETAILS ON ASSURANCE AUDITS

Definition

An assurance
audit is a process which examines and reports on the extent to which
a governing/managing body meets its obligations and undertakings to
the Crown. These obligations and undertakings are contained in
legislation and regulations and include any specific undertakings entered
into through a licence, charter, or other agreements.

Assurance audits
are undertaken in all state schools and privately owned schools and
in all licensed or chartered state and privately owned early childhood
services.

Purpose of
Assurance Audit Reports

Assurance Audits
are undertaken in order to:

  • assure the Minister
    responsible for the Education Review Office about the performance of
    governing and managing bodies in fulfilling their obligations and undertakings
  • inform the governing/managing
    bodies about their performance in fulfilling their obligations and undertakings
  • inform policy
    development and decision making by governing/managing bodies and of
    government; and
  • provide information
    for the public.

Conducting
an Assurance Audit

It is undertaken
in a way which demonstrates the belief that governing/managing bodies
have the intention to comply. Both primary and corroborative evidence
of compliance are rigorously sought, analysed, interpreted and documented.
Audit judgements are supported by sufficient documented evidence to
justify them.

Sources of
Obligations and Undertakings

There are two
sources of obligations and undertakings of governing/managing bodies
to the Crown which are included in all assurance audits. The first
source is the set of obligations and undertakings to the Crown which
are managed by the secretary for Education as the Crown's agent.
These provide most of the parameters within which governing and managing
bodies are required to perform. These obligations and undertakings
are stated in a number of separate statutes and regulations and in specific
agreements, including:

  • education specific
    legislation;
  • licences;
  • registration;
    and
  • charters

The second
source is the set of obligations and undertakings to the Crown which
are set out in general legislation and regulations, including:

  • state sector legislation;
  • employment legislation;
    and
  • local authority
    legislation and regulations. (A list of specific requirements
    appears in Appendix 6.)

Other Specific
Agreements

These include
property occupancy agreements, early childhood services' leases of
Crown land, and various types of discretionary funding. At present
these agreements are not included in an Assurance Audit unless they
are included in an institution's charter.

The Integration
Agreement between the Crown and the proprietor of an integrated school
is not included in the assurance audit. Agreements that are held
with other Crown agencies such as the New Zealand Qualifications Authority
(accreditation) or the Department of Social Welfare (child subsidies)
are only reviewed if a specific and formal arrangement exists with that
agency to do so.

Charters
- Schools

These are the
signed agreements between the governing/managing bodies and the Crown.
Individual school charters comprise undertakings directly from the Education
Act 1989, the 1993 National Education Guidelines and the schools own
local objectives.

All explicit
undertakings listed in a school's charter form part of the total contractual
relationship between the board of trustees and the Crown and are included
in an assurance audit.

Charters
- National Education Guidelines

Under the Education
Act 1989 Section 60A the Minister may publish in the New Zealand Gazette,
National Education Guidelines. These are made up of National Education
Goals, National Administration Guidelines and the New Zealand Curriculum
Framework which embraces the national curriculum statements.

The National
Education Guidelines form part of all school charters. Every charter
is deemed to contain the aim of achieving, meeting and following the
National Education Guidelines. In the course of an assurance audit
of a school reviewers seek assurance that boards of trustees have taken
all reasonable steps to achieve, meet and follow the National Education
Guidelines.

Charters
- Early Childhood Services

Under sections
312 and 313 of the Education Act 1989, every early childhood service
charter has the effect of a set of undertakings by the managing body
to the Minister to take all reasonable steps to ensure that the service
is managed, organised, conducted and administered in accordance with
the objectives contained in the Charter.

The objectives
and practices set out in the Early Childhood Education Charter Guidelines
(A Statements of Desirable Objectives and Practices) gazetted on the
6 December 1990 are considered to be the core of each Charter document.
A revised statement of DOPs is due to be implemented on 1 August 1988.

In the course
of an assurance audit of an early childhood service, reviewers will
seek assurance that managing bodies have taken all reasonable steps
to manage, organise, conduct and administer in accordance with the Charter.

Assurance
Audit Approach

There are three
broad conceptual phases to be worked through in order to reach each
judgement of compliance or non compliance during an assurance audit.

Phase 1 Compliance
Intentions

The intention
of the governing/managing body to meet the requirements is ascertained
from documentation held at the district office, from information supplied
by the institution and from interviews held with appropriate personnel.

Phase 2 Preliminary
Compliance Conclusions

These are drawn
as to whether or not the planning and intentions of the governing/managing
body are likely to result in the requirements being met.

Phase 3 Compliance
Verification

This is sought
to confirm that the plans and intentions which the school/centre has
to meet the requirements are in fact being put in to action.

Assurance
Audit Report

The report
is objective, clear and concise. The tone is positive.
The actions required for compliance are clearly sourced and written
in ways which make it clear to governing /managing bodies what is required
of them. Suggested developments are stated and should lead to
improved practices.

Findings
for State and Integrated Schools

Where appropriate,
findings are reported under each of the following sub-headings:

  • Board Administration
  • Curriculum Management
  • Student Support
  • Provision of Education
    for Senior Students
  • Management of
    Maori Education
  • Personnel Management
  • Financial Management
  • Asset Management
  • Attached Units/Teachers

Findings
for Private Schools

The following
sub-headings are used:

  • Administration
  • Curriculum Management
  • Personnel Management
  • Asset Management

Findings
for Early Childhood Services

The following
sub-headings are used:

  • Partnership and
    Consultation
  • Curriculum
  • Personnel Management
  • Health, Safety
    and Environment
  • Administration

Actions Required
to Meet Legal Obligations and Undertakings

Appropriate
actions that the governing/managing body needs to undertake in order
to meet their legal obligations and undertakings are stated in specific
terms along with suggestions for further development that would add
value over and above the standard requirements.

APPENDIX 8: DETAILS ON EFFECTIVENESS REVIEWS

Definition

An effectiveness
review is an evaluation of student achievement and the impact of the
teaching services and management practices within a school on that achievement.
It is concerned with determining the extent to which the school knows
about student achievement and uses that information to impact positively
on students' learning.

Purpose

Effectiveness
reviews are undertaken to :

  • inform the Minister
    about the impact a school is making to the educational performance of
    students and the merits of continued public investment;
  • inform the decision
    making of the board of trustees, management, staff of a school about
    the school's ability to impact positively on the educational achievement
    of all students;
  • analyse and report
    on the conditions and factors in a school which impact on student achievement;
  • inform parents,
    caregivers, other interested parties and the general public about a
    school's impact on student achievement;
  • provide information
    which can be analysed by the office to inform policy development and
    decision making by government, and for reporting on aspects of the national
    education system.

Objectives

Effectiveness
reviews therefore:

  • evaluate the schools
    expectations of student achievement;
  • verify the difference
    demonstrated in student achievement;
  • evaluate the impact
    of the school on student achievement during the time the students have
    been enrolled at the school;
  • evaluate the progress
    made by students against the achievement objectives of the New Zealand
    curriculum and local objectives
  • evaluate the competence
    of the school in monitoring, assessing and analysing information to
    improve student achievement;
  • identify and evaluate
    a range of factors present within the school which impact on student
    achievement.

APPENDIX 9: ACCOUNTABILITY REVIEWS TO BE INTRODUCED
IN 1998

This is a more
focused approach to evaluating how early childhood centres, schools
and other education providers (eg home schoolers) discharge the various
accountabilities placed on them by the charter, legislation and regulations.
The aim is to concentrate on the quality of performance
of schools or centres delivery of education. This will be achieved
by focusing on the delivery of the National Administration Guidelines
and identifying the risks to educational achievement.

Review Officers
skills will be deployed as a national resource and targeted for greatest
effect.

Key areas for
investigation and analysis will be based on performance information
provided through several new sources:

  • an annual board
    performance declaration;
  • schools' and
    centres' self review;
  • statistics held
    by ERO, MoE and NZQA.

This information
is in addition to the information from previous ERO reviews.

Accountability
review reports will address in specific terms, both consumer (parent)
interest and government purchase and ownership interest. The reports
will cover:

  • quality of education
    received by students; and
  • the performance
    of governing/managing bodies in:

-
managing their employees, assets and resources;

-
monitoring the quality of their educational services and

-
communicating with and responding to their communities.

Changing
Focus

There will
be greater reliance on institution generated data which includes an
annual board declaration and documentation from the self review processes
undertaken. The frequency will remain the same i.e. 3-4 yearly
and more frequent when required.

Performance
Comparison

Accountability
Reviews will allow examination of a school's or centre's performance
and progress over time as well as comparisons with other providers.

Note

Further details
will be available prior to accountability review processes being put
in place in 1988.

APPENDIX 10: TYPES OF PERFORMANCE INDICATORS:
A GUIDE

The Panel has used the literature
to develop the points outlined in Appendix 10 for schools to consider
when constructing their Performance Indicators.

Purpose of Performance Indicators

To provide a standard (i.e.
a baseline) or criterion (i.e. a topic) against which progress and accomplishment
can be determined.

Areas for Use of Indicators

  • Curriculum provision
  • Financial management
  • Human resource
    management
  • Pastoral care
  • Information management
  • Community involvement

Nature of Indicators

Quantitative Indicators
- usually countable or measurable (eg achievement rates, cash investments,
time allocations); normally collected without great difficulty; and
able to be used for cross-comparisons.

Qualitative Indicators
- usually more descriptive and sometimes in the form of comments (eg
reasons for choices, comments on student performance, lists of activities);
often more difficult to collect with reliability.

Indicators might be concerned
with inputs - which enable the system to operate (eg resources
- money, space, time, people), processes
- which determine what is done during activities such as teaching, learning
and managing (eg choices available to students, activities in specific
programmes), outputs - which indicate the results of activities
(eg achievement rates, skills acquired, resources used) or outcomes
- which indicate the impact or implications of activities (eg. social
development, proceeding to further education, staff promotion).

Principles for Constructing
Indicators

  • As few as necessary
    to achieve their purpose,
  • Related to overall
    aims and objectives,
  • Acceptable to
    users,
  • Reliable, useful,
    standardised, able to be used over time and
  • Capable of identifying
    signs or trends and conveying messages.

Sample Indicator Topics

  • Attendance patterns
  • Student subject
    choices
  • Time spent on
    teaching
  • Reasons for guidance
    referrals
  • Incidence of vandalism
    and graffiti
  • Funding from external
    sources
  • Achievement levels
    in tests and examinations
  • Rates of suspensions
    and expulsions
  • Provision of student
    social facilities
  • Types of staff
    professional development activities
  • Opportunities
    for leadership by staff
  • Teacher involvement
    in professional activities
  • Types of parental
    involvement
  • Communication
    methods
  • Estimates of attitude
    change
  • Student aspirations
  • Entry-exit levels
    of student performance
  • Staff and student
    movement rates
  • Range of extra-curricular
    activities
  • Work activities
    of key staff
  • Use of rewards
    and sanctions
  • Staff qualifications
    and levels of experience
  • Costs/investments
    per student - library, equipment, technology, professional development,
    etc.

Development of Indicators

Schools and centres should
develop indicators across a broad range of areas to ensure coverage
of the National Administration Guidelines. An indicator, typically,
should relate to a clear objective for which a target is specified in
addition to a way of determining the extent to which the target has
been achieved. Indicators might include probable costs.