Accountability of a dispute resolution scheme
Dispute resolution schemes need to be monitored, evaluated and reported on to ensure they remain fit for purpose. Intelligence gathered also provides valuable insights into the regulatory system.
On this page
Deciding how and what
When developing your scheme, you need to consider how the monitoring, evaluation, and reporting will be done, including what data to collect and how, who is responsible, timeframes, and who receives reports.
If these decisions are taken by the scheme, your agency will need to be thinking about how it will influence those decisions. Due to the sensitive nature of disputes, and confidentiality and privacy issues in data collection, anonymising data and compliance with the Privacy Act 1992 will be important.
Benefits of monitoring and evaluating
The benefits of monitoring and evaluating a scheme include:
- identifying whether the scheme is doing the right thing and doing it right
- identifying any improvements required to the scheme
- ensuring accountability of the scheme and providers
- providing data to report to stakeholders, including through case studies
- providing early warning of emerging issues and helping to guard against future regulatory failure.
For example…
Issues in the building sector such as those associated with weathertightness and the use of steel mesh in construction might have been picked up earlier if there had been close monitoring and evaluation of building complaints and subsequent disputes.
Monitoring using a database
Monitoring involves collecting and recording scheme data on an ongoing basis and is best done using a database.
The types of data collected will depend on the nature of the complaints and disputes,what insights you want to get from the scheme and why you need them. The following data is a suggested minimum:
- who is using the scheme and their demographics the number and nature of complaints and disputes
- the timeframes for processing complaints and disputes
- the outcomes of complaints and disputes
- any escalation of disputes (eg, unresolved disputes going to court).
You should set clear targets (eg, timeframes) for relevant data and monitor performance against these targets. However, you should balance achieving targets against the need to ensure appropriate resolution of disputes, particularly early in a new scheme’s operation.
Evaluation - systematic assessment
Evaluation (sometimes also called ‘assessment’ or ‘review’) involves the systematic assessment of a scheme against a set of objectives or criteria, to determine its quality, value, or importance.
You should carry out evaluations regularly to help your agency ensure the scheme is operating as intended.
Many schemes are required to have an independent ‘review’ within a set timeframe (eg, 5 years) after set up.
Have an evaluation plan outlining the purpose of the evaluations, main questions and methodology.
Best practice scheme evaluations should include examination of:
- the process against the dispute resolution best practice principles, and the satisfaction of parties with the process and the outcome
- providers' compliance with any service standards and the appropriateness of those standards
- staff compliance with any established measures, such as timeframes
- feedback from staff, practitioners, users and other stakeholders, on their experiences of the scheme.
It is also preferable that the evaluation of your scheme is independent (ie, undertaken by an external party).
Reporting the results
Reporting involves communicating scheme results to your stakeholders, regularly and systematically. It should include:
- the scheme’s performance against objectives
- the dispute resolution best practice principles
- targets
- outcome trends
- issues arising.
Make it clear who the reporting is for and who is responsible for taking any actions in response to reported issues. Otherwise emerging risks may go unidentified and opportunities for management or improvement may be lost.
The policy team responsible for the scheme should generally assess what action is required, particularly to resolve issues with policy settings or legislation. It may be necessary to bring together different sector participants to discuss these issues and agree on the right course of action to address them.
Complaints mechanism
There needs to be an impartial process to deal with complaints made about the scheme itself. At the most basic level, the person complained about should not be the person investigating the complaint about them.
The process should be clearly documented, including who is responsible and the timeframes for each part of the process.
If the scheme has particularly vulnerable users, or there is a significant power imbalance (eg, residents at a retirement home) or lack of trust in the relationship between the parties (eg, prisoners), it will be necessary to support access to relevant external avenues of complaint (eg, the Ombudsman’s Office).