Assessment and scoring guidance - Research Programmes 2024 Investment Round
Research Programmes will be assessed using the guidance, criteria, and scoring tables below. All applications will be scored against the Excellence and Impact criterion.
On this page
The Impact criterion is split into 2 categories:
- Protect and Add Value
- Transform
Proposals will be assessed and scored either in the Protect and Add Value Impact category or the Transform Impact category.
Excellence Assessment guidance
When assessing Excellence, Assessors will use the information provided below to form an assessment and determine a score.
Specific points to note for Excellence
Consider Excellence in the context of:
- Research horizons: Early-stage research may pose higher scientific or technical risk than later-stage research. Both approaches are valid.
- Areas of research: Excellent research should be appropriate to the relevant discipline(s).
Dissemination
Risk
Risk management
Novelty
Innovation
Well-positioned
A credible research plan
Skills mix
Excellence: Science Criterion (25% weighting)
Research should be well-designed, involve risk and/or novelty, and leverage additional value from wider research.
When assessing the Science criterion, Assessors will consider how well the proposal addresses each of the following questions:
- Will the research, science or technology, or related activities progress and disseminate new knowledge?
- Does the proposal have a well-designed research plan and a credible approach to risk management?
- Is the proposal ambitious in terms of scientific risk, technical risk, novelty and/or innovative approaches?
- Is the proposal well-positioned in the domestic and international research context?
- Does the proposal recognise the distinctive research, science and innovation contributions of Māori people, knowledge and resources, including Mātauranga Māori?
The scores in the table below range from 1 (low quality) to 7 (high quality).
| Score | Keywords | Example |
| 1 |
None Not/no Not fit for purpose Negligible Missing Not credible |
No additional value is leveraged from wider research. No new knowledge will be progressed or disseminated. The proposal design is not fit for purpose. Risk is not managed. There is no risk and/or novelty. There is no recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 2 | Minimal Poor Little Inadequate Insufficient Doubtful Lacking Unlikely |
Little additional value is leveraged from wider research. Minimal new knowledge will be progressed or disseminated. The proposal design is poor. Risk management is doubtful. There is minimal risk and/or novelty. There is insufficient recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 3 | Limited Uncertain Some Partial Incomplete Lacks detail Possible |
Some additional value is leveraged from wider research. Limited new knowledge will be progressed and disseminated. The proposal design is limited. Risk management lacks detail. There is some risk and/or novelty. There is partial recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 4 | Acceptable Sufficient Suitable Adequate Reasonable Quite likely |
Sufficient additional value is leveraged from wider research. Adequate new knowledge will be progressed and disseminated. The proposal design is sufficient. Risk management is adequate. There is acceptable risk and/or novelty. There is suitable recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 5 | Significant Good Substantial Well Clear Large Probable |
Substantial additional value is leveraged from wider research. Significant new knowledge will be progressed and disseminated. The proposal design is clear. Risk management is good. There is significant risk and/or novelty. There is good recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 6 | Strong High Comprehensive Very good Very well Extensive |
Comprehensive additional value is leveraged from wider research. Extensive new knowledge will be progressed and disseminated. The proposal design is strong. Risk management is very good. There is high risk and/or novelty. There is very good recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 7 | Excellent Exemplary Impressive Outstanding Total Definite |
Outstanding additional value is leveraged from wider research. Impressive new knowledge will be progressed and disseminated. The proposal design is excellent. Risk management is exemplary. There is impressive risk and/or novelty. There is excellent recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
Excellence: Team Criterion (25% weighting)
The proposed team should have the mix of complementary skills, knowledge, and resources to deliver the proposed research, science or technology, or related activities, and to manage risk.
When assessing the Team criterion, Assessors will consider how well the proposal addresses each of the following questions:
- Does the team have the appropriate mix of complementary skills, knowledge, and resources for the research?
- Does the team give confidence in their ability to deliver the proposed research, science, technology, or related activities and manage risks?
- Does the team have the appropriate Māori expertise for the project?
The scores in the table below range from 1 (low quality) to 7 (high quality).
| Score | Keywords | Example |
| 1 |
None Not/no Not fit for purpose Negligible Missing Not credible |
The team has none of the skills, knowledge, or resources needed. The team gives no confidence in their ability to deliver the research or manage risks. Appropriate Māori expertise in the team is missing. |
| 2 | Minimal Poor Little Inadequate Insufficient Doubtful Lacking Unlikely |
The team has an inadequate mix of the skills, knowledge and resources needed. The team gives little confidence in their ability to deliver the research or manage risks. There is minimal appropriate Māori expertise in the team. |
| 3 | Limited Uncertain Some Partial Incomplete Lacks detail Possible |
The team has a limited mix of the skills, knowledge and resources needed. The team gives some confidence in their ability to deliver the research and manage risks. There is limited appropriate Māori expertise in the team. |
| 4 | Acceptable Sufficient Suitable Adequate Reasonable Quite likely |
The team has a suitable mix of the skills, knowledge and resources needed. The team gives adequate confidence in their ability to deliver the research and manage risks. There is reasonable appropriate Māori expertise in the team. |
| 5 | Significant Good Substantial Well Clear Large Probable |
The team has a good mix of the skills, knowledge and resources needed. The team gives good confidence in their ability to deliver the research and manage risks. There is significant appropriate Māori expertise in the team. |
| 6 | Strong High Comprehensive Very good Very well Extensive |
The team has a comprehensive mix of the skills, knowledge and resources needed. The team gives high confidence in their ability to deliver the research and manage risks. There is comprehensive appropriate Māori expertise in the team. |
| 7 | Excellent Exemplary Impressive Outstanding Total Definite |
The team has an impressive mix of the necessary skills, knowledge and resources needed. The team gives outstanding confidence in their ability to deliver the research and manage risks. There is exemplary appropriate Māori expertise in the team. |
Protect and Add Value Impact Assessment guidance
Applications will be assessed for Impact in 1 of 2 categories. Either the Protect and Add Value category or the Transform category. Assessors will use the information provided below to form an assessment and determine a score.
Scale
Extent
Alignment with areas of future value, growth, or critical need
Protect and Add Value Impact Category: Benefit to New Zealand (25% weighting)
Research should have direct and indirect benefits or effect on individuals, communities or society as a whole, including broad benefits to New Zealand’s economy, environment or society.
When assessing the Benefit to New Zealand criterion, Assessors will consider how well the proposal addresses each of the following questions:
- What is the scale and extent of potential direct and indirect benefits from the proposed research, science or technology, or related activities?
- What is the extent of alignment with 1 or more areas of future additional value, growth, or critical need for New Zealand?
- To what extent has the project identified and evaluated the potential impacts for Māori?
Specific points to note for Protect and Add Value: Benefit to New Zealand
Consider Impact in the context of the breadth/extent of the proposed benefits, which may include aspects that go beyond the direct benefits associated with the output of the research. These can include:
- benefits across multiple sectors
- faster uptake of results in multiple areas
- improved state of the environment
- potential to scale up regional initiatives to nation-wide implementation
- consistency of standards or approaches for regulators
- improved social well-being
- better use of resources
- preservation or enhancement of cultural heritage and values
- more efficient processes
- upskilling industry
- support for emerging new sectors.
The scores in the table below range from 1 (low quality) to 7 (high quality).
| Score | Keywords | Example |
| 1 |
None Not/no Not fit for purpose Negligible Missing Not credible |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is negligible. The outcomes have no alignment with any areas of future additional value, growth, or critical need. The project has not identified and evaluated the potential impacts for Māori. |
| 2 | Minimal Poor Little Inadequate Insufficient Doubtful Lacking Unlikely |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is minimal. The outcomes have doubtful alignment with 1 or more areas of future additional value, growth, or critical need. The project poorly has identified and evaluated the potential impacts for Māori. |
| 3 | Limited Uncertain Some Partial Incomplete Lacks detail Possible |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is limited. The outcomes have some alignment with 1 or more areas of future additional value, growth, or critical need. The project has partially identified and evaluated the potential impacts for Māori. |
| 4 | Acceptable Sufficient Suitable Adequate Reasonable Quite likely |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is adequate. The outcomes have reasonable alignment with 1 or more areas of future additional value, growth, or critical need. The project has adequately identified and evaluated the potential impacts for Māori. |
| 5 | Significant Good Substantial Well Clear Large Probable |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is significant. The outcomes have good alignment with 1 or more areas of future additional value, growth, or critical need. The project has clearly identified and evaluated the potential impacts for Māori. |
| 6 | Strong High Comprehensive Very good Very well Extensive |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is extensive. The outcomes have very good alignment with 1 or more areas of future additional value, growth, or critical need. The project has comprehensively identified and evaluated the potential impacts for Māori. |
| 7 | Excellent Exemplary Impressive Outstanding Total Definite |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is outstanding. The outcomes have outstanding alignment with 1 or more areas of future additional value, growth, or critical need. The project has outstandingly identified and evaluated the potential impacts for Māori. |
Credible implementation pathway(s)
Strength of the relationships
Protect and Add Value Impact Category: Implementation Pathway(s) (25% weighting)
Research should have a credible pathway(s) to deliver public benefit to New Zealand that is not limited to a single firm or end-user and has a line of sight to impact.
When assessing the Implementation Pathway(s) criterion, Assessors will consider how well the proposal addresses each of the following questions:
- Does the proposal provide credible end- or next-user information, to confirm that the implementation pathway(s) is appropriate for the state of the sector or the stage of the research?
- Does the proposal provide a credible implementation pathway(s) to deliver benefits to New Zealand, not limited to a single end-user?
- Is there evidence of a strong relationship with end- or next-users and stakeholders?
- Is the mix of skills and experience within the team complementary and relevant to achieving impact from what is proposed?
- Does the proposal include sufficient input from Māori that is adequately resourced, at the appropriate stage(s) of the project, to ensure effective implementation?
Specific points to note for Protect and Add Value: Implementation Pathway(s)
- creation of a research platform which has additional utility for new users
- job creation, for example, via new start-ups
- development of a cluster of businesses
- multinational business attraction to or retention in New Zealand
- protecting existing markets, or impact on New Zealand’s reputation
- diversification of the economy.
Note: If the Implementation Pathway(s) is limited to a single firm or end-user, the Implementation Pathway(s) score must be 1.
The scores in the table below range from 1 (low quality) to 7 (high quality).
| Score | Keywords | Example |
| 1 |
None Not/no Not fit for purpose Negligible Missing Not credible |
The implementation pathway(s) is not credible because:
The proposal does not include appropriate input from Māori. |
| 2 | Minimal Poor Little Inadequate Insufficient Doubtful Lacking Unlikely |
The implementation pathway(s) has minimal credibility because:
The proposal has little input from Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
| 3 | Limited Uncertain Some Partial Incomplete Lacks detail Possible |
The implementation pathway(s) has partial credibility because:
The skills and experience of the team, relevant to achieving impact, are limited. The proposal has some input from Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
| 4 | Acceptable Sufficient Suitable Adequate Reasonable Quite likely |
The implementation pathway(s) has acceptable credibility because:
The skills and experience of the team, relevant to achieving impact, are suitable. The proposal has reasonable input from Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
| 5 | Significant Good Substantial Well Clear Large Probable |
The implementation pathway(s) has good credibility because:
The skills and experience of the team, relevant to achieving impact, are good. The proposal has significant input from Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
| 6 | Strong High Comprehensive Very good Very well Extensive |
The implementation pathway(s) has very good credibility because:
The skills and experience of the team, relevant to achieving impact, are very good. The proposal has extensive input from Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
| 7 | Excellent Exemplary Impressive Outstanding Total Definite |
The implementation pathway(s) has outstanding credibility because:
The skills and experience of the team, relevant to achieving impact, are outstanding. The proposal has impressive engagement with Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
Specific points to note for the Transform category
Proposals in the Transform category should meet the following criteria:
- Outcomes - Is the new, or changed, technology, process, practice, business model or policy, that is enabled by the research, a Radical Change and/or a leap in performance versus the status quo?; and
- Impact - Could the research ultimately lead to a transformational change within the New Zealand economy, society or environment by, for example, creating or disrupting economic activities, creating a new sustainable resource use or eliminating environmental damage, or changing the character of risks and opportunities faced by individuals and society?
Important note: Proposals that do not meet both criteria should be submitted under the ‘Protect and Add Value’ category.
Scale
Extent
Alignment with areas of future value, growth, or critical need.
Transform Impact Category: Benefit to New Zealand (25% weighting)
Research should have direct and indirect benefits, or effect on individuals, communities, or society as a whole, including broad benefits to New Zealand’s economy, environment or society.
When assessing the Benefit to New Zealand criterion, Assessors will consider how well the proposal addresses each of the following questions:
- To what extent are the outcome(s) enabled by the research a Radical Change and/or a leap in performance versus the status quo, and will the impact also transform New Zealand’s economy, society or environment?
- To what extent will the outcomes of the proposal align with 1 or more areas of future value, growth, or critical need for New Zealand?
- To what extent has the project identified and evaluated the potential impacts for Māori?
The focus of assessment for proposals in the Transform category is on the nature rather than the size of the impact. Evidence of a large impact will not be taken as conclusive evidence of a transformational impact.
Specific points to note for the Transform category: Benefit to New Zealand
Note: these specific points are the same as the Protect and Add Value category: Benefit to New Zealand.
Consider Impact in the context of the breadth/extent of the proposed benefits, which may include aspects that go beyond the direct benefits associated with the output of the research. These can include:
- benefits across multiple sectors
- faster uptake of results in multiple areas
- improved state of the environment
- potential to scale up regional initiatives to nation-wide implementation
- consistency of standards or approaches for regulators
- improved social well-being
- better use of resources
- preservation or enhancement of cultural heritage and values
- more efficient processes
- upskilling industry
- support for emerging new sectors.
The scores in the table below range from 1 (low quality) to 7 (high quality).
| Score | Keywords | Example |
| 1 |
None Not/no Not fit for purpose Negligible Missing Not credible |
The outcomes enabled by the research:
The project has not identified and evaluated the potential impacts for Māori. |
| 2 | Minimal Poor Little Inadequate Insufficient Doubtful Lacking Unlikely |
The outcomes enabled by the research:
The project has poorly identified and evaluated the potential impacts for Māori. |
| 3 | Limited Uncertain Some Partial Incomplete Lacks detail Possible |
The outcomes enabled by the research:
The project has partially identified and evaluated the potential impacts for Māori. |
| 4 | Acceptable Sufficient Suitable Adequate Reasonable Quite likely |
The outcomes enabled by the research:
The project has adequately identified and evaluated the potential impacts for Māori. |
| 5 | Significant Good Substantial Well Clear Large Probable |
The outcomes enabled by the research:
The project has clearly identified and evaluated the potential impacts for Māori. |
| 6 | Strong High Comprehensive Very good Very well Extensive |
The outcomes enabled by the research:
The project has comprehensively identified and evaluated the potential impacts for Māori. |
| 7 | Excellent Exemplary Impressive Outstanding Total Definite |
The outcomes enabled by the research:
The project has outstandingly identified and evaluated the potential impacts for Māori. |
Credibility
Team impact track record
Transform Impact Category: Implementation Pathway(s) (25% weighting)
Research should have (a) credible indicative Implementation Pathway(s) to deliver public benefit to New Zealand that is not limited to a single firm or end-user, and an understanding of the barriers to impact.
When assessing the Implementation Pathway(s) criterion, Assessors will consider how well the proposal addresses each of the following questions:
- Does the proposal demonstrate an understanding of the enablers and barriers in potential implementation pathway(s) to deliver public benefits to New Zealand?
- Does the proposal provide a credible indicative implementation pathway(s) to deliver benefits to New Zealand, not limited to a firm or single end-user?
- Have the indicative end- or next-users, beneficiaries, and stakeholders been identified?
- Is the mix of skills and experience within the team complementary and relevant to achieving impact from what is proposed?
- Does the proposal include sufficient input from Māori that is adequately resourced, at the appropriate stage(s) of the project, to ensure effective implementation?
Specific points to note for the Transform Category: Implementation Pathway(s)
Proposals submitted in the Transform category should meet both the following criteria:
- Outcomes – is the new, or changed technology, process, practice, business model, or policy, that is enabled by the research, a Radical Change and/or a leap in performance versus the status quo; and
- Impact – could the research ultimately lead to a transformational change within the New Zealand economy, society, or environment by, for example, creating or disrupting economic activities, creating a new sustainable resource use or eliminating environmental damage, or changing the character of risks and opportunities faced by individuals and society.
The focus of assessment for proposals in the Transform category is on the nature rather than the size of the impact. Evidence of a large impact will not be taken as conclusive evidence of a transformational impact.
Note: If the Implementation Pathway(s) is limited to a single firm or end-user, the Implementation Pathway(s) score must be 1.
The scores in the table below range from 1 (low quality) to 7 (high quality).
| Score | Keywords | Example |
| 1 |
None Not/no Not fit for purpose Negligible Missing Not credible |
The indicative implementation pathway(s) is not credible because:
The proposal does not include appropriate input from Māori. |
| 2 | Minimal Poor Little Inadequate Insufficient Doubtful Lacking Unlikely |
The indicative implementation pathway(s) has minimal credibility because:
The skills and experience of the team, relevant to achieving impact, are doubtful. The proposal has little input from Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
| 3 | Limited Uncertain Some Partial Incomplete Lacks detail Possible |
The indicative implementation pathway(s) has partial credibility because:
The skills and experience of the team, relevant to achieving impact, are limited. The proposal has some input from Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
| 4 | Acceptable Sufficient Suitable Adequate Reasonable Quite likely |
The indicative implementation pathway(s) has acceptable credibility because:
The skills and experience of the team, relevant to achieving impact, are suitable. The proposal has reasonable input from Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
| 5 | Significant Good Substantial Well Clear Large Probable |
The indicative implementation pathway(s) has good credibility because:
The skills and experience of the team, relevant to achieving impact, are substantial. The proposal has significant input from Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
| 6 | Strong High Comprehensive Very good Very well Extensive |
The indicative implementation pathway(s) has very good credibility because:
The skills and experience of the team, relevant to achieving impact, are comprehensive. The proposal has extensive input from Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
| 7 | Excellent Exemplary Impressive Outstanding Total Definite |
The indicative implementation pathway(s) has outstanding credibility because:
The skills and experience of the team, relevant to achieving impact, are exemplary. The proposal has outstanding input from Māori, at the appropriate stage(s) or levels, to ensure effective implementation. |
Vision Mātauranga assessment
Through the Vision Mātauranga Policy, we encourage appropriate and distinctive research arising from the interface between Māori knowledge and science to deliver effective and innovative products, services, and outcomes for New Zealand. Vision Mātauranga is designed to inspire researchers to find innovative responses to opportunities and solutions to issues and needs facing our country. This includes encouraging and building the capability, capacity and networks of Māori and the research community to collaborate in carrying out this work. The policy applies across, and is integrated within, all MBIE science investment mechanisms.
Proposals can give effect to the Vision Mātauranga policy by considering the various ways in which Māori communities, knowledge and resources may be enabled, mobilised, and empowered in research, science and innovation. The manner and extent to which proposals give effect to Vision Mātauranga may differ depending on the research.
Very strong applications, giving effect to Vision Mātauranga, may be Māori-led or co-led. Strong applications may have Māori researchers or traditional knowledge holders as part of the team; or may work meaningfully with Māori communities, interest groups, businesses, or key individuals. Strong applications enabling Māori knowledge may use kaupapa Māori approaches or draw deeply on mātauranga Māori. Others may incorporate Māori principles or perspectives into the research.
Enabling Māori people will be an important part of giving effect to Vision Mātauranga, as the appropriate understanding, development, and protection of Māori knowledge and resources will have Māori input.
Research positioned to give effect to Vision Mātauranga will create distinctive and meaningful impact for Māori and for New Zealand in any or all of the following ways:
- Enhancing productivity and performance of Māori and non-Māori enterprise through new products, processes, and services.
- Achieving environmental sustainability by utilising distinctive Māori relationships with the environment.
- Improving health and social well-being for Māori (the Endeavour Fund does not fund research with primarily health outcomes but will support this as a secondary outcome as long as the main health outcomes contribute less than 50% of the proposal’s outcomes).
- Generating new knowledge at the interface between indigenous knowledge (including mātauranga Māori) and research, science and innovation.
- Generating new indigenous knowledge (including mātauranga Māori) and research, science, and innovation.
The Excellence and Impact assessments include Vision Mātauranga. Assessors will be asked:
In their opinion, how well will the project give effect to the Vision Mātauranga Policy (for example, realise the potential of Māori people, knowledge, and resources), and reflect genuine, fit-for-purpose approaches? Consider the specific activities, output and outcomes described, and whether they will create impact for Māori.
Assessors will select from the following to best describe their opinion:
Exceptional / Very Well / Well / Not Well / Absent.