Assessment and scoring guidance - Smart Ideas 2024 investment round – Endeavour Fund
Smart Ideas Concepts and Full Proposals will be assessed using the guidance, criteria, and scoring tables below.
On this page
Excellence Assessment guidance (Smart Ideas Concepts and Full Proposals)
When assessing Excellence, assessors will use the information provided below to form an assessment and determine a score.
Specific points to note for Excellence
Consider Excellence in the context of:
- Research horizons: Early-stage research may pose higher scientific or technical risk than later-stage research. Both approaches are valid.
- Areas of research: Excellent research should be appropriate to the relevant discipline(s).
Dissemination
Risk
Risk management
Novelty
Innovation
Well-positioned
A credible research plan
Skills mix
Excellence: Science Criterion (50% weighting)
Research should be well-designed, involve risk and/or novelty, and leverage additional value from wider research.
When assessing the Science criterion, Assessors will consider how well the proposal addresses each of the following questions:
- Will the research, science or technology, or related activities progress and disseminate new knowledge?
- Does the proposal have a well-designed research plan and a credible approach to risk management?
- Is the proposal ambitious in terms of scientific risk, technical risk, novelty and/or innovative approaches?
- Is the proposal well-positioned in the domestic and international research context?
- Does the proposal recognise the distinctive research, science and innovation contributions of Māori people, knowledge and resources, including Mātauranga Māori?
The scores in the table below range from 1 (low quality) to 7 (high quality).
| Score | Keywords | Example |
| 1 |
None Not/no Not fit for purpose Negligible Missing Not credible |
No additional value is leveraged from wider research. No new knowledge will be progressed or disseminated. The proposal design is not fit for purpose. Risk is not managed. There is no risk and/or novelty. There is no recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 2 | Minimal Poor Little Inadequate Insufficient Doubtful Lacking Unlikely |
Little additional value is leveraged from wider research. Minimal new knowledge will be progressed or disseminated. The proposal design is poor. Risk management is doubtful. There is minimal risk and/or novelty. There is insufficient recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 3 | Limited Uncertain Some Partial Incomplete Lacks detail Possible |
Some additional value is leveraged from wider research. Limited new knowledge will be progressed and disseminated. The proposal design is limited. Risk management lacks detail. There is some risk and/or novelty. There is partial recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 4 | Acceptable Sufficient Suitable Adequate Reasonable Quite likely |
Sufficient additional value is leveraged from wider research. Adequate new knowledge will be progressed and disseminated. The proposal design is sufficient. Risk management is adequate. There is acceptable risk and/or novelty. There is suitable recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 5 | Significant Good Substantial Well Clear Large Probable |
Substantial additional value is leveraged from wider research. Significant new knowledge will be progressed and disseminated. The proposal design is clear. Risk management is good. There is significant risk and/or novelty. There is good recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 6 | Strong High Comprehensive Very good Very well Extensive |
Comprehensive additional value is leveraged from wider research. Extensive new knowledge will be progressed and disseminated. The proposal design is strong. Risk management is very good. There is high risk and/or novelty. There is very good recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
| 7 | Excellent Exemplary Impressive Outstanding Total Definite |
Outstanding additional value is leveraged from wider research. Impressive new knowledge will be progressed and disseminated. The proposal design is excellent. Risk management is exemplary. There is impressive risk and/or novelty. There is excellent recognition of the distinctive research, science and innovation contributions of Māori people, knowledge, and resources, including Mātauranga Māori. |
Excellence: Team Criterion (15% weighting)
The proposed team should have the mix of complementary skills, knowledge, and resources to deliver the proposed research, science or technology or related activities, and to manage risk.
When assessing the Team criterion, Assessors will consider how well the proposal addresses each of the following questions:
- Does the team have the appropriate mix of complementary skills, knowledge, and resources for the research?
- Does the team give confidence in their ability to deliver the proposed research, science, technology, or related activities and manage risks?
- Does the team have the appropriate Māori expertise for the project?
The scores in the table below range from 1 (low quality) to 7 (high quality).
| Score | Keywords | Example |
| 1 |
None Not/no Not fit for purpose Negligible Missing Not credible |
The team has none of the skills, knowledge, or resources needed. The team gives no confidence in their ability to deliver the research or manage risks. Appropriate Māori expertise in the team is missing. |
| 2 | Minimal Poor Little Inadequate Insufficient Doubtful Lacking Unlikely |
The team has an inadequate mix of the skills, knowledge and resources needed. The team gives little confidence in their ability to deliver the research or manage risks. There is minimal appropriate Māori expertise in the team. |
| 3 | Limited Uncertain Some Partial Incomplete Lacks detail Possible |
The team has a limited mix of the skills, knowledge and resources needed. The team gives some confidence in their ability to deliver the research and manage risks. There is minimal appropriate Māori expertise in the team. |
| 4 | Acceptable Sufficient Suitable Adequate Reasonable Quite likely |
The team has a suitable mix of the skills, knowledge and resources needed. The team gives adequate confidence in their ability to deliver the research and manage risks. There is reasonable appropriate Māori expertise in the team. |
| 5 | Significant Good Substantial Well Clear Large Probable |
The team has a good mix of the skills, knowledge and resources needed. The team gives good confidence in their ability to deliver the research and manage risks. There is significant appropriate Māori expertise in the team. |
| 6 | Strong High Comprehensive Very good Very well Extensive |
The team has a comprehensive mix of the skills, knowledge and resources needed. The team gives high confidence in their ability to deliver the research and manage risks. There is comprehensive appropriate Māori expertise in the team. |
| 7 | Excellent Exemplary Impressive Outstanding Total Definite |
The team has an impressive mix of the necessary skills, knowledge and resources needed. The team gives outstanding confidence in their ability to deliver the research and manage risks. There is exemplary appropriate Māori expertise in the team. |
Impact Assessment guidance (Smart Ideas Full Proposals only)
When assessing Impact, assessors will use the information provided below to form an assessment and determine a score.
Specific points to note for benefit to New Zealand
Consider Impact in the context of the breadth/extent of the proposed benefits, which may include aspects that go beyond the direct benefits associated with the output of the research. These can include:
- benefits across multiple sectors
- faster uptake of results in multiple areas
- improved state of the environment
- potential to scale up regional initiatives to nation-wide implementation
- consistency of standards or approaches for regulators
- improved social well-being
- better use of resources
- preservation or enhancement of cultural heritage and values
- more efficient processes
- upskilling industry
- support for emerging new sectors.
Scale
Extent
Alignment with areas of future value, growth, or critical need.
Impact: Benefit to New Zealand (25% weighting)
Research should have direct and indirect benefits or effect on individuals, communities, or society as a whole, including broad benefits to New Zealand’s economy, environment or society.
When assessing the Benefit to New Zealand criterion, Assessors will consider how well the proposal addresses each of the following questions:
- What is the scale and extent of potential direct and indirect benefits from the proposed research, science or technology, or related activities?
- What is the extent of alignment with 1 or more areas of future additional value, growth, or critical need for New Zealand?
- To what extent has the project identified and evaluated the potential impacts for Māori?
The scores in the table below range from 1 (low quality) to 7 (high quality).
| Score | Keywords | Example |
| 1 |
None Not/no Not fit for purpose Negligible Missing Not credible |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is negligible. The outcomes have no alignment with any areas of future additional value, growth, or critical need. The project has not identified and evaluated the potential impacts for Māori. |
| 2 | Minimal Poor Little Inadequate Insufficient Doubtful Lacking Unlikely |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is minimal. The outcomes have doubtful alignment with 1 or more areas of future additional value, growth, or critical need. The project has poorly identified and evaluated the potential impacts for Māori. |
| 3 | Limited Uncertain Some Partial Incomplete Lacks detail Possible |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is limited. The outcomes have some alignment with 1 or more areas of future additional value, growth, or critical need. The project has partially identified and evaluated the potential impacts for Māori. |
| 4 | Acceptable Sufficient Suitable Adequate Reasonable Quite likely |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is adequate. The outcomes have reasonable alignment with 1 or more areas of future additional value, growth, or critical need. The project has adequately identified and evaluated the potential impacts for Māori. |
| 5 | Significant Good Substantial Well Clear Large Probable |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is significant. The outcomes have good alignment with 1 or more areas of future additional value, growth, or critical need. The project has clearly identified and evaluated the potential impacts for Māori. |
| 6 | Strong High Comprehensive Very good Very well Extensive |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is extensive. The outcomes have very good alignment with 1 or more areas of future additional value, growth, or critical need. The project has comprehensively identified and evaluated the potential impacts for Māori. |
| 7 | Excellent Exemplary Impressive Outstanding Total Definite |
The scale and extent of the potential (direct or indirect) benefits of the proposed work is outstanding. The outcomes have outstanding alignment with 1 or more areas of future additional value, growth, or critical need. The project has outstandingly identified and evaluated the potential impacts for Māori. |
Specific points to note for Implementation Pathway(s)
- creation of a research platform which has additional utility for new users
- job creation, for example, via new start-ups
- development of a cluster of businesses
- multinational business attraction to or retention in New Zealand
- protecting existing markets, or impact on New Zealand’s reputation
- diversification of the economy.
Credible implementation pathway(s)
Strength of the relationships
Impact: Implementation Pathways (10% weighting)
Research should have a credible indicative implementation pathway(s) to deliver public benefit to New Zealand that is not limited to a single firm or end-user, and which may be uncertain in nature.
When assessing the Implementation Pathway(s) criterion, Assessors will consider how well the proposal addresses each of the following questions:
- Does the proposal provide credible end- or next-user information, to confirm that the implementation pathway is appropriate for the state of the sector or the stage of the research?
- Does the proposal provide a credible implementation pathway(s) to deliver benefits to New Zealand, not limited to a single end-user?
- Is there evidence of a strong relationship with end or next-users and stakeholders?
- Does the proposal include sufficient input from Māori at the appropriate stage(s) of the project that is adequately resourced, to ensure effective implementation?
Note: If the Implementation Pathway(s) is limited to a single firm or end-user, the Implementation Pathway(s) score must be 1.
The scores in the table below range from 1 (low quality) to 7 (high quality).
| Score | Keywords | Example |
| 1 |
None Not/no Not fit for purpose Negligible Missing Not credible |
The indicative implementation pathway(s) is not credible because:
|
| 2 | Minimal Poor Little Inadequate Insufficient Doubtful Lacking Unlikely |
The indicative implementation pathway(s) has minimal credibility because:
The proposal has little input from Māori at the appropriate stage(s) or levels to ensure effective implementation. |
| 3 | Limited Uncertain Some Partial Incomplete Lacks detail Possible |
The indicative implementation pathway(s) has limited credibility because:
The proposal has some input from Māori at the appropriate stage(s) or levels to ensure effective implementation. |
| 4 | Acceptable Sufficient Suitable Adequate Reasonable Quite likely |
The indicative implementation pathway(s) has acceptable credibility because:
The proposal has adequate input from Māori at the appropriate stage(s) or levels to ensure effective implementation. |
| 5 | Significant Good Substantial Well Clear Large Probable |
The indicative implementation pathway(s) has good credibility because:
The proposal has significant input from Māori at the appropriate stage(s) or levels to ensure effective implementation. |
| 6 | Strong High Comprehensive Very good Very well Extensive |
The indicative implementation pathway(s) has very good credibility because:
The proposal has very good input from Māori at the appropriate stage(s) or levels to ensure effective implementation. |
| 7 | Excellent Exemplary Impressive Outstanding Total Definite |
The indicative implementation pathway(s) has outstanding credibility because:
The proposal includes outstanding engagement with Māori at the appropriate stage(s) or levels to ensure effective implementation. |
Vision Mātauranga assessment
Through the Vision Mātauranga Policy, we encourage appropriate and distinctive research arising from the interface between Māori knowledge and science to deliver effective and innovative products, services, and outcomes for New Zealand. Vision Mātauranga is designed to inspire researchers to find innovative responses to opportunities and solutions to issues and needs facing our country. This includes encouraging and building the capability, capacity and networks of Māori and the research community to collaborate in carrying out this work. The policy applies across, and is integrated within, all MBIE science investment mechanisms.
Proposals can give effect to the Vision Mātauranga policy by considering the various ways in which Māori communities, knowledge and resources may be enabled, mobilised, and empowered in research, science and innovation. The manner and extent to which proposals give effect to Vision Mātauranga may differ depending on the research.
Very strong applications, giving effect to Vision Mātauranga, may be Māori-led or co-led. Strong applications may have Māori researchers or traditional knowledge holders as part of the team; or may work meaningfully with Māori communities, interest groups, businesses, or key individuals. Strong applications enabling Māori knowledge may use kaupapa Māori approaches or draw deeply on mātauranga Māori. Others may incorporate Māori principles or perspectives into the research.
Enabling Māori people will be an important part of giving effect to Vision Mātauranga, as the appropriate understanding, development, and protection of Māori knowledge and resources will have Māori input.
Research positioned to give effect to Vision Mātauranga will create distinctive and meaningful impact for Māori and for New Zealand in any or all of the following ways:
- Enhancing productivity and performance of Māori and non-Māori enterprise through new products, processes, and services.
- Achieving environmental sustainability by utilising distinctive Māori relationships with the environment.
- Improving health and social well-being for Māori (the Endeavour Fund does not fund research with primarily health outcomes but will support this as a secondary outcome as long as the main health outcomes contribute less than 50% of the proposal’s outcomes).
- Generating new knowledge at the interface between indigenous knowledge (including mātauranga Māori) and research, science and innovation.
- Generating new indigenous knowledge (including mātauranga Māori) and research, science, and innovation.
The Excellence and Impact assessments include Vision Mātauranga. Assessors will be asked:
In their opinion, how well will the project give effect to the Vision Mātauranga Policy (for example, realise the potential of Māori people, knowledge and resources), and reflect genuine, fit-for-purpose approaches? Consider the specific activities, output and outcomes described, and whether they will create impact for Māori.
Assessors will select from the following to best describe their opinion:
Exceptional / Very Well / Well / Not Well / Absent.