Survey quality for the International Visitor Survey

Quality assurance metrics

The tables below contain results against quality assurance metrics that are monitored to reduce sampling error in the International Visitor Survey (IVS). These include:

• Relative Margin of Error (RME).
• Sample size.
• Response rate.

See Survey error for more information about types of errors and how they apply to the IVS.

Survey error

Relative Margin of Error (RME)

The Relative Margin of Error represents the degree to which error due to sampling influences the value of the mean we calculate. Given that we calculate RME at the 95% confidence level, if we ran the IVS 20 times, we would expect the true mean to be within one RME from the sample mean 19 times and outside 1 RME from the sample mean once.

The International Visitor Survey is designed to have an RME of:

• 5% for total spend
• 10% for the top 6 markets - Australia, China, USA, UK, Germany and Japan

As margin of error is calculated on the basis of annual data, it is too early to know whether the annual target has been achieved. However, the target was achieved for the year ending March 2020, with the RME for total spend for that year being 3.9%.

As we only have one quarter’s worth of data, RMEs for only 3 groupings have been calculated: total spend, Australia spend and rest of the world spend (everywhere that is not Australia) as sample sizes for most international markets are too small. RMEs are estimates and should be used as an indicator of accuracy when interpreting results, rather than definitive figures. RMEs for these groupings are as follows:

• Total: 8.83%
• Australia: 10%
• Rest of world: 14.5%

Sample size for Quarter 3 (July to September 2022)

The September quarter saw a reduced sample size of 1,360 compared to pre-COVID. The primary reason is the high percentage of non-qualifiers; people approached for the survey who did not qualify for participation, primarily New Zealand residents, and passengers in transit. The number of qualifiers is expected to increase over the coming months as the proportion of non-New Zealand residents travelling to New Zealand increases.

We therefore advise caution when using and interpreting quarterly level results, especially for Q3 2022.

The target overall sample size for the September 2022 quarter was designed to be the same as pre-COVID-19 (see Table 1). No country targets were put in place for the 2022/23 year due to uncertainty of ongoing changes to global travel regulations, flight schedules, and consumer travel preferences. More information on changes to the survey methodology can be found in the Methodology Paper that outlines impact on the IVS due to changes in travel patterns following COVID-19.

International Visitor Survey methodology

Table 1: Sample target for 2022/23 and pre COVID-19

Response rate for the year to March 2022

The response rate represents the percentage of people who complete the survey as a proportion of the total surveys emailed out. The International Visitor Survey was designed with an expected response rate of 30%.

Table 2 presents the response rates by country for the quarter ended September 2022.

Refusal and response rates remain similar to pre-COVID. The majority of qualified visitors surveyed were from Australia. Previously, Australian residents have typically represented between 20-25% of visitors of the sample.

Table 2: Response rates for top 6 international markets