Survey error for International Visitor Survey
Surveys are used to gather information and produce estimates of different variables, using a sample of the total population. Results are subject to both sampling and non-sampling error.
On this page
Sampling error
Sampling errors arise from estimating a population characteristic by looking at only a sample of the population, rather than the entire population. It refers to the difference between the estimate derived from a sample survey and the 'true' value that would result if everyone in the population was surveyed.
The International Visitor Survey (IVS) targets a sample of 8,900 international visitors. The sampling error in the IVS:
- is different for each estimate in the survey
- generally decreases as the sample size increases
- depends on the size of the population
- depends on the intrinsic variability of the characteristics of interest in the population.
The sampling error for a given sample is unknown, but can be estimated and represented by the margin of error.
Margin of error
A margin of error is the range of values above and below a sample statistic estimate at a certain amount of confidence. It shows how many percentage points your results can differ from the real population value, and is a way to show the uncertainty of a certain statistic — for example, from a poll or survey.
The IVS targets different margins of error
The IVS has been designed to achieve:
- a 5% Relative Margin of Error (RME) for total visitor expenditure
- a 10% RME for expenditure from the top 6 tourism markets (visitors from Australia, United Kingdom, United States, China, Japan and Germany).
As each of these RMEs increase, changes in spending are less likely to be statistically significant, that is, we're less likely to be able to say with confidence that an increase or a decrease in a value has actually occurred.
All these RMEs are calculated at the 95% confidence level.
Relative Margins of Error (RMEs)
The actual RMEs for total spend and mean spend are calculated each quarter for the most commonly reported combinations of country of permanent residence and purpose of visit. The calculation is calculated using a statistical technique called bootstrapping, with 500 replications.
When the RME for a particular set of data are above the target RME, it doesn't mean the data is wrong. Rather it means the confidence interval of the statistic produced will be wider, and therefore there is less confidence that the results are close to the 'true' figure.
Empirical equation for sampling errors
IVS sampling error (measured by RME) is provided for the top level statistics in key pivot tables, but not for other estimates derived from the microdata.
To help with assessing the sampling error of these IVS estimates, we established an empirical equation that can be used to calculate the approximate margin of error, based on the sample size associated with the relevant IVS estimates.
International Visitor Survey Relative Margin of Error Empirical Equation [PDF, 172 KB]
A technical paper is also available, explaining the method of establishing the empirical equation.
Technical paper - empirical equations for IVS relative margin of error [PDF, 395 KB]
Non-sampling errors
Aside from the sampling error associated with the process of selecting a sample, a survey is also subject to a wide variety of other errors — commonly referred to as non-sampling errors. Non-sampling errors are defined as errors arising during the course of all survey activities other than sampling. Unlike sampling errors they are generally difficult to measure.
Some examples of non-sampling errors include:
- Specification error – occurs when the observed variable (what we have measured) differs from the desired variable (what we wanted to measure).
- Frame error – arises in creating, maintaining and using the sampling frame for the selection of survey units. Examples of this could be:
- exclusion of individuals who should be in the population
- inclusion of individuals who should be in the population
- duplicating individuals in the population.
- Measurement error – includes errors arising from respondents, interviewers, imperfect or ambiguous survey questions and any other factors that affect survey response.
- Non-response error – occurs when an individual doesn’t respond or only partially responds to the questionnaire.
- Data processing error – includes mistakes in data entry, editing the data, coding, calculating weights and tabulation of data.
- Model/estimation error – arises from fitting models for different purposes such as imputation, deriving new variables and adjusting data values.
These types of errors are minimised through the use of best survey practice and monitoring.