Survey Description
Methods
- Sampling
- Since 1999, the North Dakota BRFSS has been conducted using disproportionate stratified sampling methodology. This method of probability sampling involves assigning sets of 100 telephone numbers with the same area code, prefix, and first two digits of the suffix and all possible combinations of the last two digits ("hundred blocks") into two strata (1999-2001,
2003-2006) or three strata (2002) based on the likelihood that the number represents an actual household. Different strata are sampled at different rates to improve calling efficiency.
Approximately the same number of people is called each month throughout each calendar year to reduce bias caused by seasonal variation of health risk behaviors. Potential working telephone numbers are dialed during three separate calling periods (daytime, evening and weekends) for a total of 15 call attempts before being replaced. Upon reaching a valid household number, one household member age 18 or older is randomly selected. If the selected respondent is not available, an appointment is made to call at a later time or date. Because respondents are selected at random and no identifying information is requested, all responses to this survey are anonymous. - Data Weighting
- Weighting is a process by which the survey data are adjusted to account for unequal selection probability and response bias and to represent more accurately the population from which the sample was drawn. The responses of each person interviewed are assigned a weight that accounts for the density stratum, the number of telephone numbers in the household, the number of adults in the household and the demographic distribution of the sample.
- Data Reliability
- Telephone interviewing has been demonstrated to be a reliable method for collecting behavioral risk data and can cost three to four times less than other interviewing methods, such as mail-in interviews or face-to-face interviews. The BRFSS methodology has been utilized and evaluated by the CDC and the participating states since 1984. Content of survey questions, questionnaire design, data collection procedures, surveying techniques and editing procedures have been evaluated thoroughly to maintain overall data quality and to lessen the potential for bias within the population sample.
Limitations
- Sampling
- The BRFSS survey samples the population using a technique discussed in the methodology section. Sampling yields results that are an estimate of the true answer for the entire population. The more people interviewed, the greater the precision of the estimate will be. When the data are subdivided to look at sub-populations (e.g., an age subgroup) these estimates will be less precise; if the number of people interviewed was small because the subgroup represents a small fraction of the population (e.g., diabetics less than 30 years old), the estimate may become too uncertain to be of value. Because the survey is conducted by telephone, people without telephones cannot be reached. Since phone ownership is highly correlated to income, people without a phone are more likely to have low incomes than are people with a telephone. This potentially will affect questions with responses that are highly dependent on income (e.g., health insurance) more than other questions. However, because phone ownership is high in North Dakota (greater than 95%), the resulting bias for most analyses is small. Because some populations, such as racial or ethnic subgroups, have lower phone ownership, results for these groups may be impacted to a greater extent. In addition, with ongoing changes in telephone technology there are more and more households that have cellular telephones and no traditional land lines in their homes. These households are presently not in the sampling frame for BRFSS, which may bias the survey results, especially if the percentage of cellular-telephone-only households increases in the coming years. The BRFSS is continuing to study the impact of cellular phones on survey response and the feasibility of various methods for data collection to complement present survey methods.
- Questionnaire Design and Administration
- How a question is written and which questions preceded it in the questionnaire can influence responses in unpredictable ways. Not all the questions used in the survey have been tested to ensure that all participants understand the intended meaning. Those that come from modules created by the Centers for Disease Control and Prevention usually have been tested, while those in state modules may or may not have been tested, depending upon the source of the question. Furthermore, not all questions are equally easy for respondents to answer. While it may be easy for a respondent to provide a personal opinion, it may be much harder to recall a past event (last mammogram) or provide factual information (household income).
Interviewers are trained and monitored to ensure that they administer the survey in a neutral voice and read the written question verbatim and without comment. Nonetheless, it is possible for the interviewer to bias the results through tone of voice or administration technique. Coding errors also may occur if the interviewer types in the wrong response to the question. In addition, the person being interviewed may alter his or her response to give the interviewer the most socially acceptable answer. - Response Rate
- The following table includes the Council of American Survey Research Organizations (CASRO) response rates for the North Dakota BRFSS by survey year:
Survey Year
Sample Size
CASRO response rate
1996
1811
86%
1997
1802
79%
1998
1803
73%
1999
1981
76%
2000
1918
50%
2001
2510
58%
2002
2996
66%
2003
3026
59%
2004
3045
62%
2005
4010
58%
The CASRO formula is based on the number of interviews completed, the number of households reached and the number of households with unknown eligibility status (e.g., households that were called 15 times but where no one in the household was reached). The CASRO response rate is used because, in addition to people who refuse to answer questions, lack of response also can arise because household members are not available despite repeated call attempts or household members refuse to pick up the phone based on what they discern from caller ID. The bias from nonresponse cannot be removed; it is not possible to know if those who refused to respond would have answered the questions in approximately the same ways as those who responded.
- Confounding and Causation
- Personal characteristics presented on this website are univariate (i.e., examine each risk factor in relationship to only one characteristic at a time); however, the complexity of health associations are not fully represented by examining single relationships. For example, an examination of heart disease and employment status might show a greater prevalence of heart disease among people who are retired than among people who are employed. However, people who are retired are expected to have a greater average age than people who are employed; consequently, this relationship might entirely disappear if we removed the effects of age. (If this were the case, we would say that the relationship between heart disease and employment status was being confounded by age.)
This website does not attempt to explain the causes of the health effects examined. For instance, BRFSS data might show a higher prevalence of heart disease among smokers, but one should not conclude from this that smoking causes heart disease. That smoking is indeed a causal factor for heart disease is apparent from a large body of scientific data, but that is not a conclusion that can be drawn from a cross-sectional survey such as this. Rather this is a "snapshot" of disease, risk factors and population characteristics for adult residents of North Dakota at a point in time.