Evaluation of a Modified STOP-BANG Questionnaire for Sleep Apnea in Adults from the Korean General Population
Article information
Abstract
Background and Objective
There are few studies assessing diagnostic accuracy of the STOP-BANG questionnaire, which is a screening tool for detecting obstructive sleep apnea (OSA), in the general population. Our study aimed to develop a modified version of this screening questionnaire and compare its diagnostic accuracy to the standard version in general Korean adults.
Methods
A cross-sectional polysomnographic study was conducted among 2065 male and female participants aged 48–79 years, who were members of a population-based cohort study. Half of them were assigned to the exploratory sample and the other half to the validation sample.
Results
In this study, 958 individuals (487 in an exploratory sample) were identified to have OSA. To develop a model in an exploratory sample, stepwise analysis was used with information from the standard version and additional variables; snoring, witnessed breathing cessation, hypertension, body mass index > 25 kg/m2, age groups, male, diabetes, and waist circumference > 85 cm were included in a final modified model. In the evaluation of diagnostic accuracy in a validation sample, the modified model showed higher sensitivity [79.1%, 95% confidence interval (CI): 77.3, 80.9] in detecting severe OSA than that of the standard version model (66.0%, 95% CI: 64.0, 68.0). Overall accuracy indicated as area under the receiver-operating characteristic curve for the modified model was greater (p = 0.001) than that of the standard version model.
Conclusions
These findings suggest that the modified version of the STOP-BANG questionnaire may be useful in detecting severe OSA in general adults.
INTRODUCTION
The prevalence of obstructive sleep apnea (OSA) has gradually increased, partly due to population aging and a rising prevalence of risk factors such as obesity, and thus it is now considered to be a prevalent disease [1,2]. Moreover, OSA is recognized as an emerging public health problem because it has been reported to be associated with cardiovascular and metabolic diseases [3,4] and a large number of OSA cases remain undiagnosed and untreated in the community [5]. It is recommended that a diagnosis of OSA should be established by laboratory-based or home-based polysomnography (PSG) [6]. PSG provides objective and quantitative data on sleep status and the severity of OSA, but it can be costly and time-consuming, as well as uncomfortable, and disruptive to sleep for patients undergoing the test. In addition, because PSG-related resources are unavailable in most primary care settings, patients at risk of OSA are less likely to be identified and have limited access to proper care [7]. Use of screening questionnaires such as the STOP-BANG and Berlin questionnaires has been suggested as an alternative approach for identifying potential cases of OSA in the primary care setting [8]. The STOP-BANG questionnaire in particular seems to be a useful screening tool for OSA because it is simple and has been validated [9]. A number of studies evaluating the diagnostic accuracy of the STOP-BANG questionnaire among sleep clinic patients showed the sensitivity greater than 80% for detecting all cases of OSA and greater than 95% for detecting severe cases, although the specificity was found to be less than 50% [10-17]. On the other hand, there are few studies testing the STOP-BANG questionnaire in the general population [18,19]. These studies have observed lower sensitivity and higher specificity (between 60% and 70%) for severe OSA compared to studies performed in sleep clinic patients [18,19]. To improve such a modest sensitivity, Tan et al. [19] modified the questionnaire, using a lower cutoff point for body mass index (BMI). However, they found similar sensitivity for the standard and modified version models. Thus, further studies are warranted to develop a modified version model that can improve the predictive ability of the STOP-BANG questionnaire, making it more applicable and useful for detecting OSA in the general population.
The objectives of this study were to develop predictive models using potential risk factors of OSA in addition to the components of the standard STOP-BANG questionnaire, construct a scoring method for the modified version models, and compare diagnostic accuracy parameters in the standard and modified versions for detecting all and severe cases of OSA in the general adult population.
METHODS
Study Design and Population
We conducted a cross-sectional PSG study in a population-based cohort study conducted in Ansan, Republic of Korea, a part of the Korean Genome and Epidemiology Study. This prospective cohort study is ongoing and has been carried out since 2001. Detailed information of this cohort study is available elsewhere [20].
Between August 17, 2009 and February 28, 2013, 2065 participants aged 48–79 years from Ansan underwent PSG and were included in the present study. They were assigned either to the exploratory sample or the validation sample by a gender-stratified random sampling procedure based on a half-split-sample validation method. The exploratory sample was used to construct a predictive model and scoring method in order to develop a modified version of the questionnaire. The validation sample was used for evaluating diagnostic accuracy of the standard and modified versions.
The Human Subjects Review Committee at the Korea University Ansan Hospital approved the study protocol as well as the informed consent form, which each participant signed (IRB number: ED0624).
Polysomnography and Outcome
Unattended overnight PSG was carried out either at the participant’s home or at the Korea University Ansan Hospital sleep laboratory using a portable monitoring device (Embletta X-100; Embla Systems, Broomfield, CO, USA). A detailed description of PSG method is available elsewhere [21]. To calculate the apnea-hypopnea index (AHI) (average number of apnea and hypopnea events per sleep hour), apnea was defined as a > 90% reduction in airflow from baseline for at least 10 s and hypopnea was defined as a ≥ 30% reduction in airflow from baseline accompanied by a ≥ 4% decrease in oxygen saturation. In this study, OSA was defined as having AHI ≥ 5 in order to develop a predictive model. Subgroups such as mild to moderate cases (5 < AHI < 30) and severe cases (AHI ≥ 30) were also used to calculate diagnostic accuracy.
Potential Risk Factors
Information about potential risk factors such as age, sex, smoking status, alcohol consumption status, and sleep-related symptoms was collected from the interview-based questionnaire. Anthropometric and clinical data including BMI, neck circumference (NC), waist circumference (WC), and blood pressure were collected through a comprehensive health examination conducted by health professionals according to a standardized protocol. Biomarkers such as high-sensitivity C-reactive protein (hs-CRP), insulin, leptin, and ghrelin concentrations were selected based on previous reports [22,23] and assayed in serum samples collected following minimum 8-hour fast. The presence of hypertension and diabetes mellitus (DM) was determined based on blood pressure, fasting and post-load blood glucose levels, and medication history.
Statistical Analysis
Descriptive statistics of characteristics were calculated for the exploratory sample, the validation sample, and all participants taken together. To compare characteristics between the exploratory sample and the validation sample, the chi-squared test for categorical variables and the Student’s t-test for continuous variables were used.
To analyze associations between OSA and potential risk factors, we conducted multivariate logistic regression analysis and obtained odds ratios (ORs) with 95% confidence interval (CI). First, we conducted multivariate association analysis for the components of the standard STOP-BANG questionnaire including daytime sleepiness, snoring, witnessed breathing cessation, presence of hypertension, BMI >35 kg/m2, age > 50 years, NC > 40 cm, and male sex in all participants as well as in the exploratory sample separately. Next, we constructed three multivariate models. For the first model, we considered all components of the standard STOP-BANG questionnaire, but modified the BMI cutoff point. To determine this cutoff point, we conducted univariate logistic regression analysis for the association between BMI and OSA and selected a BMI having a minimum value of the square root of [(1 - sensitivity)2 + (1 - specificity)2], which indicates the minimum distance from the upper left corner to the point on the receiver-operating characteristic (ROC) curve that is the optimal cutoff point [24]. For the second model, we considered WC, DM, and smoking and alcohol consumption status as well as all variables in the first model. In particular, the WC variable was fitted as a binary variable that was created using the optimal cutoff point, which was determined by univariate logistic regression analysis for the association between WC and OSA. For the third model, we considered biomarkers including hs-CRP, insulin, leptin, and ghrelin in addition to all variables in the second model and then selected significant variables using the forward method in multivariate regression analysis. A final model consisting of all significant variables was established for scoring.
We made a scoring system for each model. In the standard scoring method for the STOP-BANG questionnaire, one or zero is assigned for each risk factor according to its presence or absence and then all point values are summed. In this study, we mainly used this method but further considered a modified scoring method based on the effect size estimates of risk factors. After scoring each model, we evaluated the diagnostic accuracy of the standard version and the modified versions using ≥ 3 as the cutoff score and calculated measures of diagnostic accuracy such as sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the ROC curve (AUC). Furthermore, we tested the difference in the AUC between the standard version model and each of the modified models.
All testing was based on a 2-sided level of significance (p < 0.05) and was conducted using the SAS software (SAS 9.1.3; SAS Institute, Cary, NC, USA).
RESULTS
Table 1 shows characteristics for all participants (n = 2065) in both the exploratory sample (n = 1032) and the validation sample (n = 1033). Among all participants, 0.24% had a BMI > 35 kg/m2, 4.4% had a NC > 40 cm, 32% had daytime sleepiness, 79% snored, 21% had witnessed breathing cessation, and 46% had a diagnosis of OSA based on PSG. In our sample population, a total of 958 participants (487 in an exploratory sample) were determined to have OSA, which is double the number of participants who reported witnessed breathing cessation. The distribution of measured characteristics was similar between the two samples.
Characteristics of 2065 study participants and comparison between two samples
The results regarding the associations between the components of the STOP-BANG questionnaire and OSA among all participants as well as in the exploratory sample are presented in Table 2. Daytime sleepiness and BMI > 35 kg/m2 were not associated with OSA in either group. NC > 40 cm was significantly associated with OSA in the exploratory sample, but not when all participants were evaluated together.
Association between components of the STOP-BANG questionnaire and obstructive sleep apnea based on polysomnography
Table 3 shows multivariate ORs (95% CI) of OSA for the components of the STOP-BANG questionnaire and other potential risk factors in the exploratory sample. In the modified model 1, the BMI cutoff point was modified and all other variables were the same as the STOP-BANG questionnaire. The optimal BMI cutoff point, which was obtained from the univariate regression analysis for the association between BMI and OSA, was found to be 25 kg/m2. Fig. 1 presents a ROC curve of BMI with points for 25 kg/m2 and 35 kg/m2. In the modified model 1, BMI > 25 kg/m2 was significantly associated with OSA, but daytime sleepiness and NC > 40 cm were not associated with OSA. For the modified model 2, WC, DM, smoking status, and alcohol consumption status were also included. The optimal WC cutoff point was determined as 85 cm based on the association between WC and OSA. In this model, daytime sleepiness, NC > 40 cm, and smoking and alcohol consumption status were not associated with OSA. Using the modified model 2, the forward procedure identified the following significant variables which were included in the modified model 3: snoring, witnessed breathing cessation, hypertension, BMI > 25 kg/m2, age > 50 years, male sex, WC > 85 cm, and DM. All of these variables were found to be significantly associated with OSA.
Association between potential risk factors and obstructive sleep apnea based on polysomnography in the exploratory sample
The receiver-operating characteristic curve of BMI in the association with obstructive sleep apnea. Points for 25 kg/m2 and 35 kg/m2 are depicted on the curve. BMI: body mass index.
Next, we scored the modified models using the standard scoring method: one or zero was assigned for each risk factor according to its presence or absence because effect size estimates for all risk factors except age ≤ 50 years were observed to be similar as presented in Table 3. When we further analyzed using the three age groups (≤ 50 years, 51–64 years and ≥ 65 years), in the modified model 3, multivariate ORs (95% CI) were observed to be 2.70 (1.63, 4.48) and 4.50 (2.54, 7.98) for the 51–64 age group and ≥ 65 age group, respectively, compared to the ≤ 50 age group. The scoring method was modified to accommodate the differences found between age groups as follows: zero points for ≤ 50 years, one point for 51–64 years, and two points for ≥ 65 years. One or zero points were assigned for all other risk factors. Total scores were calculated by summing the point values assigned to risk factors for each model. Both the standard and the modified scoring methods were applied to the modified model 3. The distribution of total scores for the standard and the modified models is displayed in Fig. 2. The proportion of individuals with total score ≥ 3 was observed to be greater in the modified models than in the standard version model.
Distribution of the scores of the standard STOP-BANG questionnaire and the modified models in the exploratory sample. The Y axis indicates the accumulative percentage of score points according to the standard version model and the modified models. *In the modified scoring method, zero to two points (zero for ≤ 50 years, one for 51–64 years, and two for ≥ 65 years) are assigned for age groups and one or zero points for all other risk factors.
Table 4 demonstrates the diagnostic accuracy parameters, which were calculated for each model in the validation sample. For detecting mild to moderate OSA, sensitivity and specificity estimates were found to be between 56% and 64%, PPV and NPV estimates between 57% and 73%, and AUC estimates between 0.66 and 0.72. In terms of diagnostic accuracy for detecting severe OSA, sensitivity was found to be between 66% and 79%, specificity between 51% and 55%, PPV between 5% and 6%, NPV between 98% and 99%, and AUC between 0.67 and 0.78. On the basis of these results, the modified model 3 showed the greater estimate of AUC compared to the other models for detecting all cases and severe cases of OSA. Regardless of scoring method, the AUC of the modified model 3 was significantly different from that of the standard version model for detecting mild to moderate cases (p < 0.001) and severe cases (p < 0.01) of OSA. Because of the high prevalence of mild to moderate OSA, the measured diagnostic accuracy for detecting all cases of OSA was found to be similar to the accuracy for detecting mild to moderate OSA. Using the five models presented in the Table 4, ROC curves for all OSA cases are presented in Fig. 3.
Results of diagnostic accuracy for the standard STOP-BANG questionnaire model and modified models using total score ≥ 3 as cutoff in the validation sample
The receiver-operating characteristic curves of the standard and modified model scores for obstructive sleep apnea.
DISCUSSION
In this cross-sectional study, we carried out PSG in the general adult population of 2065 male and female participants and attempted to develop and validate a predictive model to improve diagnostic accuracy for detecting OSA compared to the standard STOP-BANG questionnaire. Our results show that a predictive model which includes risk factors such as daytime sleepiness, snoring, witnessed breathing cessation, hypertension, BMI > 25 kg/m2, middle-aged and older groups (51–64 years and ≥ 65 years), male sex, WC > 85 cm, and DM had significantly greater AUC with higher sensitivity, specificity, PPV, and NPV when compared to the diagnostic accuracy parameters of the standard version model. In particular, because this predictive model was found to have a higher sensitivity (79%) than the standard version model (66%) for detecting severe OSA, it may be a useful tool for screening for high-risk cases of OSA in the general adult population.
A meta-analysis study reported that the prevalence of OSA ranges from 9% to 38% in the general population [1]. Such a large variation in the estimates may be due to diverse characteristics of the study population, such as age range, gender distribution, and the prevalence of obesity, as well as discrepancies in sampling methods. Nonetheless, OSA prevalence is likely increasing because of aging population trends and increases in obesity prevalence.
Data regarding the causal associations of OSA with cardiovascular and metabolic diseases, occupational accidents, and other adverse consequences, such as excessive daytime sleepiness or decreased work performance, are accumulating [3,4,25,26]. Improved screening questionnaires to identify potential cases of OSA are needed to help prevent these adverse outcomes, as most cases remain undiagnosed and patients may be unaware of difficulty breathing and repeated awakenings during sleep [7]. Several studies have highlighted the use of the STOP-BANG questionnaire, which is short and includes simple questions, in patients who visited sleep clinics [10-17] as well as in the general population [18,19]. This questionnaire has been found to show comparable diagnostic accuracy, particularly with regards to sensitivity, for detecting OSA in a sleep clinic population when compared to other longer questionnaires [10-17]. However, there are limited studies evaluating this questionnaire in the general population. Two population-based studies observed nearly identical sensitivity for detecting severe OSA: 70% sensitivity was reported in a study of 4770 American adults [18] and 69% sensitivity in a study of 242 Singaporean adults [19]. However, the sensitivity and specificity results from these two studies for detecting moderate to severe cases were inconsistent [18,19].
We also observed similar sensitivity for detecting severe OSA in the standard version model compared with previous population-based studies [18,19]. When using the modified version (modified model 3 + modified scoring method), however, we observed higher sensitivity than what has been found in previous studies. In particular, we found that the modified version can detect all severe cases of OSA. Explaining in detail, we observed that only 21% of the study population reported witnessed breathing cessation, and thus 46% of severe cases of OSA (or 69% of all cases) were likely to be unaware of apnea. Among severe cases of OSA who reported no witnessed breathing cessation, 85% were detected with the STOP-BANG questionnaire. Meanwhile, all severe cases were detected with the final modified version, regardless of witnessed breathing cessation. The final modified version includes variables such as WC and DM in addition to the components of the standard version, and does not include daytime sleepiness and NC. Both WC and NC have been reported to be associated with OSA, although which anthropometric measure is a better predictor for OSA risk remains controversial [27,28]. In our study, WC was associated with OSA at a high significance level in the multivariate regression analysis, but NC was not (data available on request). Previous reports suggested that OSA is a risk factor for type 2 diabetes, but also vice versa, although more data are needed [29,30]. In our study, a weak association between daytime sleepiness and OSA was observed and this might be partly due to snorers, because daytime sleepiness was strongly associated with snoring (data available on request). Previous studies found an association between daytime sleepiness and snoring independent of OSA [31,32]. With regards to the modification of the cutoff for BMI, we obtained an optimal cutoff point based on the ROC curve of the association between BMI and OSA. In a previous study, either 30 kg/m2 or 27.5 kg/m2 was used, but diagnostic accuracy was not improved by this modification in the general population [19]. A recent study including Korean patients, which used 30 kg/m2 for the BMI cutoff and 36.3 cm for the NC cutoff, observed modest improvement in sensitivity and reduced specificity leading to similar AUC values compared with the standard version [33]. In our findings, modification of the BMI cutoff improved the AUC significantly (in modified model 1) for detecting severe OSA. In addition, we considered other variables such as smoking status, alcohol consumption, and some biomarkers in order to improve diagnostic accuracy, but did not observe significant associations with OSA.
The present study has some strengths: we used a large sample size from the general population, a half-split-sample validation approach, and a wide range of variables. One study limitation was that the final modified version could not improve specificity. The standard STOP-BANG questionnaire itself shows low specificity and PPV because of high false positive results [10-19]. Nonetheless, this questionnaire and the modified version may be useful to primary care physicians for educating patients about the risk factors of OSA, which overlap with those for cardiovascular disease. Another limitation of the study is limited generalizability of our findings. Further investigation is warranted to develop an advanced screening tool to improve specificity and PPV.
In conclusion, our findings suggest that the modified version of the STOP-BANG questionnaire which the modified scoring method is applied to may be useful in detecting severe OSA in general Korean adults.
Acknowledgments
This study was supported by a fund (2009-E71002-00, 2010-E71001-00, 2011-E71004-00, 2012-E71005-00) by research of Korea Centers for Disease Control and Prevention. This study was also supported by National Research Foundation of Korea Grant funded by the Korean Government (NRF-2019R1A2C2084000).
Notes
Conflicts of Interest
The authors have no financial conflicts of interest.
Authors’ Contribution
Conceptualization: Baik I. Data curation: Shin C. Formal analysis: Baik I. Funding acquisition: Shin C, Baik I. Methodology: Baik I. Project administration: Shin C. Resources: Shin C. Software: Baik I. Supervision: Shin C. Validation: Shin C. Visualization: Baik I. Writing—original draft: Baik I. Writing—review & editing: Shin C, Baik I.