Initial validation of the new browser-based application DREEP for diagnosis of common chronic sleep disorders

Potential candidates were informed about the purpose of the study by a study physician using the information for patients approved by the ethics committee. Only patients who had given their written consent to the study were recruited. All patients were examined by a sleep physician on the afternoon of admission or on the following morning. A comprehensive medical history was taken, particularly with regard to sleep problems. The browser-based application DREEP was made available to the participants on a study tablet during their inpatient stay under supervision of a study physician. They answered the questions either before the nocturnal diagnostics using PSG or in the morning after the nocturnal diagnostics. Diagnoses and disease severity determined using DREEP were then compared with corresponding clinical findings based on PSG data. If there was a reasonable suspicion of narcolepsy, a Multiple Sleep Latency Test was carried out in accordance with the respective guidelines.

The study protocol was designed by all authors and approved by the ethics committee of the Medical Faculty of the University of Duisburg-Essen (no: 22-10782-BO). All study procedures were performed in accordance with the ethical standards of the institutional research committee of the Medical University of Essen, Germany, the Good Clinical Practice guidelines as well as the principles of the Declaration of Helsinki.

Between September 2022 and April 2023, a total of 64 participants (42 male, mean age 60 ± 14 years, mean body mass index 32 ± 7 kg/m²; Table 1) with diagnosed or suspected chronic sleep disorders who were referred to the sleep laboratory of the Ruhrlandklinik, Medical University, Essen, Germany, and who experienced non-restorative sleep at least three days a week for more than one month, were invited to participate. All participants were ≥ 18 years of age and had the presence or suspected presence of snoring and/or OSA, RLS/PLMD, or chronic insomnia of different severity. Participants with user problems or noticeable difficulties in the German language were excluded prior to testing, and if language problems became apparent during the use of DREEP, these individuals were also excluded. Moreover, participants with a total sleep time of < 180 min and all those with predominantly central sleep apnea were excluded from further analysis. Another exclusion criterion was lack of cognitive ability to follow the study protocol.

Polysomnography was carried out under inpatient conditions in the research laboratory of the Clinic for Sleep and Telemedicine at the Ruhrlandklinik, University Medicine Essen. PSG was performed according to the recording recommendations of the American Academy of Sleep Medicine (AASM) [11] the night after admission using a digital polygraph (Nox T3, Nox Medical, Reykjavík, Iceland) including two electroencephalograms, two electrooculograms, submental and tibialis electromyograms, ribcage and abdominal inductance pneumograms, and an oronasal thermistor (Nox A1^TN, Nox Medical). The following transducers were used: finger pulse oximeter (SenSmart^TN, Nonin Medical Inc, Plymouth, MN, USA), nasal cannula (flow measurement with a sampling frequency of 20 Hz), and body position sensor (Nox Medical). The PSG was scored the next morning by a certified specialist in sleep medicine according to AASM standard criteria [11, 13]. The PSG was supervised by a specifically trained medical technical sleep assistant. If necessary, the electrodes/sensors were adjusted during the night by this person.

Apneas were scored if there was a 90% reduction in oronasal flow lasting more than 10 s, with preserved movements of the thorax or abdomen. Hypopneas were scored if there was a ≥ 50% to 90% reduction in the flow signal for at least 10 s compared with the stable baseline accompanied by a ≥ 3% reduction in oxygen saturation. For comparability with other studies, events that caused an arousal or a drop in oxygen saturation without qualifying as apnea or hypopnea (so-called respiratory effort-related arousals) were not counted as respiratory events [11, 13]. OSA was diagnosed when the Apnea-Hypopnea-Index (AHI) was ≥ 5/h; mild, moderate, and severe OSA were defined as an AHI of ≥ 5 to < 15/h, ≥ 15 to < 30/h, and ≥ 30/h, respectively.

A laptop or tablet was provided for the use of DREEP. Before the actual test, participants (identified only by their study number) were briefed on how to use the device and software. Based on the user’s responses, the DREEP algorithm identifies the most likely underlying chronic sleep disorder and classifies it as mild, moderate, or severe. The catalogue of disease-specific questions used by DREEP is based on established validated national and international questionnaires for characterizing each chronic sleep disorder and its severity. Since the study was conducted in German-speaking countries, questions from the disease-specific questionnaires were translated from English into German and adapted, shortened, or modified to ensure that all questions had a uniform design.

The diagnosis of OSA was based on the Lausanne NoSAS (neck circumference, obesity, snoring, age, sex) score test [14], which has been simplified for use in DREEP. This questionnaire is currently considered to be the most valid questionnaire for diagnosing sleep-disordered breathing, specifically OSA. This tool works very well not only in individuals with moderate to severe OSA, but also in those with mild OSA. In addition, NoSAS performs significantly better than three other well-established screening tools, namely the STOP-Bang (snoring, tiredness, observed apnea, high blood pressure) questionnaire, the Berlin questionnaire, and the Epworth Sleepiness Scale (ESS) [14]. The International Restless Legs Syndrome Study Group severity rating scale (IRLS) [15] was adjusted for diagnosing RLS and PLMD. Validation studies of the IRLS show that this is reliable, valid, and responsive in clinical trial settings [18, 19]. The Regensburg Insomnia Scale (RIS) [16] was modified to allow diagnosis of insomnia. In the US, the Pittsburgh Sleep Quality Index (PSQI) is the preferred method for diagnosing and assessing the severity of chronic insomnia. However, this was not used in the development of DREEP because the PSQI was developed in a population of older Black and White women and in conjunction with the ESS [17]. In contrast, the RIS was validated in a cohort of middle-aged women and men (average age 48 years) and discriminates well between individuals with insomnia and healthy controls [16]. In addition to several questions on demographics, lifestyle, sleep environment, and sleep behavior, DREEP contains a minimum of five items from each of these three sleep-specific rating scales.

Sensitivity, specificity, accuracy, positive predictive, and negative predictive values were calculated for the detection of OSA, PLMD/RLS, and insomnia, and according to disease severity (mild, moderate, severe) for OSA and PLMD/RLS. The performance of DREEP was determined using the area under the receiver operating characteristic curve (AUC) using SPSS 27.0 (IBM Corp., Armonk, NY, USA).

Of the 64 participants with diagnosed or suspected sleep disorders referred to our sleep laboratory, the PSG diagnosis was OSA without comorbidities in 34 participants, PLMD/RLS comorbid to mild or treated OSA in 57 (including 2 with mandibular advancement splints, 46 treated with positive airway pressure therapy, and 9 newly diagnosed), PLMD/RLS without OSA in 26 participants, chronic insomnia in 10 (7 participants with comorbid OSA), and 1 participant had newly diagnosed narcolepsy (Table 1).

On average, completing the questions and scales in DREEP took about 12 min (range: 9–18 min). No participant whose data were analyzed in this study reported problems in understanding the DREEP questions or using the sliders of the digital scales.

The development and validation of novel technologies for easy and reliable diagnostics or sleep monitoring without impeding patient comfort have become a huge research priority in medicine and the use of home-based sleep testing is increasing. A study by Tiron et al. [21] examined the Firefly app-based sensing technology, which also offers the potential to significantly lower the barrier of entry to OSA screening, as no hardware (other than the user’s personal smartphone) is required. The subtle breathing patterns of a person in bed are measured via a smartphone using the Firefly app technology platform and artificial intelligence (AI) algorithms to identify detailed sleep stages, respiration rate, snoring, and OSA patterns. This technology showed a performance comparable to outpatient OSA screening devices and other smartphone screening apps, with a sensitivity of 88.3% and specificity of 80.0%, with a receiver operating characteristic AUC of 0.92, for a clinical threshold for the AHI of ≥ 15 events/h of detected sleep time [21]. This is comparable to the results shown in our study, although it has to be considered that the applications are based on completely different processes.

To our knowledge, this is the first validation attempt of a web-based application designed to screen for various common chronic sleep disorders at the subjective level. Like other subjective sleep assessment methods, the DREEP app is not intended to make a definitive diagnosis, but rather to assist specialists in their clinical evaluation by providing a high pre-test probability for a specific sleep problem. While there are many validated one-dimensional questionnaires available to sleep medicine to check for the presence of one particular condition, DREEP offers a more holistic approach. Its multidimensional structure is based on a collection of several adapted, shortened, or modified test items derived from four well-established rating scales routinely used for the assessment of sleep-related (movement) disorders. Unlike this clear diagnostic purpose of the DREEP app, the focus of the vast majority of sleep apps available in commercial app stores today is on sleep tracking. Primarily, such tracking apps are supposed to fulfil the purpose of prevention and raise awareness of one’s own sleep, which can possibly entail behavioral changes to achieve improved sleep hygiene. At the same time, the use of such apps may also have a detrimental effect on sleep quality as part of an exaggerated self-optimization. Given the high performance and reliability of some of these apps, it is of course to be expected that this kind of long-term sleep monitoring will occasionally also recognize indications for the presence of a sleep disorder requiring treatment. However, since most of these apps have not yet been validated, their use as a clinical screening tool is still viewed critically [20]. In contrast, other sleep apps have already proven their valuable contribution to sleep health care, but as they were predominantly developed for therapeutic purposes, e.g., as a digital unguided self-help alternative to cognitive behavioral therapy for the treatment of insomnia [21], they can be ignored here.

Another computerized tool for the assessment of sleep disorders is the expert system Sleep-EVAL, which was introduced in 1990 and validated against polysomnography in 1999 [27]. It is a non-monotonic, level‑2 knowledge-based system endowed with a causal reasoning mode designed to provide homogeneous and standardized diagnostic evaluations. Similar to DREEP, it is designed like traditional questionnaires but much more complex. The initial responses are used to form diagnostic hypotheses. Decisional trees are then created for each hypothesis by formulating additional questions that need to be asked to complete the information on that specific diagnostic pathway. Since this is repeated for all hypotheses, the tool allows for differential diagnosis [27]. It also differs from DREEP in the way it is presented to the patient. While DREEP is a browser-based application that can be used with or without the supervision of a sleep specialist, Sleep-EVAL is a structured diagnostic interview that requires some training to conduct. On average, such an interview lasts 40.4 ± 20.0 min [28], which is much longer than the time it takes to complete the browser-based questionnaire DREEP.

For the following discussion, the present preliminary DREEP results are therefore compared with diagnostic testing accuracy measures of other multidimensional subjective sleep assessment methods that, like DREEP, can be completed without any assistance, but are, as far as we know, at best available as a computerized answer sheet lacking all advantages of a mobile app. Table 5 shows some psychometric properties of four comprehensive rating scales for global sleep assessment with respect to the sleep disorder entities insomnia, OSA, and PLMD/RLS. Their discriminative potential is quantified by sensitivity and specificity indices and, where indicated, by AUC.

First, we found that the DREEP algorithm correctly classified a large percentage of individuals who did indeed suffer primarily from problems falling asleep and/or staying asleep as insomnia patients. Furthermore, the high specificity value suggests that most otherwise sleep-disordered individuals were correctly identified as non-insomniacs by the DREEP algorithm. The global measure of diagnostic accuracy, or AUC, was 0.83, which can be considered very good. This means that the DREEP model can distinguish between positive and negative cases of chronic insomnia with a probability of 83%. This finding on the discriminative ability of DREEP related to insomnia is even superior to those of other multidimensional sleep scales. Specifically, the AUC values of the Global Sleep Assessment Questionnaire (GSAQ) and the Holland Sleep Disorders Questionnaire (HSDQ) were only 0.72 and 0.69, respectively (Table 5), indicating that the DREEP algorithm is a promising screening alternative for insomnia. The discrepancy in discriminatory performance between DREEP and the HSDQ may be partly explained by the extent to which other sleep disorders co-occurred in the subgroup of insomnia patients. Kerkhof et al. [26], who developed the HSDQ, reported a large percentage of patients from other diagnostic groups who were comorbid with insomnia, causing considerable symptom overlap, with the result that many insomnia patients also showed a positive test result on other subscales. In the present initial validation study of the DREEP app, however, the comorbidities of the insomnia patients were limited only to sleep-related breathing disorders (Table 1), which should have contributed to an overall lower symptom overlap and thus have had less influence on test performance.

Second, the DREEP app distinguishes between individuals with and without OSA with a probability of 85%, which is also considered very good. The corresponding AUCs obtained for the GSAQ and the HSDQ were slightly higher but still in the same diagnostic accuracy category, indicating that the discriminative performances were quite similar between these approaches (Table 5). Sensitivity indices of the DREEP app determined for the total OSA group were also comparable with those of the other four self-assessment instruments (Table 5). A comparison of diagnostic accuracy by OSA severity, on the other hand, was not possible here because this distinction was not made in any analysis of the other four questionnaires. However, this aspect is very important in clinical sleep medicine, because incorrect severity classification, i.e., persons with severe OSA erroneously classified as mild or vice versa, can be associated with unnecessary effort and costs, e.g., due to performing an additional PSG, which should be strictly avoided in view of the current resource scarcity in sleep medicine. Overall, the present preliminary validation data show that DREEP is suitable to detect individuals with OSA in the clinical setting.

Third, the DREEP algorithm discriminates between positive and negative cases of PLMD/RLS with a probability of 80%, which is again slightly lower than the AUC values of the GSAQ and the HSDQ (Table 5), but still very good. In addition, the DREEP app correctly identified around 79% of all individuals with clinically confirmed PLMD/RLS. The specificity value of 0.80 indicates that DREEP correctly classified otherwise sleep-disordered individuals as “negative” for PLMD/RLS to a sufficient extent for clinical purposes. These two indices were highest for patients with moderate and lowest for patients with mild PLMD/RLS, suggesting that DREEP underestimates the presence of mild cases. Compared to the other four questionnaires, the test attributes of the DREEP app are mid-range and most similar to those of the HDSQ and the SLEEP-50. Both indices are much better than those found in the Sleep Disorders Questionnaire, and the specificity is also higher than that in the GSAQ. In terms of sensitivity, however, DREEP performs worse than the GSAQ (Table 5).

Finally, a previously unknown case of narcolepsy was correctly diagnosed by DREEP during the study. However, with a total of one affected patient in the present study, it was impossible to come to any conclusions about the diagnostic abilities of DREEP with regard to the prediction of narcolepsy.

Nevertheless, based on the test performances for the other sleep disorders, we assume that the presented web application is sufficient to make an initial assessment of whether insomnia, sleep-disordered breathing, or a sleep-related movement disorder is present or not.

DREEP provides several benefits for future application because it is easy to use, inexpensive, does not require a smartphone or similar, and works paperless and is therefore sustainable. In addition, it is faster than separately completing the various questionnaires used in DREEP, or other computerized tools such as the Sleep-EVAL system [28].

Besides these strengths, we have to face some limitations. Children and adolescents were excluded from the study, and therefore the conclusions of the present initial validation study apply to adults only. Thus, DREEP is not currently suitable for use in children and adolescents with disturbed sleep. In addition, only subjects who experienced non-restorative sleep on at least three days a week for more than one month were recruited. Therefore, the use of DREEP is limited to individuals with common chronic sleep disorders and is not appropriate for use in those with acute or situational sleep disorders. Although DREEP can detect several rare chronic sleep disorders, the current study provides no validation data for this. One individual with previously undetected narcolepsy was correctly identified by the algorithm, but this observation does not allow any statements to be made about the diagnostic ability of DREEP in adults with narcolepsy. Further, although the diagnosis suggested by DREEP was often correct in relation to common chronic sleep disorders, the application provides this diagnosis only with some degree of certainty and not with a final clinical diagnosis. Therefore, it is not suitable for providing an unequivocal diagnosis or for reliably excluding the presence of a specific sleep disorder as the cause of chronically disturbed sleep. Finally, the browser-based application DREEP needs to be further investigated. In a follow-up study, a healthy control group should also be analyzed to be able to improve the clinical accuracy of DREEP. Ideally, this investigation should then not only take place in a single tertiary care center, but in a multicenter study.