Accuracy of an Artificial Intelligence Model to Predict Dementia Development with Additional Dental Checkup Data: A Retrospective Cohort Study

1. Introduction

Dementia is a condition characterized by irreversible decline in intellectual and cognitive abilities resulting from acquired organic brain damage [1]. In the early stages, it is often confused with age-related forgetfulness, but memory impairment, disorientation, and cognitive dysfunction become more pronounced as symptoms progress [2,3]. In Japan, approximately 20% of older adults (age ≥ 75 years) had dementia in 2020, with the proportion increasing annually, making it a major aging-associated physical and mental disorder [4]. Dementia progresses irreversibly [5], and a fundamental cure has yet to be developed [6]. Therefore, countermeasures should focus on predicting when dementia will develop and rapidly seeking preventive strategies rather than interventions after development.

In recent years, various artificial intelligence (AI) models have been developed with the aim of predicting disease development [7,8]. In the field of dementia, techniques that analyze patients’ speech patterns to diagnose early symptoms of dementia have been reported [9,10], as have practical applications of AI models that diagnose dementia from imaging modalities such as magnetic resonance imaging, computed tomography, and positron emission tomography [11]. However, these models focus on how early a diagnosis can be made for patients who have already developed dementia; how useful AI models can be for preventive measures before dementia develops remains unclear.

Previous reports have indicated that dementia is associated with various diseases such as hypertension, diabetes, and hyperlipidemia [12,13,14], as well as oral health conditions such as tooth loss and periodontal disease [15,16,17]. Therefore, an individual’s medical and oral health conditions may be useful for predicting future dementia development.

Against this background, Sony Network Communications (Tokyo, Japan) released Prediction One®, a software program that automatically creates AI models using existing data [18]. In Japan, the National Health Insurance Database (NDB) provides information on people with dementia or other medical conditions. In addition, regional dental checkups (GIFU SAWAYAKA Oral Checkups) are conducted once a year for older adults in Gifu Prefecture, with the results also stored in a database. Because existing data such as those from the NDB and dental checkups can be utilized, an AI model using Prediction One® may be able to predict dementia development. Therefore, in the present study, we developed and evaluated the accuracy of an AI model using Prediction One® to predict dementia development after 2 years based on NDB or dental checkup data in Japanese older adults who had undergone regional dental checkups in Gifu.

2. Materials and Methods

This retrospective cohort study targeted older adults residing in Gifu Prefecture. Using NDB and dental checkup data acquired in fiscal year (FY) 2018 (April 2018 to March 2019) as the baseline, we tracked the occurrence of dementia through FY 2020 (April 2020 to March 2021) to construct and validate an AI prediction model.

2.1. Participants

A total of 7,834 Japanese older participants met the following inclusion criteria: 1) older adult (aged ≥ 75 years) who participated in the GIFU SAWAYAKA Oral Checkups program in FY 2018, 2) individuals for whom NDB data from FY 2018 were obtainable, and 3) individuals for whom dementia onset could be tracked in the NDB in FY 2020. Among these individuals, 450 already diagnosed with dementia by FY 2018 were excluded. As a result, 7,384 participants (3,078 male, 4,306 female; average age, 80.0 years) were included in the analysis. This study employed a census survey method, registering all eligible individuals in the target area, and no prior sample size calculation was performed. In addition, follow-up was possible for all subjects using the 2020 NDB data; therefore, no subjects were lost to follow-up.

2.2. Development of the AI Model

The primary outcome measure in the present study was whether participants received a new diagnosis of dementia between FY 2018 and FY 2020. Dementia diagnoses were extracted from medical facility visit data recorded in the NDB and defined based on the presence or absence of corresponding codes for dementia from the tenth revision of the International Classification of Diseases. Next, NDB and dental checkup data were integrated and individual data were randomly assigned to a training dataset (n = 5,169) and a validation dataset (n = 2,215) using the systematic sampling method. An AI model was created from the training group using Prediction One® (Sony) based on 2018 NDB data and 2018 dental checkup data, in addition to data from participants newly diagnosed with dementia before 2020.

2.3. Survey Items in the NDB

We obtained information on gender, age, hypertension, diabetes, dyslipidemia, musculoskeletal disease, pneumonia, chronic obstructive pulmonary disease (COPD), support/care-need certification, and medication history from the NDB [19,20].

2.4. Assessment of Body Composition

A nurse measured the participants’ height and weight. Body mass index (BMI) was calculated by dividing weight in kilograms by height in meters squared (kg/m²) [21].

2.5. Assessment of Serum Hemoglobin A1c (HbA1c) Levels

Serum HbA1c in venous blood collected after an overnight fast was measured using high-performance liquid chromatography [22].

2.6. Smoking Habit, Exercise Habit, Weight Loss, Going Out, and Oral Items

Data on smoking habits, exercise habits, weight loss, going out habits, and oral items were obtained as follows: presence or absence of regular dental checkups, brushing frequency ≥ twice/day, presence or absence of using of interdental brushes/dental floss, presence or absence of difficulty in biting hard food, presence or absence of choking on tea and water, presence or absence of dry mouth, presence or absence of ≥ 20 teeth, presence or absence of decayed teeth, presence or absence of periodontal pocket depth ≥ 4 mm, presence or absence of ill-fitting dentures, presence or absence of tongue coating, presence or absence of halitosis, poor or good tongue and lip function, poor or good swallowing function, and poor or good oral hygiene. The oral items were provided by Gifu Dental Association, Japan.

Presence of smoking habits was defined as smoking at least one cigarette per day [23]. Presence of exercise habits was defined as engaging in light exercise (≥ 30 minutes) more than twice/week for ≥ 1 year [24]. Presence of weight loss was defined as having lost ≥ 2–3 kg/6 months [25]. Presence of going out was defined as going out at least once a week [24,25]. Presence of regular dental checkups was defined as having visited a dental clinic at least once/6 months [26]. The participants were asked to select “yes” or “no” for the following items on a self-administered questionnaire: “Difficulty in biting hard food”, “Choking on tea and water”, and “Dry mouth” [27]. Number of present teeth, decayed teeth, periodontal pocket depth, ill-fitting dentures, tongue coating, halitosis, tongue and lip function, swallowing function, and oral hygiene were documented by dentists. For the assessment of a periodontal pocket ≥ 4 mm, the coded values of the Community Periodontal Index were used, with periodontal pocket codes 1 and 2 evaluated as a periodontal pocket depth ≥ 4 mm [28]. Participants were classified in regard to fitting dentures (good, poor, fracture), tongue coating (minimal, moderate, heavy), and halitosis (minimal, moderate, heavy). Participants indicating poor or fracture were classified as having ill-fitting dentures, those indicating moderate or heavy were classified as presence of tongue coating, and those indicating moderate or heavy were classified as presence of halitosis. Tongue and lip function were assessed using the oral diadochokinesis test, with poor tongue and lip function defined as producing fewer than 30 syllables (“pa,” “ta,” or “ka”) within 5 seconds [29]. Swallowing function was assessed using the repeated saliva swallowing test, with poor swallowing function defined as swallowing fewer than three times within 30 seconds [30]. Participants were also classified in regard to dental plaque (minimal, moderate, heavy). Those indicating moderate or heavy were classified as having poor oral hygiene [31].

2.7. Prediction Accuracy of the AI Model and Magnitude of Each Factor’s Contribution to the Prediction of Dementia Development

When creating an AI model to predict dementia development, Prediction One® automatically calculates the prediction accuracy and magnitude of each factor’s contribution to the prediction of dementia development. Prediction accuracy was determined by creating an AI model twice using the training dataset and measuring the degree of agreement between the two AI models [18]. The magnitude of each factor’s contribution to the prediction of dementia development was evaluated by how much the accuracy of the AI model decreased when that factor was removed [18,32]. Thus, the greater the decrease in accuracy, the greater that factor’s contribution to dementia development.

2.8. Evaluation of AI Model Accuracy Using Validation Data

We used the AI model to predict dementia development after 2 years based on verification data from 2018 and evaluated its relationship with actual measurements. Specifically, we calculated the sensitivity (number of individuals predicted to develop dementia by the AI model / number of individuals who actually developed dementia) and specificity (number of individuals predicted not to develop dementia by the AI model / number of individuals who actually did not develop dementia) regarding dementia development after 2 years.

2.9. Statistical Analysis

Data normality was assessed using the Lilliefors test. Because all continuous variables were not normally distributed, the data were expressed as the median (first and third quartiles). We divided the participants into two data groups: training and validation. For the between-group comparisons of each factor, we used the chi-square test and the Mann–Whitney U test. All data were analyzed using SPSS Statistics (Version 27; IBM Japan, Tokyo, Japan). P-values < 0.05 were considered to indicate statistical significance.

2.10. Research Ethics

This study was approved by the Ethics Committee of Asahi University (No. 33006) and conducted in accordance with the Declaration of Helsinki (as revised in Brazil 2013). As the data from the NDB and dental checkups were anonymous, informed consent was not considered necessary.

3. Results

Table 1 shows the characteristics of the participants in the training and validation data groups. No significant differences in any of the factors set in our study were observed between the two groups.

Table 2 and Table 3 show the relationships between the prediction and actual occurrence of dementia development in the AI model. In total, 141 participants (6%) were newly diagnosed with dementia after 2 years.

In Table 2, we used an AI model trained solely on data from the NDB. This AI model predicted that 73.0% of the participants with the presence of actual occurrence of dementia development after 2 years would develop dementia (sensitivity, 0.730; 103/141). Furthermore, it predicted that 91.4% of the participants with the absence of actual occurrence of dementia development after 2 years would not develop dementia (specificity, 0.914; 1896/2074). In addition, Prediction One® calculated the prediction accuracy of the AI model to be 77.3%.

In Table 3, we used an AI model trained using NDB and dental checkup data. This AI model predicted that 74.5% of the participants with the presence of actual occurrence of dementia development after 2 years would develop dementia (sensitivity, 0.745; 105/141). Furthermore, it predicted that 94.9% of the participants with the absence of actual occurrence of dementia development after 2 years would not develop dementia (specificity, 0.949; 1968/2074). In addition, Prediction One® calculated the prediction accuracy of this AI model to be 79.0%.

Abbreviations: AI, artificial intelligence; NDB, the National Database of Health Insurance of Japan. * p < 0.05, using Fishers exact test. Sensitivity, 0.730; Specificity, 0.914. Prediction accuracy level, 77.3% (Calculated based on Prediction OneⓇ).

Table 4 and Table 5 show the top 10 factors among those learned that contributed to the AI model’s prediction of dementia development after 2 years. In the AI model trained solely on NDB data, the factors contributing to the prediction of dementia development after 2 years, listed in order of decreasing contribution, were as follows: support/care-need certification (support-need certification ≥ 2), HbA1c level (%) (≥ 8.0), musculoskeletal disease (presence), pneumonia (presence), hypertension (presence), BMI (kg/m²) (≤ 20.0), age (years) (≥ 86), COPD (presence), gender (female), and diabetes (presence) (Table 4). In the AI model trained using NDB and dental checkup data, the factors contributing to the prediction of dementia development after 2 years, listed in order of decreasing contribution, were as follows: support/care-need certification (support-need certification ≥ 2), pneumonia (presence), use of interdental brushes/dental floss (absence), HbA1c level (%) (≥ 8.0), regular dental checkups (absence), swallowing function (poor), choking on tea and water (presence), musculoskeletal diseases (presence), gender (female), and age (years) (≥ 86) (Table 5).

4. Discussion

To the best of our knowledge, this is the first study to evaluate the accuracy of an AI model to predict dementia development using dental checkup data in Japanese older adults aged ≥ 75 years. The results indicated that the prediction accuracy automatically calculated by Prediction One® showed a value of 77.3% when trained solely using NDB data. Furthermore, when trained using NDB and dental checkup data, it showed a value of 79.0%, indicating improved prediction accuracy. Sony Network Communications, the distributor of Prediction One®, provides the following prediction accuracy level benchmarks for AI model: a = accuracy level benchmarks, “a < 63%: Low accuracy,” “63% ≤ a < 74%: Standard accuracy,” and “74% ≤ a: High accuracy” [33]. This indicates that the AI model developed in the present study has high prediction accuracy. Furthermore, combining NDB and dental checkup data for training suggests that even more accurate AI models can be created.

In this study, 70% of the total data was allocated as training data. In a previous study that created an AI model to predict subarachnoid hemorrhage outcomes using Prediction One®, 67% of the total data was allocated as training data [18]. Furthermore, a study that created an AI model to predict blood urea nitrogen levels after dialysis using Shapley Additive Explanations utilized 70% of the total data for training data [34]. Thus, the allocation of training and validation data in our study was performed in accordance with previous studies.

In the present study, the AI model trained solely on NDB predicted participants with actual dementia development after 2 years with a sensitivity of 73.0%, and participants without actual dementia development after 2 years with a specificity of 91.4%. Furthermore, the AI model trained using NDB and dental checkup data predicted participants with actual dementia development after 2 years with a sensitivity of 74.5%, and participants without actual dementia development after 2 years with a specificity of 94.9%. In other words, when creating an AI model, incorporating dental information in addition to NDB data increased both the sensitivity and specificity of the model. These findings suggest that incorporating dental information in AI models is effective for predicting dementia development more accurately. On the other hand, while incorporating dental checkup data into the NDB training dataset improved the accuracy of the AI model, the sensitivity remained < 80%. This suggests that while our AI model is suitable for exclusionary diagnosis, its utility for definitive diagnosis is still limited, and further refinement is needed. Prediction One® can create more accurate AI models by training based on a larger number of participants and disease-related factors [34,35]. It has been reported that dementia development is associated with factors beyond those discussed in the present study, including hyperthyroidism [36], mental disorders [36], alcohol dependence status [37], and socioeconomic indicators such as income and education levels [38]. Moving forward, we aim to create a more accurate AI model by incorporating these factors into a training dataset.

In the AI model developed in the present study, when trained solely on NDB data as the factor contributing to predicting dementia development, the order of contribution magnitude was as follows: support/care-need certification (support-need certification ≥ 2), HbA1c level (%) (≥ 8.0), musculoskeletal disease (presence), pneumonia (presence), hypertension (presence), BMI (kg/m²) (≤ 20.0), age (years) (≥ 86), COPD (presence), gender (female), and diabetes (presence). Furthermore, when trained on both NDB and dental checkup data, dental factors such as the use of an interdental brush/dental floss (absence), regular dental checkups (absence), swallowing function (poor), and choking on tea and water (presence) were among the top 10 contributing factors. Previous studies have reported that higher rates of dental floss use and dental visits are associated with a lower risk of developing Alzheimer’s disease [39]. The risk of dementia development has also been associated with poor swallowing function [40] and choking [41]. However, because the present analysis is based on a prediction model rather than a causal inference framework, these factors should be interpreted as important predictors or correlates of dementia development rather than as definitive causal determinants.

In Japan, as of 2022, the proportion of older adults with dementia is 23.6% among those aged 75–79 years, 39.2% among those aged 80–85 years, and 60.3% among those aged 86–90 years [42]. Furthermore, according to the Annual Report on the Ageing Society in 2024, the number (proportion) of older adults with dementia in Japan is expected to increase to approximately 10.35 million (28.3%) in 2025, 11.16 million (30.2%) in 2030, and 11.73 million (31.1%) in 2035 [43]. This indicates that the incidence of dementia is expected to continue rising across all age groups of older adults aged ≥ 75 years. Conducting “future dementia development predictions” for participants who undergo regular dental checkups and raising awareness could be effective in helping to prevent the increase in dementia development among older adults. Furthermore, based on the results of the present study, using an AI model as a tool for predicting future dementia development is considered effective.

This study has several limitations. First, because this study used NDB data, the presence of diseases not included in the database is unknown. Second, while Prediction One® calculates the magnitude of each factor’s contribution, it does not establish a benchmark for this magnitude. Therefore, the validity of the magnitude of the contribution of each factor needs to be verified in the future. Third, our study was conducted in a single prefecture and included only individuals who participated in a regional dental checkup program, which may introduce selection bias and limit the generalizability of our findings to other regions or countries. Fourth, we did not perform external validation using an independent cohort, so additional studies are needed to confirm the robustness and transportability of our AI model. Finally, as we did not report confidence intervals for sensitivity, specificity, or accuracy measures, future work should quantify the statistical uncertainty around these estimates. On the other hand, a major strength of the present study is the sample size of over 7,300 Japanese older adults aged ≥ 75 years. This sample size is sufficient to demonstrate the creation of an AI model for predicting future dementia development and to evaluate its accuracy, and may also help in creating an accurate AI model for predicting dementia development by inferring associated factors.

5. Conclusions

The results of the present study indicated that our AI model created using a combination of NDB and dental checkup data modestly improved the accuracy of dementia development prediction compared with models trained on NDB data alone. Several dental-related variables were identified as important predictors statistically associated with future dementia development in our AI model, suggesting that obtaining dental information through medical–dental collaboration may be useful for creating AI models for the prediction of disease development. These findings support the potential role of integrating routinely collected medical and dental data into AI-based risk stratification tools for older adults. However, further refinement, external validation, and causal investigations are needed before their clinical implementation.