Developing a Health System Literacy Measure for Chinese Immigrants in Canada: Adapting the HLS19-NAV Scale

1. Introduction

The “silos phenomenon” in healthcare delivery in the Organization for Economic Co-operation and Development (OECD) countries refers to a system of separate care settings governed at different government levels and operated under different budgetary regimes [1]. Patients often receive care from different providers across uncoordinated departments such as hospitals, primary care clinics, pharmacies, homecare, and insurance companies [2]. This phenomenon can leave many patients and their caregivers feeling overwhelmed when they must navigate the healthcare system to access care [1,2].

Given the complexity and fragmentation of such health systems, patients and their caregivers must take proactive roles to navigate the health system [3]. Some navigational tasks can include understanding how the system works, knowing the scopes of different healthcare professionals, being aware of available services and supports, and applying this knowledge to find services, make treatment decisions, or advocate for themselves when their needs have not been met [3,4]. These tasks can be challenging for many of them, particular those who are unfamiliar with the healthcare system, those with multiple disease conditions, or those from low socio-economic backgrounds with limited health literacy [3,4,5].

Health literacy, defined as “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decision” [6], plays a crucial role in navigating the healthcare system. Patients must apply navigational skills to secure health coverage, access healthcare services, communicate effectively with healthcare professionals, and stand up for themselves [7]. Research found that patients with low health literacy often struggle with navigating health insurance processes and choosing appropriate health plans [6]. Several studies also showed that individuals with low health literacy face difficulties in communicating with healthcare professionals, which can result in misunderstanding or insufficient health information exchange [6].

Most empirical studies about general health literacy have used common instruments for measuring health literacy such as the full-length Test of Functional Health Literacy in Adults (TOFHLA) [8,9], the Newest Vital Sign (NVS) [10], the Rapid Estimate of Adult Literacy in Medicine (REALM) [11], and the Health Literacy Questionnaire (HLQ) [12,13]. Each tool was designed for their specific measurement purpose. While the first three instruments primarily assess reading, pronunciation, and numeracy skills [8,9,10,11,12,13], the HLQ covers a broader construct of functional health literacy. However, it has been noted that the navigation subscale within the HLQ does not comprehensively encompass the concept of health literacy [4,14].

The HLS₁₉-NAV instrument, originally developed by European countries, was designed to better measure navigational health literacy. This instrument has been validated in many countries such as Austria [14], Belgium [14], Czech Republic [14], France [14,15], Germany [14], Portugal [14], Slovenia [14], Switzerland [14], and Russia [14,16]. In Canada, navigating the complex health system poses challenges for many patients, particularly older people [17], those with multiple comorbidities [17], and immigrants [18]. However, little is known about the extent to which Canadian patients, or their caregivers, understand and effectively use the health system because of a lack of the specific instrument for this purpose.

To bridge this gap, our study aims to develop a health system literacy scale (HSL-CAN) adapted from HLS₁₉-NAV to measure how well patients can understand and navigate the Canadian healthcare system. We also assessed the psychometric properties of the HSL-CAN using data collected through an online survey in a Chinese population in Canada. Chinese Canadians were selected as our study population because the Chinese population accounts for 4.7% of the total Canadian population and represents a broad range of generational statuses from first-generation to third-generation or beyond [19].

2. Methodology

2.1. Data Source

This study followed the COSMIN Reporting Guideline for studies on measurement properties to ensure that all measurement properties of our scale, as well as risks of bias, are reported transparently [20]. (see Supplementary 1) The data for analysis came from an online cross-sectional survey conducted between March 11 and^-July 19, 2024, among adult Chinese individuals aged 30 or older who have lived in Canada for at least 6 months. The survey aimed to explore barriers to medical care utilization among Chinese immigrants in Canada during the Covid-19 pandemic. It included questions on socio-demographics, medical history, medication use, health literacy, and cancer screening.

The online survey was developed and delivered through the Qualtrics platform provided by Memorial University of Newfoundland, and was promoted through various social media platforms, including WeChat, websites of collaborative Chinese community organizations, Facebook groups, and other approaches (e.g., emails). Participants were required to give online informed consent before beginning and completing the survey. Although Qualtrics recorded all responses, including incomplete or unsubmitted ones, only those who selected “agree to submit” were included in the analysis. Participation was voluntary and anonymous. Identifying variables such as IP address, WeChat ID, or email were not included in data analysis.

Of the 797 respondents, a total of 681 eligible individuals were included for data analysis. We excluded 111 respondents because these participants either declined to submit or did not indicate whether they agreed to submit the survey, although the Qualtrics recorded these latter cases. According to COSMIN criteria, a sample with a size of 7 times the number of items in the investigating model and at least 100 responses are considered very good for structural validity using factor analysis [21]. Therefore, our sample size of 681 is suitable for analysis.

2.2. Measures of Health Literacy

The HLS₁₉ – NAV scale was used in the survey. The original 12-item scale was developed as part of the European Health Literacy Population Survey 2019-2021 (HLS₁₉) [4,14]. Item development was conducted through definitions, conceptual frameworks, and existing tools associated with navigating the healthcare system [4]. The first version was developed in German and translated into English [4]. 12 items assess patients’ abilities in accessing information (3 items), understanding information (3 items), judging information (3 items), using the information to communicate with healthcare providers and navigate the healthcare system to access care they need [4] (See Supplementary 2). Each item of the HLS₁₉ – NAV has 4-response categories: “4-Very easy”, “3-Easy”, “2-Difficult”, and “1-Very Difficult”. The HSL-CAN had five response categories: “1-Extremely difficult”, “2-Somewhat difficult”, “3-Neither easy nor difficult”, “4-Somewhat easy”, and “5-Extremely easy”, which can potentially improve its psychometric characteristics [22] (see Supplementary 3).

2.2.1. Item Development

Using Griese et al. (2020)’s approach [4], the HSL-CAN scale was developed through literature review on patients’ and caregivers’ experiences with navigating the health system and constructs from existing tools regarding healthcare system navigation. We also consulted a Patient Advisory Committee with 06 members, including three cancer patients, a physician, two community service providers, and a researcher. The scale was translated from English into Chinese Simplify and Chinese Tradition version and translated back into English. Any discrepancies were resolved through discussion between YC and ATV.

2.2.2. Content Validity

The pre-survey was sent to 12 colleagues who had worked in academia and the Canadian healthcare system. They were asked to provide their feedback on the understandability and content of the survey. YC and ATV revised the items to ensure all feedback was appropriately addressed. In addition, we also conducted a post-survey validation with our Patient Advisory Committee with five members participated (one physician, one researcher, one community service provider, and two patient partners). This post-survey validity aims to confirm the suitability of the final instrument after removing some deemed inappropriate items based on statistical analyses.

2.2.3. Statistical Analyses

2.3. Structural Validity Test

2.3.1. Exploratory Factor Analysis (EFA)

The EFA was utilized to examine the internal consistency of items and to identify the numbers of factors and their associated items [23]. The EFA explores the underlying structure of the scale when being adapted to Chinese population in the Canadian health system [24]. Starting with the EFA can assist our understanding of possible cultural difference in the adaption of the scale [25]. This technique determines how strongly an item loads onto an underlying factor indicated by factor loading values, where a loading value of > 0.3 or > 0.4 indicates moderate [26] or good correlation [27], respectively. This study used the cut-off factor loading value of ≥ 0.4.

Before extracting factors, the suitability of data for factor analysis was examined using Kaiser-Meyer-Olkin (KMO) to assess sampling adequacy, and Bartlett’s Test of Sphericity to examine whether correlations among any different items are statistically significant [23]. The KMO test values are between 0 and 1, with values over 0.7 indicating adequate sampling for factor analysis [20]. A significant p-value < 0.05 from Bartlett’s Test of Sphericity indicates the data are suitable for factor analysis [23].

Weighted least square based on a poly-choric correlation matrix for factor extraction was applied given an ordinal, rather than normal, distribution of the data [28,29]. We also used parallel analysis to determine the number of factors to retain by comparing eigenvalues with random order eigenvalues [28,30]. Factors with actual eigenvalues > 1 and surpassing the random order eigenvalues were retained [28,30]. An orthogonal varimax rotation was used to maximize high item loadings and minimize low item loadings, producing a more interpretable and simplified solution if more than one factor was retained from parallel analysis [30]. Moreover, communality values, referring to the percentage of an item’s variance explained by the factor, were reported [28]. Multicollinearity was examined using Variance Inflation Factor (VIF), where a value above 5 or 10 [31], or correlations between items > 0.9 [28] indicates multicollinearity.

2.3.2. Confirmatory Factor Analysis (CFA)

The CFA was used to test the model in EFA with items retained and to evaluate a goodness-of-fit of the model. The restricted CFA was fitted with no cross-loadings or correlated residuals, and parameters were estimated using diagonally weighted least square (WLSMV) [32,33]. This study reported goodness-of-fit statistics, including Tucker-Lewis Index (TLI), Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMR). Values of ≥ 0.95 for TLI and CFI, and ≤ 0.08 for SRMR and RMSEA were considered for good model fit [34]. The Chi-square statistic ${(\chi }^2$) was not reported because of the sensitivity to sample size [34]. Instead, ${\chi }^2$/df were reported since ${\chi }^2$ and df increase as a function of the number of variables [34]. A ${\chi }^2$/df ratio < 3.00 indicated an acceptable fit [34].

In addition to validating the scale’s structure using EFA, we assessed other structures of the scale based on the judgment of item contents: (1) underlying dimension of health literacy (understand, find, assess, and use), underlying dimension of health system navigations (system level vs. organizational level). We developed several different models: unidimensional model obtained from EFA, correlated-factor model, bifactor model, and hierarchical model based on assumptions about the correlation between items, dimensions (sub-factors), and general factor. Bifactor model assumes that these dimensions are not correlated with the general factor (health system literacy), high-order model assumes that these four factors are correlated the general factor [35]. CFA is a good technique to evaluate goodness-of-fit among models [35,36]. We used CFA model to compare improved good fit indices among models. CFA analysis enables us to explore potential structure of the scale along with the structure obtained from EFA. Additionally, if multidimensional structure is applicable, the bifactor and high order models allow us to examine whether an overall score and sub-scores are required to report [35]. We used the entire sample for CFA analysis because this approach helps identify discrepancies in results from EFA and CFA in the same population [37].

2.3.3. Reliability

Cronbach’s alpha coefficient and composite reliability were used to calculate the internal consistency and reliability of the scale and subscales if applicable. Cronbach’s alpha is the most commonly used to measure of internal consistency based on correlation of items, with a value ranging between 0 to 1 [38]. Composite reliability produces a more accurate estimate of the reliability than Cronbach’s alpha because it allows factor loadings to vary while these loadings are constrained to be equal in Cronbach’s alpha [39]. A value of Cronbach alpha or composite reliability between 0.60 and 0.70 are considered acceptable, and values above 0.7 indicate good reliability [38]. However, the value of > 0.9 may suggest item redundancy and are typically not desirable [38].

2.3.4. Cross-Validation CFA Models

Once the optimal model that obtained good fit indices and acceptable reliability was selected, we validated the goodness-of-fit model across subsamples of the dataset using k-fold cross validation. In k-fold cross validation, the entire dataset was divided into k number of folds [40]. The process was repeated k times with one-fold used as the test set and the remaining (k-1) folds used as the training test in each iteration [40]. The value of k is generally recommended from 5 to 10, but the choice depends on the sample size of dataset [40]. In this study, we set k = 3, training the model on two-fold and evaluating it on the one-fold in each iteration.

2.3.5. Construct Validity

This study used a confirmatory factor analysis to examine factor correlation between measures of similar (convergent validity) and different constructs (discriminant validity) [41]. The 12-item short form Health Literacy Scale (HLS-SF12) developed by Tuyen V. Duong for Asian individuals aged 15 or older [42] was used for this purpose. The original scale has three dimensions: ‘health care’, ‘disease prevention’, and ‘health promotion’, with each item rated on a 4-point Likert scale (1=very difficult, 2=difficult, 3=easy, 4=very easy) [42]. Duong et al. (2019) reported a Cronbach’s Alpha coefficient of 0.49 to 0.72 for Health Care subscale, 0.64 to 0.77 for Disease Prevention subscale, and 0.63 to 0.81 for Health Promotion subscale [42].

Known-group validity was assessed by comparing mean differences by sociodemographic characteristics using t-test, one-way ANOVA, Post-Hoc Tests (Tukey’s HSD for homogeneous variances and Games-Howell for heterogenous variances), and Hedge’s effect size. The instrument score was calculated by the proportion of items with valid responses choosing ‘easy’ or ‘very easy’ with at least 80% of items containing valid responses [43]. While this dichotomous score may prevent assumption of equal intervals in ordinal scales, it can result in lost information due to dichotomization [44]. The polychotomous score can be used, which is aligned with previous validation studies of HLS-EU as well as lead to statistical analysis is more convenient when data is assumed as nearly normal distribution [14,44]. However, this type of score can be inflated by extreme response styles [44]. In this study, the polychotomous score was calculated using the formular in Tuong V. Duong et al. (2019): Score = (raw mean – 1) x (100/4) where raw mean is the mean of all valid responses, 4 is the range of the mean and 100 is the maximum standardized score [45]. In both approaches, the range of score is between 0 and 100. This study used dichotomous and polytomous scores for known-group validity to examine whether these two score types produce comparable results.

Health system literacy score was considered as the dependent variable while other variables were treated as independent variables: gender (men vs. women), age group (under 50, 50-64, vs. 65 or older), education (College/university degree, Postgraduate – Master/PhD, vs. other), income (under $60,000, $60,000 - $89,999, vs. ≥ $90,000), length of stay in Canada (< 5 years, 5 to less than 10 years, vs. ≥ 10 years), marital status (married vs. others), current employment status (employed, self-employed, vs. others), self-rated health (excellent to very good, good, vs. fair to poor), number of non-cancer chronic comorbidities (none, 1, 2, > 2, vs. do not know), diagnosed with cancer (yes vs. no), having a general practitioner/family physician (yes vs. no), assistance to see an HCP (yes vs. no). Effect size was measured using Hedges’ g for standardized difference in means, with Hedges’ g value of 0.2 - < 0.5 as small, 0.5 - < 0.8 as medium, and ≥ 0.8 as large effect [46].

2.3.6. Missing Data

The amount of missing data within HLS-CAN items was very small, ranging from 0.15% to 1.47%. (see Supplementary 3) Hence, a listwise deletion was used to address missing data.

SAS 9.4 was used for data cleaning and R programming was used for data analysis. The following R-packages were used in this study, including “psych” for exploratory factor analysis and reliability test, “lavaan”, “semTools” and “semPlot” for confirmatory factor analysis and path diagram, “effsize” for calculating effect size of mean differences, “caret” for cross-validation evaluation.

3. Results

3.1. Item Development

We adapted 10 items from the HLS₁₉ - NAV (D1, D5, D7, D9-D11, D13-D16) and developed 14 new items based on literature review and consulting with the Patient Advisory Committee. Following the Griese et al. (2020) approach, all items were designed to cover four dimensions of health literacy: understand (5 items), find (8 items), assess (4 items), and use (8 items) [4,14]. Additionally, the potential subscales based on the navigational health system conceptual frameworks include system level subscale (5 items) and organizational level subscale (7 items) [14].

Table 1 presents items comprised in the HSL-CAN scale. We modified item D10 emphasizing the understanding of rather than finding out about their rights, as a patient in the healthcare system (Item 5 in HLS-NAV). Item D11 was modified to highlight the ability to assess the quality of a particular health service rather than to find information about the quality of a particular health service. In our adapted scale, 25 items captured four dimensions of health literacy concept: understanding (5 items), find (6 items), assess (6 items), and use (8 items) as well as cover three levels of navigational health system with organizational and interactional level combined as organizational level, including system level (7 items) and organizational level (18 items).

3.2. Content Validity

The pre-test survey was conducted by emailing it to 12 colleagues. They were asked to review and provide their feedback on the understandability and content of the survey. We received some feedback regarding wordy questions, unclear terminology (e.g., insurance benefits), and concerns that item D22 was not suitable in the Canadian context. There were some minor revisions based on the stakeholders’ feedback (see Table 1).

3.3. Translation

The English version of the survey was translated into Chinese Simplify and Traditional version by YC and verified by NL, then translated back into English using ChatGPT 4.0 and verified by ATV. A few variations between back-translated version and the original version were resolved through discussion.

3.4. A Description of Study Sample

Among 681 participants, 97.80% were Canadian citizens or permanent residents, 55.07% were women, and half (50.07%) were aged between 50 and 64. A majority of participants lived in Ontario (58.15%) while the remainder lived in were from other provinces across Canada, including Quebec (12.19%), British Columbia (13.66%), Alberta (6.75%), Saskatchewan (1.03%), Manitoba (2.79%), and Atlantic provinces (5.43%).

About 78.85% of participants were born in Mainland China while others were born in Hong Kong (8.96%), Taiwan (9.54%), and Canada (2.35%). Most respondents (76.65%) have lived in Canada for at least 10 years. Approximately 97.7% participants obtained a college or university degree (45.37%) or postgraduate education (52.28%), while only 2.35% had completed high-school only or other types of education. Moreover, 86.34% were married, and 62.70% were either employed (45.08%) or self-employed (17.62%). Participants with a total household income under $60,000 accounted for 13.51% while 78.71% had a household income at least $60,000. Nearly one quarter (24.08%) of participants reported a fair or poor health. 56.83% had at least one non-cancer chronic comorbidity and 18.36% had been diagnosed with a cancer. 45.67% respondents reported regularly using traditional Chinese medication. While most participants (91.92%) reported having a general practitioner when they needed care, 44.35% need assistance to access healthcare providers (see Table 2).

3.5. Exploratory Factor Analysis

The overall KMO values were 0.9 and the Bartlett test of Sphericity was significant (p < 0.05), indicating an adequate sampling and significant correlation among different items. This suggests that data are suitable for factor analysis. The results from Parallel analysis and Weighted Least Square suggested that the number of factors is one. Additionally, the Scree plot illustrated a steep drop from first factor to second factor, indicating a one-factor structure of the scale (see Figure 1). Five items (D6, D8, D18, D20, D23) with loading under 0.40 were removed (see Table 3). A repeated KMO test and the Bartlett test Sphericity for the dataset of 20 remaining items indicated the 5 items-excluded dataset are suitable for factor analysis, with the overall KMO values of 0.9 and significant Bartlett test Sphericity (p < 0.05). Moreover, the VIF values ranged from 1.18 to 1.26, and no inter-item correlations exceeded 0.8, indicating that multicollinearity among items was not present (see Supplementary 4).

Confirmatory Factor Analysis and Internal Reliability

Based on the CFA output, the bifactor two-factor model and bifactor four-factor model did not support the theoretical models that grouping factors are uncorrelated with each other and with a general factor. Additionally, standard errors could not be estimated, and some estimated variances were negative, indicating the hierarchical two-factor model is ill-conditioned. Therefore, four models: one-factor model derived from EFA, the correlated two-factor model, the correlated four-factor model, and the hierarchical four-factor model were compared to determine which model best fit the data.

Table 4. presents fit indices, statistical indicators of the internal consistency for one-factor EFA (unidimensional model), two-factor model based on the navigational health system, correlated four-factor model based on health literacy definition, and hierarchical four-factor models. All goodness-of-fit indices yielded from four models are within acceptable cutoff values: CFI ≥ 0.95 (CFI_one-factor = 0.99, CFI_{correlated two-factor} = 0.99, CFI_{correlated four-factor} = 0.99, CFI_{hierarchical four-factor} = 0.99), TLI ≥ 0.95 (TLI_one-factor = 0.99, TLI_{correlated two-factor} = 0.99, CFI_{correlated four-factor} = 0.99, CFI _{hierarchical four-factor} = 0.99), RMSEA _(90%CI) ≤ 0.08 (RMSEA_one-factor = 0.03 _{(0.02 – 0.03)}, RMSEA _{correlated two-factor} = 0.03 _{(0.02 – 0.03)}, RMSEA _{correlated four-factor} = 0.02_(0.02-0.03), RMSEA _{hierarchical four-factor} = 0.02_(0.02-0.03)), SRMR ≤ 0.08 (SRMR _one-factor = 0.04, SRMR _{correlated two-factor} = 0.04, SRMR _{correlated four-factor} = 0.04, SRMR _{hierarchical four-factor} = 0.04), ${\mathit{\chi }}^2$/df < 3.00 (${\chi }^2$/df _one-factor = 1.75, ${\chi }^2$/df _{correlated two-factor} = 1.76, ${\chi }^2$/df _{correlated four-factor} = 1.63, ${\chi }^2$/df _{hierarchical four-factor} = 1.61). However, improved fit indices were found in models with four factors compared to in those with one or two factors.

Table 4. A comparison of models from EFA and theoretical frameworks.

Indices	One-factor	Two-factor	Four-factor
		Correlated	Correlated	Hierarchy
CFA
Fit indices
n	669	669	669	669
CFI	0.99	0.99	0.99	0.99
TLI	0.99	0.99	0.99	0.99
SRMR	0.04	0.04	0.04	0.04
RMSEA (90%CI)	0.03 (0.02–0.03)	0.03 (0.02-0.03)	0.02 (0.02 –0.03)	0.02 (0.02 – 0.03)
${\mathit{\chi }}^2$	250.28	249.03	229.18	229.40
df	170	169	164	166
${\mathit{\chi }}^2$/df	1.75	1.76	1.63	1.61
Inter-factor correlation
F1 ~ F2	-	0.97	0.88	-
F1 ~ F3	-	-	0.84	-
F1 ~ F4	-	-	0.83	-
F2 ~ F3	-	-	0.92	-
F2 ~ F4	-	-	0.94	-
F3 ~ F4	-	-	0.89	-
G ~ F1	-	-	-	0.89
G ~ F2	-	-	-	0.99
G ~ F3	-	-	-	0.94
G ~ F4	-	-	-	0.94
Reliability
Cronbach’s alpha
Overall score	0.82	0.82	0.82	0.82
Factor 1	-	0.62	0.56	0.56
Factor 2	-	0.75	0.60	0.60
Factor 3	-	-	0.57	0.57
Factor 4	-	-	0.51	0.51
Composite reliability
Overall score	0.82	0.82	0.82	0.82
Factor 1	-	0.63	0.57	0.57
Factor 2	-	0.75	0.59	0.59
Factor 3	-	-	0.57	0.57
Factor 4	-	-	0.52	0.52

Table 4. presents fit indices, statistical indicators of the internal consistency for one-factor EFA (unidimensional model), two-factor model based on the navigational health system, correlated four-factor model based on health literacy definition, and hierarchical four-factor models. All goodness-of-fit indices yielded from four models are within acceptable cutoff values: CFI ≥ 0.95 (CFI_one-factor = 0.99, CFI_{correlated two-factor} = 0.99, CFI_{correlated four-factor} = 0.99, CFI_{hierarchical four-factor} = 0.99), TLI ≥ 0.95 (TLI_one-factor = 0.99, TLI_{correlated two-factor} = 0.99, CFI_{correlated four-factor} = 0.99, CFI _{hierarchical four-factor} = 0.99), RMSEA _(90%CI) ≤ 0.08 (RMSEA_one-factor = 0.03 _{(0.02 – 0.03)}, RMSEA _{correlated two-factor} = 0.03 _{(0.02 – 0.03)}, RMSEA _{correlated four-factor} = 0.02_(0.02-0.03), RMSEA _{hierarchical four-factor} = 0.02_(0.02-0.03)), SRMR ≤ 0.08 (SRMR _one-factor = 0.04, SRMR _{correlated two-factor} = 0.04, SRMR _{correlated four-factor} = 0.04, SRMR _{hierarchical four-factor} = 0.04), ${\mathit{\chi }}^2$/df < 3.00 (${\chi }^2$/df _one-factor = 1.75, ${\chi }^2$/df _{correlated two-factor} = 1.76, ${\chi }^2$/df _{correlated four-factor} = 1.63, ${\chi }^2$/df _{hierarchical four-factor} = 1.61). However, improved fit indices were found in models with four factors compared to in those with one or two factors.

Table 4. A comparison of models from EFA and theoretical frameworks.

Indices	One-factor	Two-factor	Four-factor
		Correlated	Correlated	Hierarchy
CFA
Fit indices
n	669	669	669	669
CFI	0.99	0.99	0.99	0.99
TLI	0.99	0.99	0.99	0.99
SRMR	0.04	0.04	0.04	0.04
RMSEA (90%CI)	0.03 (0.02–0.03)	0.03 (0.02-0.03)	0.02 (0.02 –0.03)	0.02 (0.02 – 0.03)
${\mathit{\chi }}^2$	250.28	249.03	229.18	229.40
df	170	169	164	166
${\mathit{\chi }}^2$/df	1.75	1.76	1.63	1.61
Inter-factor correlation
F1 ~ F2	-	0.97	0.88	-
F1 ~ F3	-	-	0.84	-
F1 ~ F4	-	-	0.83	-
F2 ~ F3	-	-	0.92	-
F2 ~ F4	-	-	0.94	-
F3 ~ F4	-	-	0.89	-
G ~ F1	-	-	-	0.89
G ~ F2	-	-	-	0.99
G ~ F3	-	-	-	0.94
G ~ F4	-	-	-	0.94
Reliability
Cronbach’s alpha
Overall score	0.82	0.82	0.82	0.82
Factor 1	-	0.62	0.56	0.56
Factor 2	-	0.75	0.60	0.60
Factor 3	-	-	0.57	0.57
Factor 4	-	-	0.51	0.51
Composite reliability
Overall score	0.82	0.82	0.82	0.82
Factor 1	-	0.63	0.57	0.57
Factor 2	-	0.75	0.59	0.59
Factor 3	-	-	0.57	0.57
Factor 4	-	-	0.52	0.52

Despite the improved goodness-of-fit in models with four factors, the factors were highly correlated with each other. In the correlated two-factor model, the inter-factor correlation was 0.97, indicating that two factors shared a large proportion of variances or probably measured the same construct. Similarly, the inter-factor correlations in the correlated four-factor model ranged from 0.84 to 0.94 and the correlation between a general factor and four factors in the hierarchical four-factor models ranged from 0.89 to 0.99. These findings strongly suggested that all four factors are measuring the same underlying construct. In other words, a unidimensional model may be more appropriate.

Moreover, the adapted scale demonstrated a good internal consistency with an overall Cronbach’s alpha coefficient of 0.82 and an overall composite reliability value of 0.82 across models. However, the internal reliability subscales are only acceptable, with the Cronbach’s alpha coefficient of 0.62 for factor 1 and 0.76 for factor 2 in two-factor model. In contrast, the subscales of the four-factor models had an unacceptable internal consistency with Cronbach’s values and composite reliabilities falling below 0.6.

3.6. Cross-Validation

CFA was conducted on the one-factor model using a sub-sample (n=227) from the entire dataset (n=681) using 3-fold cross validation procedure. The CFA model obtained a good fit across the cross-validation samples where all fit indices fell within the acceptable ranges, although these indices were not as strong as those in the model with the entire model (see Table 5). This suggested that CFA goodness-of-fit indices may be influenced by sample size.

3.7. Construct Validity

3.7.1. Convergent and Discriminant Validity

Regarding the HLS-SF12 scale, our study found that the HLS-SF12 obtained goodness-of-fit model with two factors: ‘Health Care’ and ‘Prevention and Health Promotion’ (CFI=0.95, TLI=0.93, RMSEA_90%CI = 0.04 (0.02-0.05), SRMR=0.05, ${\chi }^2$/df = 96.146/53=1.81). We found a Cronbach’s alpha coefficient of 0.53 for Health Care and 0.63 for ‘Prevention and Health Promotion’ subscale (see Supplementary 5). The two subscales of the HLS-SF12 were used to evaluate convergent and discrimination validity in this study. We hypothesized that our construct was more likely related to ‘Health Care’ construct and was not closely related to ‘Prevention and Health Promotion’ construct. The CFA outputs showed a strong correlation with the “health care” subscale (r=0.79, p<0.05), supporting a good convergent validity. In other words, both measures may measure a similar construct. Moreover, our scale demonstrated a good discriminant validity, as evidenced by a lower correlation coefficient between our scale and ‘prevention and health promotion’ subscale (r=0.21, p=0.002), suggesting that two measures may be measuring distinct constructs (see Supplementary 6).

3.7.2. Known-Group Validity

Table 6 and Supplementary 7 present differences in mean scores of health system literacy across socio-demographic characteristics and their effect sizes. No significant difference in mean health system literacy scores and uncertain or very small effect sizes were found for length of stay in Canada and marital status (p-value > 0.05) with dichotomous scores and polytomous scores.

Significant differences in mean health system literacy scores with both dichotomous and polytomous was observed in gender, age group, highest level of education, income level, number of non-cancer chronic comorbidities, and self-rated health. Among these groups, education group, income level, and number of non-cancer chronic comorbidities have the moderate to high effect sizes (Hedges’s g ≥ 0.5). Participants with a postgraduate degree (Master/PhD) reported the highest health system literacy score, particularly when compared to those with the lowest education level (ES=0.73 in dichotomous scores; ES=1.05 in polytomous scores).

The highest health system literacy scores were observed among individuals with a total annual household income of $60,000 to $89,999, followed by those with total income ≥ $90,000 (ES=-0.72 in dichotomous scores; ES=-0.71 in polytomous scores). Conversely, those with a total annual household income of < $60,000 reported the lowest score (ES=-0.84 in dichotomous scores; ES=-0.96 in polytomous scores). Non-cancer chronic comorbidity status was significantly associated with health system literacy score. The highest scores were found in participants with at least two comorbidities while the lowest score was observed in those uncertain about their health conditions, with effect sizes ranging from 0.96 to 1.12 (dichotomous scores) and 0.88 to 1.09 (polytomous scores).

In contrast, differences in mean literacy scores among age groups, genders, and self-rated health had small to moderate effect sizes. Individuals aged under 50 reported the highest score, followed by those aged 50 to 64 and 65 or older, with small to moderate differences ranging from 0.29 to 0.76 (dichotomous scores) and 0.28 to 0.72 (polytomous scores). Men had a higher mean score compared to women, with a small effect size (ES=0.23). Individuals who rated their health as excellent to very good had the highest health system literacy score whereas those with good self-rated health had the lowest score (ES=0.27)

However, inconsistent findings between dichotomous scores and polytomous scores were found for current employment status, cancer diagnosis, having a general practitioner/family physician, and the need for assistance to see an HCP. Individuals employed or self-employed as well as those who need for assistance to access an HCP reported a higher mean health system literacy score (both dichotomous score and polytomous scores) than their counterparts, but these differences are small and only significant for dichotomous scores. Similarly, cancer patients reported lower mean scores than those without cancer; however, only the difference in dichotomous scores was significant and its true effect size was uncertainty (-0.17, 95% CI: -0.37 - 0.02). In contrast, participants who have a general practitioner/family physician had higher scores than those without having HCPs, but a significant difference was found only in polytomous score, with a small effect size (ES=0.49)

3.8. Patient and Public Engagement

Based on insightful feedback from our Patient Advisory Committee, most members agreed that the final scale is relevant and effectively captures the construct of health system literacy. The scale reflects patient experiences related to navigating the health system. One patient partner recommended retaining item D6 (How easy is it for you to find a family doctor or primary healthcare provider near you?) and D8 (How easy is it for you to book an appointment with a healthcare provider?). However, we considered that D7 (How easy is it to understand how to get an appointment with a particular health service?) may cover the content of D6 and D8.

The committee found the scale generally clear and easy to understand. However, some medical jargon terms (e.g., Healthcare System, Services, Providers, Insurance, ‘right person’) should be clarified or illustrated with simple examples. To minimize a high non-response rate due to questions with more words, we kept our online survey more concise, although this limited our ability to define the technical terms in detail. A few suggestions for improving the scale can included rewording D5, using simpler response categories (e.g., very difficult, difficult rather than extremely difficult, somewhat difficult), and providing examples to clarify the technical terms.

4. Discussion

The study presents the validation of psychometric properties of the HSL-CAN scale among a Chinese immigrant population aged 30 or older in Canada. Our findings addressed literature gaps about a need for understanding how well Chinese populations understand and use the Canadian healthcare system.

The HSL-CAN is comprised of 20 items, including 10 items adapted from the HLS₁₉-NAV (D1, D5, D7, D9-D11, D13-D16) and 10 newly developed items (D2-D4, D12, D17, D19, D21, D22, D24, D25). These items cover four dimensions of health literacy: understanding (5 items), finding (6 items), accessing (5 items), and using (4 items). Guided by Griese et al. (2020)’s conceptual framework [4], our items capture a broad range of navigational health system tasks across three levels: system (7 items – D1, D2, D5, D10, D12, D22, D24), organization (11 items – D3, D4, D7, D9, D11, D13, D14, D17, D19, D21, D25), and interaction (2 items- D15, D16). Our scale aimed to encompass various aspects of health literacy across multiple levels of health system navigation to identify areas where healthcare providers and decision-makers should focus their attention.

However, the exploratory factor analysis and confirmatory factor analysis indicated that the 20-item scale obtained goodness-of-fit within a one-factor construct. This finding aligns with the results of Griese et al. (2022) study, which reported acceptable fit indices for a single-factor model [4,14]. In contrast, one study conducted in Russia by Drapkina et al. (2025) indicated that HLS₁₉-NAV-RUS demonstrated improved fit values in a two-factor CFA model compared to one-factor CFA model [16]. This difference suggests cultural differences that might influence cross-cultural adaptation of the HLS₁₉ – NAV scale across countries with different healthcare systems [16,47].

The HLS₁₉-NAV scale evaluated how well patients understand and navigate the healthcare system. The structure of a healthcare system (e.g., decentralized vs. centralized healthcare systems) might contribute to variation in the factor structure of the scale. In decentralized health systems, power, responsibilities, resources, and decisions are typically redistributed to local governments. In contrast, centralized systems typically place these responsibilities in the national/sub-national government [48]. Because the delivery of health services and resources at smaller jurisdictional areas are influenced by organizational operations and policies [49], patients from countries with decentralized health systems (e.g., European countries [50], Canada [51], USA [52]) must navigate various types of services, healthcare providers, and regulations at multiple levels. As a result, a one-factor scale can reflect a general ability to navigate the healthcare system among populations from decentralized health systems. Conversely, in centralized health system countries (e.g. Russia), regulations and healthcare services that are established by national governments are more uniform across regions [49,53], allowing their patients to distinguish between system level experiences (understanding how the system works) and organizational-level experiences (navigating healthcare services and providers).

Items in our scale with factor loadings ≥ 0.4 were retained in the model, but the communality values were relatively low, ranging from 0.154 to 0.225. Communality refers to the proportion of each item’s variance explained by the factors [54]. Higher communality values indicate that the variance of the item is more likely to be explained by extracted factors [26]. Acceptable cutoff values for communality should be between 0.25 and 0.4, with an ideal value above 0.6 [55,56]. However, Eaton et al. (2019) argued that stricter values might suggest better goodness-of-fit of the model, but this must be balanced with retaining an adequate number of appropriate items [57]. Although item D7 was from the original HLS₁₉ – NAV scale and conceptually aligned with the construct, it had the lowest communality value in this study (communality=0.154). Some factors can contribute to this. First, participants may have responded to the question based on their belief rather than their skill or experience, leading to measurement error. Second, many Chinese patients often rely on their family members to schedule health appointments, so their response might not reflect their navigational skills and thus have a reduction in their correlation with the factor.

Total variance in exploratory factor analysis is a sum of the variance explained and unique variances [54]. In our study, the risk of measurement error is substantial. Factors contributing to potentially great measurement errors in the survey include cognitive efforts to correctly retrieve the information asked (e.g., recall bias, salience), respondent cooperation (e.g., willing to make effort to provide accurate responses), survey mode [58], ambiguous or complex questions, limited prior experience with the healthcare system, and a lengthy questionnaire [59].

The instrument obtained a good reliability and construct validity. Cronbach’s alpha value was 0.81 and composite reliability was 0.812 on Chinese immigrants aged 30 or older in Canada. These results are consistent with the Griese et al. (2022) study, where Cronbach’s alpha and composite reliability values ranged from 0.88 to 0.94 across European countries [14], and with Drapkina et al. (2025), where Cronbach’s alpha value was 0.85 [16]. Moreover, this study also examined convergent and discriminant validity through evaluating the relationship between the HSL-CAN and two constructs of the HLS₁₉-SF12 (Health Care and Prevention and Health Promotion). The construct of HSL-CAN was significantly associated with the HLS₁₉-SF12 health care construct but weakly related to the construct of prevention and health promotion, suggesting that all items in the scale measure a specific type of health literacy rather than general health literacy.

Regarding known-group validity, health system literacy was associated with some sociodemographic factors, including gender, age group, highest level of education, income level, self-rated health, number of non-cancer chronic comorbidities, diagnosed with cancer, having a general practitioner for health need, and a need for assistance to access an HCP for either dichotomous scoring or polytomous scoring. Particularly, education group, income level, and number of non-cancer chronic comorbidities showed the largest effect size. Griese et al. (2022) similarly revealed that lower NAV-HL scores were associated with high financial deprivation and older age [14]. Therefore, this scale demonstrates a practical applicability in distinguishing subgroups and evaluating population health system literacy for research purposes.

However, the choice of scoring method may cause inconsistences in results. Significant differences in mean health system literacy scores were found among gender, age group, highest level of education, income level, self-rated health, and number of non-cancer chronic comorbidities for both dichotomous and polytomous. In contrast, significant differences were observed with dichotomous scoring for current employment status, diagnosed with cancer, and need assistance to access an HCP while differences in scores were significantly found with polytomous scores for having a general practitioner group.

These inconsistencies may result from differences in scoring methods and the natures of the variables. First, dichotomous scoring is more likely to cause information loss since it is sensitive only to responses of ‘easy’ or ‘very easy’ and disregarding other response options. In contrast, polytomous scoring covered all responses of the scale, which might provide more nuanced variances. As a result, significant differences in dichotomous scores were found for variables groups where respondents were more likely to choose ‘easy’ or ‘very easy’. Second, the nature of variable groups may impact the results. For example, cancer patients may be less likely to rate navigational health system tasks as “easy’ or ‘very easy’ than non-cancer patients because of the known complexity and fragmentations of cancer care systems [60]. Therefore, the choice of scoring method should align with the study’s objectives whether that is to measure generally individual’s health literacy score or to assess the perceived ease of the navigating a specific health system services or process.

Strengths and Limitations

The strength of this study lies in the comprehensive testing and reporting of psychometric properties for a health system literacy scale adapted from HLS₁₉-NAV among Chinese populations in Canada. This study provides strong evidence supporting the successful cross-cultural adaptation of HLS₁₉-NAV into Chinese, within the context of Canadian health system. Moreover, adherence to the COSMIN reporting guidelines allowed for a rigorous evaluation of the psychometric properties and enhanced the transparency of the findings including identification of potential risks of bias. This can open more opportunities for future studies to improve their qualities of methodological quality as well as the instrument’s measurement properties.

However, our study has some limitations. First, psychometric properties of the HSL-CAN scale were evaluated only in individuals aged 30 or older, so future studies are required to assess reliability and validity on younger populations under the age of 30. Second, potential selection bias must be acknowledged in our data collection. A large percentage of individuals in our study held Master/PhD degrees, which cannot accurately represent the whole of Chinese Canadians. Furthermore, because the data were collected through an online survey, individuals who do not have the Internet access or mobile devices could not participate in the study. We used an online survey due to its affordability and common use, although this modality posed limitations in interacting with respondents. To provide a more comprehensive and accurate assessment of health system literacy, future studies should considered other alternative modes of survey administration such as face-to-to-interview or computer-assisted interviewer administration, which allow researchers to interact and clarify vague questions.

5. Conclusions

Adequate knowledge of the Canadian health system and navigational skills are necessary for patients to navigate the health system and access the care they need. Understanding health system literacy enables healthcare providers and decision-makers to identify what and how healthcare structures and policies require revision to facilitate health navigation processes as well as to develop support services that enhance health system literacy among patients and caregivers. This study demonstrates that the health system literacy scale adapted from HLS₁₉-NAV has obtained suitable structure validity, good construct validity, and high reliability within the Chinese population in Canada. It can serve as a valuable tool for healthcare providers and researchers to assess how well Chinese Canadians understand and use the healthcare system. However, the scale should be revised and validated for use with the general Canadian population to ensure reliable and comprehensive assessment of health system literacy.