Article Abstract

Online ISSN: 1099-176X    Print ISSN: 1091-4358
The Journal of Mental Health Policy and Economics
Volume 26, Issue 1, 2023. Pages: 33-57
Published Online: 1 March 2023

Copyright © 2023 ICMPE.

Health Reporting from Different Data Sources: Does it Matter for Mental Health?

Joyce Mason,^1* Audrey Laporte², James Ted McDonald³, Paul Kurdyak⁴, Claire de Oliveira⁵

¹PhD , Institute for Mental Health Policy Research, Centre for Addiction and Mental Health Toronto, Ontario, Canada.
²PhD, Director, Canadian Centre for Health Economics; Director and Professor, Institute of Health Policy Management and Evaluation; Professor, Department of Economics, University of Toronto, Senior Adjunct Scientist, ICES, Toronto, Ontario, Canada.
³PhD, Professor of Economics, Department of Political Science, University of New Brunswick, Fredericton, New Brunswick, Canada.
⁴MD, PhD, Psychiatrist and Clinician Scientist, Institute for Mental Health Policy Research, Centre for Addiction and Mental Health; Senior Core Scientist and Mental Health and Addictions Program Lead, ICES; Professor, Department of Psychiatry and Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, Ontario, Canada.
⁵PhD, Senior Scientist and Senior Health Economist, Institute for Mental Health Policy Research, Centre for Addiction and Mental Health; Senior Adjunct Scientist, ICES; Associate Professor, Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, Ontario, Canada.

Background: Mental disorders are typically stigmatized conditions associated with negative stereotypes, which may lead individuals to underreport them. Thus, survey data may be subject to biases. Although administrative data has some limitations, it is an alternative data source that may be considered more objective.

Aims of the Study: This study aimed to identify the degree of agreement between survey and administrative health care data for mental health conditions, factors affecting underreporting, and whether underreporting also occurs for physical health conditions.

Methods: We used Ontario data from the Canadian Community Health Survey linked to health records to examine the presence of mental health conditions (i.e., schizophrenia and mood disorders) and select physical health conditions (i.e., diabetes and cancer). Using administrative data as the reference standard, we created four categories for each health condition based on the level of agreement between the two data sources: consistent cases and non-cases (i.e., individuals with concordant data based on their reported health condition), and people who were found to underreport and overreport a condition (i.e., where the condition was present in the administrative data, but not in the survey data and vice-versa, respectively). The overall level of agreement was assessed using Cohen’s kappa statistic. Probit regressions were estimated to determine the factors affecting underreporting.

Results: The Kappa statistics for mood disorder was fair (k= 0.26) and moderate for schizophrenia (k = 0.49). Physical health conditions had higher kappa values (diabetes, k = 0.81; ever having cancer, k = 0.68), with the exception of currently having cancer (k = 0.24). Underreporting was highest for the most stigmatizing condition, schizophrenia (63%), followed by mood disorders (39%) and cancer (39%), and lowest for diabetes (25%). Older age, being born in Africa and Asia, and being employed all increased the probability of underreporting among individuals identified in the administrative data; the opposite held for social assistance.

Discussion: We extended previous work on mental health reporting by combining survey data with administrative data to examine the level of agreement between respondents’ self-reported mental health and administrative records. The data include some mental disorders not studied previously. We examined the entire adult population; this is important because prevalence of schizophrenia may be less common among older population groups due to higher mortality among this patient population. Additionally, there may be potential age-related differences in stigma and mental health conditions. The administrative health data captured only health services covered by the public provincial health insurance plan and thus did not capture medical care provided by psychologists, social workers, and nurses. While this would affect Kappa statistic values, it does not directly affect the underreporting analyses.

Implications for Health Care Provision and Use: Our results suggest that disclosure of mental health conditions may differ by the level of stigma, which has implications for obtaining accurate estimates of mental health prevalence from self-reported data sources.

Implications for Health Policies and Implications for Further Research: It may be useful to use a combination of both survey and administrative data when estimating the prevalence of mental disorders. Future research should seek to examine overreporting and its determinants.