Evaluation of Mental Health and Mental Illness Activities of Health Canada and the Public Health Agency of Canada 2010-2011 to 2014-2015

Executive summary

Mental health and mental illness in Canada are complicated issues that have both direct and indirect impacts on a significant number of Canadians every year. Addressing mental health and mental illness involves a variety of stakeholders across multiple jurisdictions and in both the public and private sector.

This evaluation examines four specific activities of Health Canada and the Public Health Agency of Canada related to mental health and mental illness. These activities are: the funding, and management of the funding, to the Mental Health Commission of Canada; funding, and the management of funding, to the Mood Disorder Society of Canada for the Canadian Depression Research and Intervention Network; surveillance of mental health, mental illness and suicide; and mental health promotion including the work to develop a Federal Framework for Suicide Prevention.

The evaluation does not attempt to address the broad spectrum of federal government activities in the area of mental health and mental illness. Within Health Canada, a concurrent evaluation is underway with regards to the First Nations and Inuit Health Branch (FNIHB) Mental Wellness Programs. Learnings from both of these evaluations will be used for future programmatic decisions.

This evaluation covered mental health and mental illness activities of Health Canada and the Agency for the period from 2010-11 to 2014-15. Data collection was completed in July 2015.

The evaluation was undertaken in fulfillment of the requirements of the Financial Administration Act and the Treasury Board of Canada's Policy on Evaluation (2009).

Evaluation Purpose and Scope

The purpose of the evaluation was to assess the relevance and performance of a specific subset of mental health and mental illness activities within Health Canada and the Public Health Agency of Canada.

Program Description

The activities of Health Canada covered by this evaluation include the funding, and management of the funding, to the Mental Health Commission of Canada (MHCC) and the Mood Disorder Society of Canada (MDSC) for the Canadian Depression Research and Intervention Network (CDRIN). The Public Health Agency of Canada's activities covered in this evaluation include mental health promotion and suicide prevention, as well as positive mental health and mental illness surveillance.

Conclusions - Relevance

Continued Need

A significant number of Canadians are directly and indirectly affected by poor mental health and mental illness. Every year, one in ten Canadians will experience a mental illness and one in three will experience a mental illness or substance use disorder at some point in their lifetime. In 2012, 40% of adults in Canada had a family member with substance use, emotional or mental health problems. Each year close to 4,000 Canadians die by suicide.

Certain segments of the Canadian population have higher rates of mental illness, suicide or suicide ideation. Suicide is the second leading cause of death among youth, and the prevalence of suicide is a significant problem in some First Nations and Inuit communities.

The direct cost of poor mental health and mental illness was estimated as close to $8 billion in 2008. The indirect annual costs have been estimated to range between $11 billion and $50 billion.

Alignment with Government Priorities

Various federal announcements and international declarations have shown the importance of mental health as a federal government priority. Since the 2006 publication of Out of the Shadows at Last, mental health has been a priority. The creation of the MHCC and the intent to support a renewed mandate in Budget 2015 shows the support for focussed attention on mental health. Additional funding has also been provided to the MDSC to focus on depression and post-traumatic stress disorder.

In May 2013, members of the World Health Assembly, including Canada, formalized mental health as a priority by adopting the World Health Organization's Comprehensive Mental Health Action Plan 2013-2020. This supports the belief that "there is no health without mental health".^{Footnote 1} The United Nations has recently adopted Sustainable Development Goals which include the promotion of mental health and well-being.^{Footnote 2}

The passing of An Act respecting a Federal Framework for Suicide Prevention in 2012, and the resulting work to develop a Framework has highlighted the priority the federal government is placing on suicide prevention.

Alignment with Federal Roles and Responsibilities

The Department of Health Act details Health Canada's responsibilities for the promotion and restoration of the mental well-being of Canadians. The Public Health Agency Act outlines the Public Health Agency of Canada's role in health promotion and health surveillance. The federal government also has responsibilities for mental health services for specific federal populations. Mental health services are under the jurisdiction of provinces and territories for the majority of Canadians. However, the federal government has a role to play in coordination and collaboration of mental health and mental illness activities, in conducting and reporting on mental health, mental illness and suicide surveillance, as well as being a resource for information on best practices and innovation.

Conclusions - Performance

Achievement of Expected Outcomes (Effectiveness)

The MHCC has been successful in addressing the three specific elements of its original mandate. The original mandate called for the development of a national strategy, an anti-stigma campaign and the development of a knowledge exchange centre. Changing Lives, Changing Directions: A Mental Health Strategy for Canada was released in 2012, after extensive national consultations. Work has been undertaken across the country to respond to the six Strategic Directions, the 26 priorities and the 109 recommendations. Provincial and territorial governments along with non-governmental organizations (NGO) and federal departments have been working to implement the Strategy. The lack of clear action plans and accountability for recommendations has been seen as a hindrance to full implementation of the Strategy.

The other aspects of the MHCC's original mandate, the Anti-Stigma campaign (Opening Minds) and the Knowledge Exchange Centre (KEC), have been successful in their initial efforts. Opening Minds is showing success in reducing stigma amongst target populations. Expansion and reiteration of messaging will be important to continue reducing stigma. The KEC has developed partnerships and increased the accessibility of research and guidelines related to mental health. The consistent implementation of dissemination plans for research has increased the reach and availability of reliable information.

The MDSC through the Canadian Depression Research and Intervention Network has set up multiple Depression Hubs across Canada and has successfully incorporated the voice of people with lived experiences into the research process, including the identification of research priorities.

The MHCC has been recognized as a convenor that is able to bring partners together to advance the priorities of the Mental Health Strategy for Canada. The Public Health Agency of Canada was also seen as successful in creating partnerships, with respect to data initiatives and the work to develop the Federal Framework for Suicide Prevention. As there are multiple federal departments, levels of governments and NGOs engaged in mental health, partnerships were seen as crucial to share best practices, prevent duplication of efforts, and coordinate activities in mental health and mental illness.

Concerns were raised that the roles and responsibilities across the health portfolio related to mental health, mental health promotion, mental illness and suicide prevention were not clearly defined or understood. The lack of an overall vision was also noted. Furthermore, there are numerous federal departments with activities or responsibilities in these areas. There was interest in the development of a community of practice or hub for mental health within the federal government.

The Public Health Agency of Canada's work on positive mental health surveillance and mental illness surveillance has resulted in new knowledge products and has expanded and improved surveillance systems.

Mental health promotion work by the Public Health Agency of Canada has advanced the understanding of the difference between mental health and mental illness. This has been accomplished through support to international and national efforts.  Such understanding is at the core of the work on mental health promotion. Many of the Public Health Agency of Canada's community-based programs address elements of positive mental health and could be better leveraged by stakeholders to share consistent messaging regarding mental health, and promote mental health across the lifespan. The lack of consistent terminology (e.g., mental wellness, positive mental health, mental health and mental illness) is seen as an impediment to full collaboration on work in the area of mental health promotion.

The Public Health Agency of Canada's work to develop the Federal Framework for Suicide Prevention was recognized as a strong collaborative effort.  As per the requirements of the Act respecting a Federal Framework for Suicide Prevention, the Framework will outline federal guiding principles, strategic objectives and priorities in suicide prevention. The requirement for ongoing monitoring and reporting on the implementation of the Framework, were seen as crucial elements to ensure accountability, decision making and continuous improvement of federal efforts in suicide prevention.

Demonstration of Economy and Efficiency

The funding arrangements with MHCC and MDSC have had active management by Health Canada. MHCC has managed to leverage over 350 partnerships to further their work on anti-stigma, suicide prevention, workplace mental health and Mental Health First Aid. Further work with Aboriginal and non-traditional organisations was identified as an area for future opportunity.

No additional funds were provided for priority initiatives within the Public Health Agency of Canada, such as the positive mental health surveillance and the work to develop the Federal Framework for Suicide Prevention. Internal branch reallocations of funds have allowed the Public Health Agency's mental health, mental illness and suicide work to continue.

The Public Health Agency of Canada priority areas for mental health and mental illness have been defined; however the evaluation noted that the roles and responsibilities of Health Canada, the Public Health Agency of Canada and the Canadian Institutes of Health Research regarding mental health have not been clearly articulated and there is a lack of consistent terminology. Gaps were identified in terms of information sharing to all relevant program areas within the  Health Portfolio. It is expected that a clearly articulated vision would facilitate communication within the Health Portfolio and with the public.

Recommendations

Recommendation 1

Recognizing the continued significant need across Canada in the area of mental health and mental illness, a mental health vision for the Health Portfolio should be articulated, including clarification and communication of roles and responsibilities regarding mental health, mental health promotion, mental illness, suicide prevention, and surveillance of mental illness and mental health.

Currently, Health Canada, the Public Health Agency of Canada and the Canadian Institutes of Health Research engage in activities related to mental health and mental illness. Activities within these organizations are complementary and areas of intersection exist. However, the vision or overall goals for mental health and mental illness has not been clearly articulated or communicated. By articulating the vision and detailing the relationship between the activities of organizations and the vision, there will be opportunities to ensure that efforts are aligned and the potential for duplication is minimized. Having a clearly articulated vision will aid in communication with external and internal partners.

Many federal departments and external partners working in the broad area of mental health and mental illness were unsure of who to contact regarding various initiatives. Fluid communication between the Health Portfolio and these partners is important; however these relationships are frequently managed on a project basis. There was a desire for a more coordinated community of practice to bring together those working on mental health within the federal government. The feasibility of this could be explored.

Recommendation 2

Encourage MHCC, for the balance of its mandate, to expand outreach to stakeholders, particularly organizations that the Commission has not traditionally worked closely with, such as organizations addressing substance abuse and National Aboriginal Organizations.

The Mental Health Commission of Canada has strong partnerships across various sectors, but there are areas where partnerships and collaborations could be strengthened. Evidence of co-morbidities are becoming more apparent as data become available. Co-morbidity between substance abuse and mental health has garnered particular attention with several provinces and territories combining mental health programming with addictions and substance abuse. The MHCC and the Canadian Centre on Substance Abuse could work more closely on issues of joint interest.

Another area that could be strengthened by further outreach includes Indigenous populations. While some work has been undertaken on Strategic Directions 4 and 5 of the Mental Health Strategy, there has not been a strong focus on Aboriginal mental health by the Mental Health Commission of Canada. Stronger partnerships and relationships with National Aboriginal organizations, Aboriginal communities and Health Canada's First Nations and Inuit Health Branch could improve efforts in this area. It will be important for the MHCC's activities to include these important populations.

Management Response and Action Plan

Evaluation of Mental Health and Mental Illness Activities of Health Canada and Public Health Agency of Canada

The election of a new government in October 2015, after the investigative phase of this evaluation had concluded, has changed the context of the work of the Health Portfolio in relation to mental health and mental illness.  The new government's priorities include working with provinces and territories to improve access to mental health services through a new Health Accord and measures directed at specific populations (first responders and veterans, among others).

The new government has also made a commitment to a strong focus on results and to regular assessments of the effectiveness of programs and services.  These new priorities will shape the work of the Health Portfolio going forward, and are reflected in the recommendations of this report. Future evaluations will be better positioned to assess the impact of the Portfolio's mental health initiatives.

1.0 Context

Mental health and mental illness in Canada are complicated issues that have both direct and indirect impacts on a significant number of Canadians every year. Addressing mental health and mental illness involves a variety of stakeholders across multiple jurisdictions and in both the public and private sector.

This evaluation examines four specific activities of Health Canada and the Public Health Agency of Canada related to mental health and mental illness. These activities are: the funding, and management of the funding, to the Mental Health Commission of Canada; funding, and the management of funding, to the Mood Disorder Society of Canada for the Canadian Depression Research and Intervention Network; surveillance of mental health, mental illness and suicide; and mental health promotion including the work to develop a Federal Framework for Suicide Prevention.

The evaluation does not attempt to address the broad spectrum of federal government activities in the area of mental health and mental illness. Within Health Canada, a concurrent evaluation is underway with regards to the First Nations and Inuit Health Branch Mental Wellness Programs. Learnings from both of these evaluations will be used for future programmatic decisions.

2.0 Evaluation Purpose

The purpose of the evaluation was to assess the relevance and performance of four specific mental health and mental illness activities of Health Canada and the Public Health Agency of Canada for the period of 2010-2011 to 2014-2015. The evaluation is not an exhaustive assessment of federal investments in the area of mental health and mental illness.

This was a scheduled evaluation as per the Public Health Agency of Canada/Health Canada approved Five-Year Evaluation Plan 2013-2014 to 2017-2018, in fulfillment of the requirements associated with the Financial Administration Act and the Treasury Board of Canada's Policy on Evaluation (2009).

3.0 Program Description

The terms mental health and mental illness are often used interchangeably and can have different meanings depending on how they are used. For the purpose of this evaluation we will be using the following definitions:

• Mental health is the capacity of each and all of us to feel, think, and act in ways that enhance our ability to enjoy life and deal with challenges we face. It is a positive sense of emotional and spiritual well-being that respects the importance of culture, equity, social justice, interconnections and personal dignity.^{Footnote 3}
• Mental illness is a biological condition that causes alterations in thinking, mood or behaviour (or some combination) associated with significant distress and impaired functioning.^{Footnote 4} Examples of specific mental illnesses include depression, schizophrenia, anxiety disorders, and personality disorders.

3.1 Program Context

This evaluation will examine four specific mental health and mental illness activities of Health Canada and the Public Health Agency of Canada which comprise multiple program areas and are part of a larger landscape of mental health activities. Two of these activities are found within Health Canada and two are from the Public Health Agency of Canada. These will be described in the program profile (See 2.2). The two Health Canada activities are situated within the Strategic Policy Branch, and include funding and support to the Mental Health Commission of Canada (MHCC) and the Mood Disorder Society of Canada (MDSC) for the Canadian Depression Research and Intervention Network (CDRIN). The Public Health Agency of Canada's mental health promotion and suicide prevention activities reside in the Centre for Health Promotion (CHP) within the Health Promotion and Chronic Disease Prevention Branch, and the mental health, mental illness and suicide surveillance activities reside in the Centre for Chronic Disease Prevention (CCDP).

The broader context for mental health and mental illness activities

The mental health and mental illness landscape is complex and includes many players at numerous levels and jurisdictions. There are also a significant number of organizations engaged in mental health or mental illness activities. Many of these are focused on specific illness or specific populations.

This evaluation does not cover the full range of mental health and mental illness activities undertaken by Health Canada and the Public Health Agency of Canada; nor does it address the full range of mental health and/or mental illness programs within the federal government. Although the evaluation only addresses the performance of the previously identified four areas, these do not exist in isolation. It is important to understand the larger context of mental health and mental illness activities in Canada to gain a better appreciation of the complexity of the issues and the inter-connectedness. Figure 1 is a simplified overview of the mental health and mental illness landscape as it relates to the federal Health Portfolio only. The depiction is not intended to be an exhaustive overview of all players in this area.

In addition to direct collaborative relationships between federal level key actors, both the MHCC and the Public Health Agency of Canada have collaborative relationships with other federal government departments (OGDs), the private sector, non-governmental organizations (NGOs), National Aboriginal Organizations (NAOs) and Canadian provinces and territories (P/Ts). Health Canada has a direct funding relationship with MHCC and MDSC.

Federal Departments

Several federal government departments have mandates to deliver mental health programs, services, or supports to specific populations: First Nations on-reserve and Inuit (Health Canada); federally incarcerated individuals (Correctional Services Canada); members of the Canadian Forces (Department of National Defense); veterans (Veterans Affairs Canada); members of the Royal Canadian Mounted Police; immigrants and refugees (Citizenship and Immigration Canada); and, federal public service employees (Health Canada's Employee Assistance Program, Treasury Board Secretariat). Other departments contribute to mental health by adopting a social determinants of health approach, providing supports to parents, seniors, and new Canadians, as well as job re-training and literacy programs.

The Public Health Agency of Canada utilizes a population health approach to address mental health. One of the current priority areas for the Innovation Strategy is mental health. Through the development, implementation and evaluation of innovative population health interventions, the Innovation Strategy provides funding to external organizations to focus on actions addressing the underlying environmental, social, demographic and economic conditions related to these priority areas, including a focus on northern and remote communities, children and youth, and those with low incomes. The evaluation of the Innovation Strategy^{Footnote 5} found indications that the program is contributing to improved health outcomes, particularly in the area of mental health. These outcomes include improved knowledge and skills in children and families, including those in Aboriginal settings.

Community-based health promotion programs generally include a mental health promotion element. Mental health promotion aims to improve positive mental health by increasing psychological well-being, competence, and resilience, and by creating supportive living conditions and environments.^{Footnote 6} The Aboriginal Head Start in Urban and Northern Communities (AHSUNC) program components (e.g., social support, parental involvement and health promotion) promote mental health and the development of protective factors. Programming in the Canadian Prenatal Nutrition Program (CPNP) and the Community Action Program for Children (CAPC) addresses social isolation, and many program sites provide links to direct mental health services for both parents and children.

Within Health Canada, the FNIHB is responsible for providing health services including mental health services to First Nations and Inuit. The Mental Wellness program funds and supports community-based programming and services that aim to reduce risk factors, promote protective factors, and improve health outcomes associated with the mental wellness (mental wellness is the preferred term within FNIHB, as it is a broader, positive term that has been associated with wellness, inherent strengths, and functioning in life) of First Nations and Inuit. Programming provides a range of culturally-relevant mental health and addictions programs and services which are guided by community priorities. The Mental Wellness component includes seven programs under three activity areas. These programs support a continuum of care that includes primary, secondary, and tertiary prevention activities and knowledge development. Other community based programming within FNIHB, such as the Aboriginal Head Start On Reserve Program, also support mental wellness through increasing protective factors.

The Canadian Institutes of Health Research (CIHR) funds research concerning mental health and mental illness. Approximately $50M/year is directed to both mental health studies proposed by researchers as well as strategic research proposed by the CIHR. Some of the current research areas include access to mental health services, the impacts of other diseases on mental health, and suicide prevention.

Provinces and Territories and Organizations

While discussing the federal government's activities in mental health and mental illness, it is important to recognize that the provincial and territorial governments have the primary authority for the planning and delivery of mental health services, treatment, and supports in their respective jurisdictions.

In addition, there are a variety of non-governmental organizations and professional associations that are involved in mental health, mental illness and suicide prevention. These include the Canadian Mental Health Association, the Centre for Addiction and Mental Health, the Canadian Association for Suicide Prevention, Partners for Mental Health, the Canadian Psychological Association, and the Canadian Psychiatric Association.

The Canadian Mental Health Association (CMHA) provides direct service to more than 100,000 Canadians through the combined efforts of more than 10,000 volunteers and staff across Canada in over 120 communities. CMHA branches across Canada provide a wide range of innovative services and supports to people who are experiencing mental illness and their families. These services are tailored to the needs and resources of the communities where they are based. One of the core goals of these services is to help people with mental illness develop the personal tools to lead meaningful and productive lives.

The Centre for Addiction and Mental Health (CAMH) is Canada's largest mental health and addiction teaching hospital, and is one of the leading research centres in the area of addiction and mental health. CAMH combines clinical care, research, education, policy development and health promotion to help transform the lives of people affected by mental health and addiction issues.

The Canadian Association for Suicide Prevention (CASP) provides information and resources to communities to reduce the suicide rate and minimize the harmful consequences of suicidal behaviour. CASP works toward the achievement of its purpose by facilitating, advocating, supporting, and advising, rather than by the provision of direct services.

Partners for Mental Health is a more recently created charity that aims to improve mental health in Canada by mobilizing and engaging Canadians to drive fundamental changes. Partners for Mental Health began as a part of the Mental Health Commission of Canada, but became a separate entity in November 2010. Through campaigns and initiatives, including the workplace based "Not Myself Today", Partners for Mental Health encourages support and implementation of the Mental Health Strategy for Canada and the recommendations contained within.

The Canadian Psychiatric Association is the national voluntary association for Canada's psychiatrists. The Association is engaged in research and the dissemination of knowledge regarding psychiatric disorders and the promotion of mental health. They publish the Canadian Journal of Psychiatry/La Revue canadienne de psychiatrie, a monthly peer-reviewed medical journal.

The Canadian Psychological Association represents psychology in Canada. They advocate for improved mental health care for all Canadians through the inclusion of psychological services into primary care. They also support and promote the science and practice of psychology in the service of its membership and the public good.

By understanding the broader context for the mental health and mental illness activities, it is easier to situate the specific activities that are within the scope of the evaluation.

3.2 Program Profile

The Mental Health and Mental Illness Activities of Health Canada covered by this evaluation are situated within the Strategic Policy Branch, and include the funding, and management of the funding, to the Mental Health Commission of Canada and the Mood Disorder Society of Canada for the Canadian Depression Research and Intervention Network.

Creation of the MHCC was announced in Budget 2007, based on recommendations of the Standing Senate Committee on Social Affairs, Science and Technology^{Footnote 7} found in Out of the Shadows at Last: Transforming Mental Health, Mental Illness and Addiction Services in Canada. The initial ten year mandate (2007-2017) was "to act as a catalyst to improve the mental health system in Canada, develop a mental health strategy for Canada, reduce stigma and discrimination faced by people living with mental illness and mental health problems, and create a knowledge exchange centre."

In Budget 2012, the government provided $5.2 million in funding to MDSC to establish and integrate a network of mental health-research professionals. Research in this network was centered on treating depression, with a focus on suicide prevention and post-traumatic stress disorder. These funds were used to establish CDRIN.

Public Health Agency of Canada's activities covered in this evaluation reside in the Health Promotion and Chronic Disease Prevention Branch. The Centre for Health Promotion is the focal point for mental health promotion and suicide prevention within the Public Health Agency of Canada, and the Mental Health and Mental Illness Surveillance activities are the responsibility of the Centre for Chronic Disease Prevention.

Public Health Agency of Canada's mental health promotion and suicide prevention activities are one aspect of its broader role to enhance the conditions within which healthy development takes place. Mental health was identified as a priority in the Public Health Agency of Canada's Strategic Plan, Strategic Horizons (2013-2018).  In December 2012, An Act respecting a Federal Framework for Suicide Prevention became law, requiring the Government of Canada to enter into consultations with relevant federal departments, provincial, and territorial governments, non-governmental organizations, and national Aboriginal organizations to inform the development of a federal framework for suicide prevention. The Act recognizes that suicide, in addition to being a mental health issue, is a public health issue. The Public Health Agency of Canada, on behalf of the Government of Canada, is leading the development of the Federal Framework for Suicide Prevention.

The Public Health Agency of Canada's work on the development of the Federal Framework for Suicide Prevention was informed by the World Health Organization's Comprehensive Mental Health Action Plan^{Footnote 8}. This action plan includes global targets to enhance the focus on the development of policies and strategies that address the promotion of mental health and the prevention of mental disorders.

Surveillance is a core public health activity. Budget 2013 directed the Public Health Agency of Canada to reallocate $2 million to improve data collection and reporting of mental illness and mental health as a means to improve knowledge and foster collaboration, as recommended in the Mental Health Strategy for Canada. This internal reallocation focussed resources on the development of positive mental health indicators and the expansion/enhancement of mental illness surveillance. The initial report on indicators was released in January 2015. The Integrated Strategy for Healthy Living and Chronic Disease (2005) supports mental illness surveillance activities under its enhanced surveillance for chronic disease component.

The four activity areas, covered by this evaluation, are distinct and do not share a common governance structure.

1. Mental Health Commission of Canada - Health Canada funded

The MHCC is an arm's length pan-Canadian health organization funded by Health Canada to act as a national focal point for mental health issues, as well as a catalyst for improving the mental health system and changing the attitudes of Canadians regarding mental health. The goal of the MHCC is to foster change and innovation in the organization and delivery of mental health services and supports. It is expected that their activities will result in a real difference in the lives of Canadians living with mental illness, their families, caregivers and communities. To this end, the Commission encourages cooperation and collaboration among governments, mental health service providers, employers, the scientific and research communities, as well as Canadians living with mental illness, their families and caregivers.

According to the original funding arrangement the Commission has three core funded activities:

• lead the development of a national mental health strategy;
• launch an Anti-Stigma Campaign, including Public Education and Awareness; and
• create a Knowledge Exchange Centre.

Changing Directions, Changing Lives: The Mental Health Strategy for Canada was developed by the MHCC and released in 2012. The six Strategic Directions are:

• Promote mental health across the lifespan in homes, schools, and workplaces, and prevent mental illness and suicide wherever possible.
• Foster recovery and well-being for people of all ages living with mental health problems and illnesses, and uphold their rights.
• Provide access to the right combination of services, treatments and supports, when and where people need them.
• Reduce disparities in risk factors and access to mental health services, and strengthen the response to the needs of diverse communities and Northerners.
• Work with First Nations, Inuit and Métis to address their distinct mental health needs, acknowledging their unique circumstances, rights and cultures.
• Mobilize leadership, improve knowledge, and foster collaboration at all levels.

Under these strategic directions, there are 26 priorities and 109 specific recommendations. The recommendations and priority areas recognize that multiple sectors and levels of government need to be engaged in addressing mental health issues.

The anti-stigma campaign, Opening Minds, is designed to change the attitudes and behaviours of Canadians towards people living with mental illness. Opening Minds addresses stigma within four main target groups: health care providers, youth, the workforce and the media. As such, the initiative has multiple goals, ranging from improving health care providers' understanding of the needs of people with mental health problems to encouraging youth to talk openly and positively about mental illness. Ultimately, the goal of Opening Minds is to cultivate an environment in which those living with mental illness feel comfortable seeking help, treatment and support on their journey toward recovery.

The Knowledge Exchange Centre (KEC) of the MHCC is engaged in research and research support, across all of the Commission. It ensures public dissemination of research, programs, guidelines and tools. The focus of the KEC is on collaboration and increasing the use of research in mental health community.

Apart from these core activities, the MHCC pursued multiple projects and initiatives that were not explicitly identified in their grant agreements but are related to the recommendations from the Mental Health Strategy for Canada. These included activities related to workplace mental health, such as the Mental Health First Aid training program and the National Standard for Psychological Health and Safety in the Workplace (released in January 2013), as well as emerging priority areas, such as suicide prevention, and the development of guidelines for recovery-focussed practice.

2. Mood Disorders Society of Canada - Health Canada funded

MDSC is a national, not for profit, health charity committed to ensuring that the voices of consumers, family members and caregivers are heard on issues relating to mental health and mental illness; and in particular with regard to depression, bipolar illness and other associated mood disorders.

In Budget 2012, the Government of Canada announced funding to the MDSC for the creation of CDRIN. CDRIN was officially launched on January 29, 2013, with the signing of a contribution agreement between Health Canada and the MDSC.

CDRIN is a collaborative cross-Canada network with the mission to create and share knowledge that leads to more effective prevention, early diagnosis, and treatment of depression and depression-linked illnesses. People participating in CDRIN include researchers, clinicians (of many professions), people with lived experience of depression, families and caregivers, educators, and interested members of the general public.

The MDSC was also funded to develop a national training program for post-traumatic stress disorder (PTSD) to be included as part of continuing medical education for family physicians and specialists. This included curriculum development, and accreditation from the College of Family Physicians of Canada and the Royal College of Physicians and Surgeons of Canada.

3. Mental Health Promotion and Suicide Prevention - Public Health Agency of Canada

The Centre for Health Promotion is the focal point for mental health promotion and suicide prevention within the Public Health Agency of Canada. CHP works closely with governments and stakeholders across Canada to enhance and improve environments, such as schools and workplaces, to support mental wellbeing, prevent mental illness and suicide.  Evidence demonstrates that mental health promotion activities, especially those targeted towards at risk children and youth, in specific settings such as schools that teach them social and emotional skills, can contribute to preventing mental illness and suicide and improving well-being. CHP also works with CCDP, within the Public Health Agency of Canada, to strengthen collection of mental health data, analysis and dissemination.

CHP  is also leading the development of the Federal Framework for Suicide Prevention. The Framework will guide the Government of Canada's efforts in six key areas described within the Act respecting a Federal Framework for Suicide:

• providing guidelines to improve public awareness and knowledge about suicide;
• disseminating information about suicide, including information concerning its prevention;
• making existing statistics about suicide and related risk factors publically available;
• promoting collaboration and knowledge exchange across domains, sectors, regions and jurisdictions;
• defining best practices for the prevention of suicide; and
• promoting the use of research and evidence-based practices for the prevention of suicide.

CHP supports the work on international initiatives in the area of mental health promotion. In particular, the Public Health Agency of Canada represents Canada on the Sponsoring Country Leadership Group of the International Initiative for Mental Health Leadership (IIMHL). The IIMHL is a network of government and non-government organizations in eight countries dedicated to developing leadership in mental health and substance abuse through networking, information sharing and problem solving to advance these issues. At the 2015 Leadership Exchange, hosted by Canada, the Public Health Agency of Canada co-hosted topic meetings on suicide prevention and mental health promotion with other partners, including the MHCC.

As well, during Canada's two-year Chairmanship of the Arctic Council (May 2013 - April 2015), the Public Health Agency of Canada and CIHR were the Canadian co-leads of a suicide prevention research initiative aimed at enabling Arctic communities to learn from other communities on best practices in the promotion of mental wellness and resilience, with a particular focus on youth. This research was supported by Denmark, Norway, the Russian Federation, the United States of America and the Inuit Circumpolar Council. The research indicated that suicide prevention requires culturally-grounded solutions that are community-based and community-driven, as well as strong collaborative partnerships between researchers and Indigenous Peoples.

4. Mental Health/Mental Illness Surveillance- Public Health Agency of Canada

The Mental Health Strategy for Canada identifies the improvement of mental health and mental illness data collection, research and knowledge exchange as one of the six key pillars. In response, the Government of Canada reallocated $2M annually from 2013-14 to 2015-16 to improve data collection and reporting of mental health and mental illness^{Footnote 9}. In response, the CCDP has developed enhanced national data regarding key target populations, and protective and risk factors. This work has been done in collaboration with provinces and territories, other federal departments, academics and relevant NGOs.

The aim of the data collection and reporting is to provide a national picture of the state of mental illness and mental health across the life cycle that can be useful to all jurisdictions to inform opportunities to improve the mental health status of Canadians. There are three main areas that will be covered in this evaluation:

• Positive mental health: A core set of positive mental health indicators for Canada has been developed. These indicators include positive mental health outcomes (e.g., emotional and social well-being) align with the efforts of the Mental Health Promotion Task Group (MHPTG) and consistent with the work from the Mental Health Commission of Canada.
• Mental illness: The Canadian Chronic Disease Surveillance System (CCDSS) has been expanded to better track mental illness morbidity and impact. Impacts of living with mood and anxiety have been analyzed through the Survey on Living with Chronic Disease in Canada (SLCDC). Indicators of prevalence, health service use and mortality for mental illnesses are reported through products such as reports, peer-reviewed papers, the Chronic Disease and Injury Indicator Framework and through online data cubes.
• Suicide-related Surveillance: The CCDP reports on self-inflicted injuries presenting to emergency departments and admitted to hospital, as well as suicides, through a variety of products including the Chronic Disease and Injury Indicator Framework, reports and peer-reviewed articles. Using multiple data sources and comprehensive analysis, surveillance reports on self-inflicted injuries and suicide for all ages are available.

3.3 Program Narrative

This is the first evaluation to examine concurrent activities of Health Canada and the Public Health Agency of Canada that are not part of a horizontal initiative. As such, there was no existing common logic model or outcome statement. The logic model used to guide the evaluation and narrative has been based on information available from the programs' objectives, operational plans, and other corporate documents. This work was then validated by the Mental Health and Mental Illness Evaluation Working Group, comprised of representatives from the four activity areas.

The long term expected outcome for the mental health and mental illness activities of Health Canada and the Public Health Agency of Canada is that evidence-based mental health and mental illness activities are integrated into programs and policies across sectors.

Activity areas and outputs include knowledge products, surveillance systems and data, and networks, collaboration and engagement. The immediate outcomes expected from these activities are increased development of and access to knowledge products, increased awareness of mental health, mental illness, and suicide prevention issues, improved surveillance and monitoring systems for mental health and mental illness and the development of strategic partnerships. It is expected that positive results for these immediate outcomes will increase collaboration and coordination on mental health promotion and mental illness activities and planning, policy and program development that reflects that current knowledge in mental health and mental illness. Policy and programs that adopt evidence-based practices will be better equipped to improve the mental health of Canadians and the lives of those affected by mental illness

The connection between these activity areas and the expected outcomes is depicted in the logic model (Appendix 1). The evaluation will assess the degree to which the defined outcomes have been achieved over the timeframe of the evaluation.

3.4 Program Alignment and Resources

Through funding of the MHCC and the MDSC for CDRIN, Health Canada aims to foster change in the mental health system. This is aligned with Health Canada's work on Health System Priorities (subprogram 1.1.1) which "works closely with provincial and territorial governments, domestic and international organizations, health care providers, and other stakeholders to develop and implement innovative approaches, improve accountability, and responses to meet the health priorities and health services needs of Canadians"^{Footnote 10}. This subprogram is one of two that supports Canadian Health Systems Policy (Program 1.1) which in turn addresses Strategic Outcome 1: A health system responsive to the needs of Canadians.

The Public Health Agency of Canada's mental health and mental illness activities with various sectors and levels of government to address gaps in mental health knowledge and tool development are aligned with the Public Health Agency of Canada's work to develop Healthy Communities (sub-sub program 1.2.2.2.). The work aims to improve community capacity to contribute to better health outcomes for Canada's population. The sub-sub program relates to the Public Health Agency of Canada's broader sub-program addressing the Conditions for Healthy Living (sub-program 1.2.2) which seeks to "improve health outcomes for Canada's population throughout their lives by promoting positive mental, social, and physical development and by enabling the development of healthy communities"^{Footnote 11}.  Mental health, mental illness and suicide surveillance align with sub-program on Chronic (non-communicable) Disease and Injury Prevention (sub-program 1.2.3) which aims to improve the use of evidence in chronic disease policy and program decisions. The sub-program works to track injuries, chronic diseases, their risk factors and related inequalities, and analyses the risks to public health, and determines priorities for action. All of the work regarding mental health, mental illness and suicide prevention align with Program 1.2: Health Promotion and Disease Prevention and the strategic outcome of protecting Canadians and empowering them to improve their health.

The financial information for the years 2010 through 2015 was collected and verified by the program areas and the Offices of the Chief Financial Officer. Table 1 presents the actual spending for Health Canada's and Public Health Agency of Canada's mental health and mental illness activities for the years 2010-2011 to 2014-2015. Areas of activity have been grouped according to the component of the health portfolio that provided the funding.

4.0 Evaluation Description

4.1 Evaluation Scope, Approach and Design

The scope of the evaluation covered the period of April 2010 to March 2015, and focuses only on the examination of the Health Canada grants to MHCC and MDSC, and the Public Health Agency of Canada's mental health promotion and suicide prevention activities, and mental health and illness related surveillance systems. Data collection for this evaluation was completed in July 2015.

The mental wellness programming in the First Nations and Inuit Heath Branch of Health Canada was not considered for this evaluation. The following activities by the Public Health Agency of Canada were also considered out of scope for this evaluation: investments in mental health promotion through the Innovation Strategy, child and parent mental health activities funded through CAPC, CPNP and AHSUNC, child maltreatment surveillance, and neurological conditions activities. These activities and programs have been addressed in other evaluations.

The evaluation issues were aligned with the Treasury Board of Canada's Policy on Evaluation (2009) and considered the five core issues under the two themes of relevance and performance, as shown in Appendix 3. Corresponding to each of the core issues, specific questions were developed based on program considerations and these guided the evaluation process.

This evaluation used an outcome-based approach to assess the progress made toward the achievement of the expected outcomes, whether there were any unintended consequences and what lessons were learned.

The Policy on Evaluation (2009) guided the identification of the evaluation design and data collection methods so that the evaluation would meet the objectives and requirements of the policy. A non-experimental design was used based on the evaluation matrix, which outlined the evaluation strategy for these activities.

Data collection activities were carried out between January and July 2015. Information for the evaluation was collected using various methods, including: literature review, document and file review, financial data review, international review, stakeholder survey and interviews with internal and external key informants. More specific details on the data collection and analysis methods are provided in Appendix 3. Data were analyzed by triangulating information gathered from the different methods listed above. The use of multiple lines of evidence and triangulation were designed to increase the reliability and credibility of the evaluation findings and conclusions.

4.2 Limitations and Mitigation Strategies

Most evaluations face constraints that may have implications for the validity and reliability of evaluation findings and conclusions. Table 2 outlines the limitations encountered during the implementation of the selected methods for this evaluation. Also noted are the mitigation strategies put in place to ensure that the evaluation findings can be used with confidence to guide program planning and decision making.

5.0 Findings

This section provides a summary of the findings organized under two broad headings:

• Relevance: the need, priorities, and federal health role in mental health. This is based on the mandate of the program areas at the time of the evaluation.
• Performance: the effectiveness, efficiency and economy of the four program areas under this evaluation

5.1 Relevance: Issue #1 - Continued Need for the Program

Canadians continue to be affected directly and indirectly by poor mental health and mental illness. There is a continued need to promote mental health and reduce the negative impact of mental illness including addressing stigma.

A significant number of Canadians are affected directly and indirectly by poor mental health and mental illness. Every year, one in ten Canadians will experience a mental illness, and one in three Canadians will experience a mental illness or substance use disorder sometime in their lives.^{Footnote 12} Of the close to 4,000 Canadians who die every year as a result of suicide, most were confronting a mental illness.^{Footnote 13} In 2012, 40% of adults in Canada had a family member with drug, alcohol, emotional or mental health problems.^{Footnote 14}

The projected healthcare and economic burden of mental illness in Canada is expected to increase. As a result of the aging of the Canadian population, it is anticipated that the number of older Canadians living with mental illness will rise. Rates of mental illness for adults between the ages of 70 and 89  are projected to be higher than for any other age group by 2041.^{Footnote 15}

Mental health and mental illness have a direct impact on the Canadian economy, both in terms of lost productivity and health care costs. The Public Health Agency of Canada's Economic Burden of Illness Tool estimated that in 2008, the direct cost of mental illness in Canada was close to $8 billion. Depending on the expenditures included, indirect costs related to mental illness in Canada have been shown to range from $11 billion to $50 billion.^{Footnote 16} A 2014 Organisation for Economic Co-operation and Development (OECD) report on mental health estimated that direct and indirect costs of mental ill health can exceed as much as 4% of gross domestic product in OECD countries.^{Footnote 17} Mental health and mental illnesses have significant impacts on individuals, families and communities, including workplaces. Mental health problems and illnesses have been shown to account for approximately 30 per cent of short- and long-term disability claims.^{Footnote 18} Lost productivity related to poor mental health and mental illness  (such as absenteeism, presenteeism and turnover) was estimated to cost business in Canada more than $6 billion in 2011.^{Footnote 19}

Evidence suggests that certain populations are impacted by poor mental health and mental illness more often. Vulnerable and at-risk populations (e.g., women, adolescents, seniors, Aboriginal peoples, the homeless, those living with chronic diseases, and people identifying as lesbian, gay, bisexual, transgender, Two Spirited or queer/questioning (LGBTQ)) have higher rates of mental health problems and mental illnesses compared to the rest of Canadians.  A number of factors, including social determinants of health, impact the higher rates of mental health problems and illnesses.

Different segments of the population often experience different patterns of mental health and mental illness. For example, women are twice as likely as men to develop depression in their lifetime.^{Footnote 20} Whereas males account for 75% of deaths by suicide, females attempt suicide 3 or 4 times more often.^{Footnote 21}  The CMHA estimates that between 10-20% of Canadian youth are affected by a mental illness or disorder.^{Footnote 22} Suicide is the second leading cause of death in Canadians between the ages of 10 and 24.^{Footnote 23} First Nations youth die by suicide about five to six times more often than non-Aboriginal youth. The suicide rates for Inuit are among the highest in the world, at 11 times Canada's national average, and for young Inuit men the rates are 28 times higher.^{Footnote 24} Middle-aged men (between the ages of 40 to 59) have the highest suicide rate of any age group and men account for the vast majority (81%) of suicide deaths among seniors.^{Footnote 25}

There is a strong association between chronic diseases and mental illnesses. It is recognized that people with a mental illness are more likely to have a co-morbid chronic disease or condition.^{Footnote 26} As well, people with chronic diseases are more likely to experience depression and anxiety.

Stigma related to mental health and mental illness can marginalize those living with mental illness and their family members. As outlined by the MHCC's Opening Minds, stigma, grounded in stereotypes and prejudicial behaviours, acts as a barrier, reducing help-seeking behaviour, which in turn limits ability to recover. Almost 40% of Canadians living with mental illness indicate that they have experienced discrimination or unfair treatment.^{Footnote 27}

The mental health and mental illness landscape in Canada, characterized by Out of the Shadows at Last: Transforming Mental Health, Mental Illness and Addiction Services in Canada^{Footnote 28} as a "fragmented patchwork of programs and services", remains a mixture of service delivery, treatment and policy initiatives, to which a wide variety of stakeholders contribute.^{Footnote 29} The mental health and mental illness stakeholder community is composed of a large number of actors, who engage across a broad range of issues, vary in scope from the community-level to a Canada-wide focus, and represent specific population groups (e.g., seniors' mental health) as well as specific mental health issues (e.g., mood and anxiety disorders).  The fragmented reality of services benefits from a national approach.

5.2 Relevance: Issue #2 - Alignment with Government Priorities

The Public Health Agency of Canada's and Health Canada's mental health and mental illness activities align with current Government of Canada priorities and international agreements. Priorities focus on strengthening the Canadian mental health system through national leadership and strong partnership to promote positive mental health, and prevent mental illness and suicide.

Mental health was underscored as a federal government priority with the publication of Out of the Shadows at Last: Transforming Mental Health, Mental Illness and Addiction Services in Canada^{Footnote 30} by the Senate Standing Committee on Social Affairs, Science and Technology, and the subsequent federal funding to launch the Mental Health Commission of Canada in 2007. In April 2015, the federal government announced its intent to support the MHCC's renewal for a second ten-year mandate, to start in 2017, through Economic Action Plan 2015. Additionally, the Act respecting a Federal Framework for Suicide became law in 2012, requiring the Government to create a federal framework for suicide prevention in consultation with federal partners and stakeholders. Budget 2012 committed $5.2 million over 2012-13 to support CDRIN and a continuing medical education training course on Post-Traumatic Stress Disorder. Budget 2013 included an internal reallocation in the Public Health Agency of Canada of $2 million over three years (2013-14 to 2015-16) to improve data collection and reporting of mental health and mental illness.

Health Canada and the Public Health Agency of Canada have identified mental health as a priority in several key planning documents. Health Canada's 2014-15 Report on Plans and Priorities identifies "Health System Priorities" as a priority, aiming to ensure that Canadians have access to quality and cost-effective health care services through the provision of national leadership and strong partnerships contributing to health system improvements, including in the mental health system. The Public Health Agency of Canada's five-year strategic plan (Strategic Horizons 2013-18) made "enhancing collaborative efforts on the promotion of positive mental health, the prevention of mental illness and suicide" a key priority.  A key Public Health Agency of Canada activity highlighted in the 2014-15 Report on Plans and Priorities is "Strengthening the evidence base for effective mental health promotion and suicide prevention through developing tools and resources for health professionals and the development of a Federal Framework on Suicide Prevention".

A number of activities at Health Canada and Public Health Agency of Canada align with these mental health priorities. From the winter of 2013 to the spring of 2015, the  Public Health Agency of Canada has led consultations with 376 organizations and individuals regarding the Federal Framework for Suicide Prevention, and theFrameworkdevelopment is underway.  In response to the Mental Health Strategy recommendation (Priority 6.2 of the National Strategy calls for improved mental health data collection, research and knowledge exchange across Canada) and Budget 2013, the Public Health Agency of Canada addressed gaps in current data collection by expanding existing surveillance data platforms and participating in international research activities, such as the Arctic Council initiative to explore best practices in preventing suicide in circumpolar communities by promoting mental wellness, hope and resilience.

Several of Health Canada's current activities align with its priority to contribute to health system improvements by providing national leadership and strong partnership, as evidenced by its investment in MDSC to establish CDRIN. CDRIN launched the Depression Hubs in March of 2014 to advance research on depression and PTSD. Since 2007, Health Canada has provided continued support for the MHCC and its priorities of fostering change in the mental health system, reducing the stigma associated with mental illness, combatting suicide, supporting mental health education and awareness, and promoting the Mental Health Strategy for Canada.

In May 2013, members of the World Health Assembly, including Canada, formalized mental health as a priority by adopting the World Health Organization's (WHO) Comprehensive Mental Health Action Plan 2013-2020. The Plan's objectives focus on effective leadership, integrated services, mental health promotion and prevention strategies, and strengthened research and information systems. In announcing the launch of the Mental Health Action Plan, the Director General of the WHO reiterated that there is "no health without mental health".^{Footnote 31}

The recent United Nations (UN) Sustainable Development Goals^{Footnote 32} include the promotion of mental health and well-being as a sub-goal under the overall goal to ensure healthy lives and promote well-being for all.

The federal government also has international obligations related to mental health under the United Nations Convention on the Rights of Persons with Disabilities (UNCRPD), which requires parties to raise awareness about the rights of people with disabilities, foster respect, combat stereotypes, prejudices and harmful practices. According to the UNCRPD , persons with disabilities include those who have long-term physical, mental, intellectual or sensory impairments which in interaction with various barriers may hinder their full and effective participation in society on an equal basis with others.^{Footnote 33} Programming undertaken by the MHCC's Opening Minds has helped Canada in meeting its obligations under the UNCRPD.

5.3 Relevance: Issue #3 - Alignment with Federal Roles and Responsibilities

The federal government has a clear role in coordinating and collaborating on mental health and mental illness activities, in conducting and reporting on mental health, mental illness and suicide surveillance and research, and in being a resource for information on best practices and innovation. Activities of Health Canada and the Public Health Agency of Canada are clearly aligned with the federal roles in mental health and mental illness.

The Department of Health Act provides that the Minister of Health's powers, duties and functions relating to health include the promotion and preservation of the physical, mental and social well-being of the people of Canada.. The Public Health Agency of Canada's role in public health promotion and prevention of public health risks and health surveillance is outlined in the Public Health Agency Act. The Public Health Agency of Canada's more specific roles related to surveillance for chronic diseases and risk factors, including for positive mental health and mental illness surveillance, are presented under the Integrated Strategy for Healthy Living and Chronic Disease (ISHLCD) authorities.

In Canada, public health is a shared responsibility between federal, provincial and territorial governments. While the administration and delivery of health care services is the responsibility of each province and territory, the federal government has a role for the delivery of health services, including mental health care, for federal populations (i.e., the federally incarcerated, serving members of the Canadian Forces, members of the Royal Canadian Mounted Police (RCMP), Veterans, refugees and immigrants, and First Nations and Inuit).

The Public Health Agency of Canada coordinates activities focused on health promotion, illness prevention, and control of diseases, public health emergency response and support of intergovernmental collaboration. The federal government's role as coordinator of intergovernmental action in the area of mental health is implemented by Public Health Agency of Canada's mental health promotion program, which coordinates federal/provincial/territorial action on mental health promotion, including working with stakeholders on policy and program design to promote positive mental health. Key informants flagged the benefits of the federal coordination role across jurisdictions, noting the importance of creating a national dialogue on mental health and the federal government's role in ensuring continuity in the delivery of mental health services for federal client groups

Key informants suggested that there is a continued need for the federal government to undertake policy leadership and act as a catalyst in the development of a strategic direction on mental health. Other appropriate federal roles noted by key informants include: acting as a knowledge broker, and leading by example as an employer to support workplace mental health.

Key informants noted that the federal role was not always clear. Those outside of the federal government were unclear on the distinction between the responsibilities of the federal departments.   A majority of the federal key informants indicated that while they were clear on the role of their own organization with regards to mental health and mental illness, they often were unsure of the roles and responsibilities of other departments.   This has created challenges in obtaining information or finding the appropriate contacts. Informants reported that the inconsistent use of language and a lack of consistent definitions regarding mental health and mental illness has at times impacted on communication both internally and externally.

5.4 Performance: Issue #4 - Achievement of Expected Outcomes (Effectiveness)

The activities and performance of the four activity areas will be presented separately. The relationships and collaboration between the program areas will also be explored.

5.4.1 To what extent has the Mental Health Commission of Canada achieved its outcomes?

The Mental Health Commission of Canada succeeded in completing the three tasks set out in its 2007 mandate: the development of a national mental health strategy, the development of an anti-stigma campaign, and the creation of a Knowledge Exchange Centre.

Uptake and Use of the Mental Health Strategy for Canada

The Mental Health Strategy for Canada and the preceding framework have been identified as important reference points for organizations and governments in shaping their own mental health strategies and programs.

Work has been undertaken across the country to respond to the strategic directions and priority areas identified in Changing Directions, Changing Lives: The Mental Health Strategy for Canada. Some recommendations have been targeted for action more than others.  

Changing Directions, Changing Lives: The Mental Health Strategy for Canada was released on May 8, 2012. It provides strategic direction to governments and other stakeholders to improve Canada's mental health care system, leading to improved mental health outcomes. Prior to the release of the Strategy, a framework document was released in 2009. Both of these documents were developed through extensive consultations with government representatives and other stakeholders. Overall, the Strategy outlines 26 priorities and 109 recommendations under six strategic directions.

To ensure uptake of the Strategy, it needs to be accessible to those to whom  it is targeted to and others who could benefit from it. The Strategy is available in hard copies and on the MHCC website. A complete version of the Strategy is available in both official languages. The Strategy was circulated across Canada; 6,828 full copies (5966 in English, 862 in French) and 6,650 summaries (4,575 in English, 2,075 in French) were distributed over a two-year period ending June 2014. The Strategy is the most accessed document on the MHCC website. It was accessed or viewed over 40,000 times over the same two-year period. A summary version is available in both official languages, as well as Woods Cree, Plains Cree, Inuktitut and Inuinnaqtun. To increase accessibility, an adapted version entitled The Mental Health Strategy for Canada: A youth perspective was released in May 2015. This version includes examples of best practices and evidence of the Strategy in action. Use by the research community is another important element of uptake. The Mental Health Strategy has been referenced over 20 times in different Canadian and international scientific journals since its release.

Interviews with key informants confirmed that there has been interest in the Strategy.  Most key informants were aware of the Strategy. Those involved in policy development or mental health directly were more aware than those at the community level, or those who only had mental health as a component of their responsibilities. Those who were aware of the Strategy described it as accessible and relevant. Some key informants reported that they referred to the Strategy very often and the majority were able to see how their work aligned with the strategic directions and even specific recommendations.

All eight of the provincial and territorial mental health strategies that have been developed since 2009 reference either the Strategy or the previous framework. The priority areas within the provincial and territorial strategies align with those of the Mental Health Strategy for Canada. Provincial and territorial strategies that align with the Strategy speak to the uptake and impact of the Strategy. These strategies are also a response to recommendation 6.1.1 under Strategic Direction 6 "Mobilize leadership, improve knowledge and foster collaboration". Nine out of 12 strategies (the Yukon does not have a territorial strategy) include, or reference, addictions as part of their strategies. The separation of mental health and addictions was raised as an issue by some provincial and territorial representatives, particularly if their jurisdiction closely aligns mental health and addictions. The MHCC has recognized the need to work more closely with the Canadian Centre on Substance Abuse, and this was identified by MHCC staff as an area for future activities.

Provinces and territories have the jurisdictional responsibility for mental health services; there are also federal departments with responsibilities to certain federal populations. This includes inmates in federal penitentiaries. Corrections Canada, in collaboration with the MHCC and Provincial and Territorial Corrections departments, released the Mental Health Strategy for Corrections in Canada. This Strategy is consistent with the goals and principles outlined in Toward Recovery & Well Being: A Framework for a Mental Health Strategy for Canada. Other departments are looking to work with the MHCC to guide, develop or update frameworks and/or strategies addressing the mental health of their populations.

The majority of recommendations within the Mental Health Strategy for Canada fall outside the scope of the federal government, and in particular outside the mandate of Health Canada or the Public Health Agency of Canada. The Strategy was designed to be a national strategy not a federal strategy. As such, other jurisdictions, departments and NGOs have become involved in the fulfillment of the recommendations. Overall, the Strategy has assisted in advancing key priorities related to mental health and mental illness. However, as illustrated in Table 3, the majority of actions to date have centered on strategic directions 1 (promotion and prevention), 3 (access to services) and 6 (leadership and collaboration). The uptake and implementation of the Strategy is voluntary and is not monitored. Table 3 is based on information gathered during the evaluation, and may not represent all actions undertaken in support of the Mental Health Strategy for Canada. The information available was insufficient to determine the impact of the initiatives, or if they were fully addressing the recommendations.

Key informant interviews revealed concern regarding the broad nature of the Strategy and the absence of clear responsible actors for the implementation of recommendations. Both positive and negative aspects of the absence of areas of responsibility were noted.  Some informants reported it allows jurisdictions freedom to determine what is of most importance to their residents and then act accordingly. Others saw the vagueness as an issue as only "low hanging fruit" was being selected for action and that there was no lead agency identified to coordinate efforts.

Key informants raised concerns that the Strategy did not have an implementation plan and that the federal government had limited levers to implement it. The MHCC does not have resources to provide to jurisdictions to implement particular recommendations, and neither do Health Canada or the Public Health Agency of Canada. This was seen by many stakeholders as an impediment to the full implementation of the Strategy.

That being said, the MHCC has focused its efforts on the implementation of the Strategy in a few key areas. These include workplace health, recovery guidelines and the development of indicators. The MHCC supports and delivers the Mental Health First Aid (MHFA) in Canada program in response to Recommendation 1.1.3 (Increase people's understanding of how to recognize mental health problems and illnesses, how to get support if they need it, and how to get help for someone else). This program, adapted from the MHFA programs in Australia and Scotland, is a cost-recovery initiative of the MHCC, as such, its impact and effectiveness are outside the scope of this current evaluation. A specific Northern MHFA has been developed, and modifications for First Nations, Inuit, seniors and veterans and their families are under development.

Creating mentally healthy workplaces is a priority area (1.3) from the Mental Health Strategy for Canada. In 2013, the National Standard of Canada for Psychological Health and Safety in the Workplace (the Standard) was released. The Standard was developed by the Canadian Standards Association and the Bureau de normalisation du Québec, and championed by the MHCC. Health Canada and the Public Health Agency of Canada representatives were part of the technical committee that developed the Standard, along with other private and public sector partners. The Standard is a voluntary set of guidelines, tools and resources focused on promoting employees' psychological health and preventing psychological harm due to workplace factors. It was developed in the context of a large body of scientific literature from areas of workplace health and safety, law, and social science. As Canada's largest employer, the Treasury Board Secretariat through a Joint Task Force with the Public Service Alliance of Canada is exploring the adoption of this Standard in federal workplaces. Some federal departments (e.g., the RCMP and the Canadian Security Intelligence Service), provincial governments (e.g., Manitoba, Nova Scotia), and private sector firms (e.g., Bell, Enbridge, Great West Life) are participating in a case study on the implementation of the Standard. Other organizations and departments (including Health Canada and the Public Health Agency of Canada) are implementing the Standard but are not participating in the case study. The case study research project will end in January 2017 and identify promising practices, challenges, barriers, costs and impact associated with implementation. As the Standard is voluntary there is no mechanism in place for tracking its broader implementation.

Other actions taken by the MHCC to support priority 1.3 include the development of two programs:  The Working Mind and The Aspiring Workforce. Partners for Mental Health is also engaged in workplace mental health, through its "Not Myself Today" campaign.

An increased focus on mentally healthy workplaces is intended to  lead to a mentally healthier workforce and decrease the number and/or duration of disability claims related to mental health conditions and increase productivity.

Anti-Stigma Campaign

The Opening Minds Anti-Stigma Campaign has demonstrated progress in addressing stigma within four targeted populations (youth, health care professionals, the workforce and the media).

The MHCC defines stigma as "negative attitudes and negative behaviours they produce"^{Footnote 35}. Stigma spreads fear and misinformation, labels individuals, and perpetuates stereotypes. More than 60% of people with mental health problems and mental illness will not seek the help they need; stigma is one of the main reasons.^{Footnote 36}

The Opening Minds Anti-Stigma Campaign targeted stigma held by youth, health care professionals, the workforce and the media. Existing programs and models for anti-stigma in these populations were reviewed, common success factors were identified and toolkits and resources were developed for larger scale application.

In the initial round of 78 research and behavioural change projects there were over 100 partners. The projects were located in 8 provinces and one territory. Projects that were successful in reducing stigma are intended to be replicated at a larger scale. Reduction in stigma was measured through social acceptance scales, administered pre- and post-intervention. Key elements of successful anti-stigma projects included contact-based sessions with persons with lived experience, repetition of messaging through multiple sources and peer teachings.

In November 2014, MHCC launched HEADSTRONG, a national youth anti-stigma campaign.  In attendance at the national summit were 132 students from all provinces and territories. These attendees have since organized 28 regional summits. These regional summits were attended by over 5000 student champions. The student champions were to take back anti-stigma messages and activities to their individual school populations. It is estimated that the HEADSTRONG messaging will reach 100,000 Canadian students by the end of 2015. Actual numbers are not yet available.

Perceived stigma from healthcare providers is a barrier to individuals with poor mental health and mental illness  seeking help, but there are limited anti-stigma projects targeting this problem. However, the Opening Minds campaign was able to identify 13 programs that had led to improvements in health care providers' behaviours. These programs often incorporated an element of skills training, included an incentive for participation, such as continuing education credits, or were specific to a certain illness. Successful programs were also incorporated into the university curriculum for occupational therapists (University of Alberta), psychiatric nurses (Brandon University), pharmacists (Dalhousie University, Memorial University, University of Saskatchewan) and doctors (University of Calgary). The work of Opening Minds has influenced the anti-stigma materials produced for healthcare providers by the Canadian Coalition for Seniors' Mental Health.

The assessment of workplace based anti-stigma campaigns is still in an early stage but the MHCC has been able to identify some elements required for successful initiatives. One of the most important elements is senior executive endorsement of any anti-stigma initiatives. This is aided by the recognition of the potential impact of mental illness and psychological health and safety in the workplace. Stigma reduction is an aspect of the National Standard of Canada for Psychological Health and Safety in the Workplace.

Media anti-stigma initiatives have been different than those for the other target populations. Opening Minds worked with Canadian media to better inform journalists about mental health issues and avoid stigmatizing language and assumptions that can perpetuate misconceptions. A media resource guide Mindset: Reporting on Mental Health was released in April 2014, through a contract with the Canadian Journalism Forum on Violence and Trauma. CBC is the main media partner and has been instrumental in fostering acceptance by journalists. The Mindset project is directed toward general-assignment reporters and editors because their stories have significant influence on public perception. The guide and associated website provide practical tips and resources for those in the media. CBC, The Globe and Mail, Radio-Canada and the CNW Group (Canadian-based commercial press release service) are all supportive of the resource and have contributed to its development and promotion. Further, Opening Minds is conducting a media monitoring study that will examine news articles until the end of 2015. In analysing over 11, 000 French and English newspaper articles from 2005 to 2011, researchers from McGill University found that 40% of Canadian newspaper articles negatively associated crime, violence, and danger with mental illness, while treatment, recovery or rehabilitation was only discussed 18-19% of the time. Examples from this article review are highlighted in the Mindset resource to identify overt and implied stigmatization. With the support of Canada's respected journalists behind it, the Mindset guide aims to change the way mental health and mental illnesses are reported in the media, and thereby decrease stigma experienced by those with mental health issues.

The Opening Minds campaign has been recognized as an international best practice. In 2015, MHCC received the Innovator Award from the World Psychiatric Association at the Together Against Stigma international conference. The MHCC also founded the Global Alliance Against Stigma in 2012, and participates in knowledge exchange activities and best practice sharing with other member countries, including the United Kingdom, Sweden and Australia.

Many of the stakeholders interviewed reported that the largest and most important contribution of the MHCC has been the change in tone around mental health and an increased acceptability for public discussion. Key informants stated that the MHCC has led the way in anti-stigma campaigns, and has provided opportunities for other campaigns to flourish. Stakeholders pointed out that it is unlikely that any national anti-stigma campaign would have existed prior to the creation of the MHCC. While this is not a causal relationship, stakeholders felt that the MHCC has created the climate of change that has advanced the awareness of mental health issues, including stigma.

In addition to the MHCC campaigns that have focused on four specific groups, there have been several other Canadian initiatives to address the stigma that surrounds mental illness. One of the most prominent is the Bell Let's Talk campaign launched in September 2010 by Bell Canada. This campaign focusses on talking openly and without fear about mental illness and mental health. Additionally, Bell Let's Talk provides funding grants to hospitals, universities and community organizations to support mental health initiatives. Since its inception, Canadians' have reported changes in awareness and attitudes toward mental health. According to a poll conducted in September 2015 by Neilsen Consumer Insights for Bell Let's Talk, 57% of respondents reported that stigma surrounding mental health issues has been reduced since 2010, and 81% reported that they were more aware of mental health issues^{Footnote 37}. Although these results cannot be attributed to any specific campaign considering the number of  initiatives and activities over this time period, they speak to the positive outcomes resulting from the combined efforts of all stakeholders working to reduce stigma around mental health issues.

Another element of stigma awareness is the ability to collect population level information about stigma faced by people with mental health issues. The MHCC with Statistics Canada developed a stigma module that was included in a sample of the Canadian Community Health Survey (CCHS) in 2010. The module was then repeated in the full CCHS in 2012. The availability of this data in 2016 will act as a baseline and facilitate the monitoring and evaluation of targeted anti-stigma campaigns. These actions to measure stigma also fulfill Canada's monitoring obligations under the UNCRPD.

Knowledge Exchange Centre

The MHCC Knowledge Exchange Centre has developed and shared key research material related to the Strategic Directions outlined in the Mental Health Strategy for Canada. This has resulted in increased access to mental health and mental illness knowledge products.

The key objectives of the KEC are to facilitate the development and mobilization of evidence-informed knowledge in the mental health community and leverage existing best and promising practices across the country. MHCC initiates research both in-house and through contracts with partners. Partners are often used to disseminate the research more broadly. The KEC is involved in various networks across Canada, aiding in inter-organizational collaboration and sharing of best and promising practices. The KEC also convenes both the Advisory Council and the Youth Council of the MHCC.

The KEC supports the initiatives of the MHCC, including the Standard. All MHCC projects incorporate a knowledge exchange plan, including social media elements. To support the knowledge exchange capacity of others in the mental health community, the KEC has developed a training program SPARK (Supporting the Promotion of Activated Research and Knowledge) to help participants move evidence-informed research and knowledge into practice. This program has been held three times within Canada, and has been replicated in both Sweden and New Zealand. Program participants were supported by mentors in the creation and implementation of knowledge exchange work plans.

In a survey conducted on knowledge products and engagement, 46% of respondents (n=25) rated the MHCC as one of their top three sources of information. The highest rated source was academic journals (83%). Many stakeholders also identified that the MHCC provided a good source of information on a broad range of topics. Others felt that the MHCC brand on research allowed it to gain a wider audience and potentially increased the uptake of information.

Some recent initiatives of the KEC have focused on the development of guidelines for family and caregivers, as well as the support for a multicultural mental health resource centre. These respond to recommendation 2.1.6 and 4.2.2 from the Mental Health Strategy for Canada.

The KEC has undertaken a Data Project that released an initial set of Mental Health Indicators for Canada in early 2015. This responded to priority 6.2 of the Mental Health Strategy for Canada to "improve mental health data collection, research and knowledge exchange across Canada", and in particular recommendation 6.2.1 "Gather and report to the public on data from the initial set of indicators for the Strategy while developing a framework for gathering and reporting on comprehensive data on outcomes over the longer term". The initial 13 indicators were released in January 2015 as the report "Informing the Future: Mental Health Indicators for Canada" and in their first three months were viewed close to 5000 times on the MHCC website. An additional 42 indicators were published in the spring of 2015. The initial set of indicators was released the same week as the Public Health Agency of Canada's releases of an infographic related to the measurement of positive mental health. Further information on the Public Health Agency of Canada's indicator work will be discussed in Section 4.4.3 of this evaluation. There was some confusion by stakeholders on the overlap of these initiatives, particularly since surveillance is a main role of the Public Health Agency of Canada. However, staff from both the MHCC and the Public Health Agency of Canada did not see this as a concern, as the two organizations have differing mandate. For example, the MHCC examined system performance which is not the responsibility of the Public Health Agency of Canada.

Over half of the survey respondents (n=25) were aware of "Informing the Future: Mental Health Indicators for Canada". Some concerns around the appropriateness of specific indicators were raised during interviews. However, the Data Project was seen as useful in identifying existing gaps in surveillance, particularly with regards to children and youth, and Aboriginal Canadians. An additional eight indicators related to the mental health of First Nations, Inuit, and Métis will be released at a later date.

5.4.2 To what extent have the outcomes from the Canadian Depression Research and Intervention Network of the MDSC been achieved?

The Canadian Depression Research and Intervention Network has made progress in developing research hubs across the country, and has worked to include the experiences of people with lived experience into the research process.

The intention of CDRIN was to develop new approaches to prevention and treatment of depression and PTSD by building new connections and cohesion among Canada's mental health research community and the vast array of service providers and persons living with these illnesses.

CDRIN established a series of Depression Hubs covering the Maritimes, Central Canada and Alberta, British Columbia, Ontario and Quebec. The Network continues to develop and has most recently added an Aboriginal hub to address the needs of this population. Knowledge and research products and training have been offered at the annual CDRIN conference, and through regional training sessions directed at researchers and those with lived experience. CDRIN's annual national conference highlights its ability to develop formal partnerships and information sharing mechanisms. Several activities taking place at the regional hub level were highlighted at the 2015 conference. In particular, the Ontario Depression Network has presented on their knowledge translation activities including: patient and family advisory meetings, public education events, academic presentations, workshops and website development.

The development of CDRIN's sustainability plan was not finalised at the time of this evaluation; however, much work had been accomplished resulting in a draft sustainability plan that was completed and shared with the hubs for consideration.

Considering CDRIN's early stage of development, it has made progress in developing on-going and sustained coordinated research efforts on depression and PTSD. The Depression Hubs National Advisory Panel has identified five research priorities for future work. These include:

• Develop and implement a standardized client-based clinical data collection and providing mental health support/portal via smart phone apps;
• Big Data: building over-arching data capture capacity to integrate regional data into a large database;
• Develop a better understanding of Depression-associated co-morbid illnesses (e.g., heart disease and diabetes);
• Pathways to Care: develop strategies to assist clients, families and care-providers with tools to navigate access to appropriate care in a timely fashion; and
• Suicide prevention research.

Although much of CDRIN's work to improve mental health research, tools and strategies to prevent/reduce, diagnose and treat depression and PTSD is ongoing, one aspect of this work (i.e., the inclusion of persons with lived experience into the research process) has been successfully implemented. All of the key informants involved with CDRIN commended the MDSC on its inclusivity. Both researchers and people with lived experiences felt that these collaborations were mutually beneficial.

There is limited performance information available to determine the degree to which CDRIN was able to strengthen collaborative networks and information sharing among stakeholders. Qualitative evidence from interviewees and conference survey respondents showed that CDRIN has contributed to enhancing collaboration and coordination on mental health research.

5.4.3 To what extent have the outcomes from the Public Health Agency of Canada been achieved?

Evidence of enhanced mental health and mental illness surveillance systems

The recent work on positive mental health surveillance by the Public Health Agency of Canada and supported by the Mental Health and Addictions Information Collaborative has filled a data gap and complements the expanded surveillance of mental illness and suicide.

Since 2013, the Public Health Agency of Canada's surveillance activities in the area of mental health and mental illness have expanded to include positive mental health. This required the development of an indicator framework and data sets, based on a common understanding of the core underlying principles. This work was led by the Public Health Agency of Canada and supported by the Mental Health and Addictions Information Collaborative. The Public Health Agency of Canada consulted with the Public Health Network's Mental Health Promotion Task Group (MHPTG) of the Healthy People and Communities Steering Committee (HPCSC). The Public Health Agency of Canada's work on the indicator framework has been considered ground breaking at the international level. A process paper describing the development of the indicator framework and measures will be published in Health Promotion and Chronic Disease Prevention Canada: Research, Policy and Practice in 2016. The framework spans individual, family, society and structural levels with the initial focus on adult measures due to data availability.

The Positive Mental Health Infographic was released in January 2015. In a separate survey tied to the release of the infographic, over 75% of respondents (n=70) agreed that the infographic conveyed the information effectively. As well, two-thirds of respondents confirmed that the infographic was relevant to their work. Respondents indicated that the visual presentation was helpful as a teaching resource, and served as a reminder of the importance of positive mental health. Interviewees who were aware of the positive mental health infographic were pleased with the ease in which information was conveyed. The inclusion of statistics related to the low risk drinking guidelines was seen as a positive.

External stakeholders expressed some concern about overlap between several indicators on the infographic that were also part of the MHCC Indicators document. The same data sources were used, but were reported on slightly differently. This could cause some confusion. The release of the positive mental health infographic was not coordinated with the release of the MHCC's Indicators project. Both the Public Health Agency of Canada and MHCC were aware of the other projects but the release dates were not communicated or coordinated. MHCC did later share the Public Health Agency of Canada's infographic through its social media channels and thereby increased the reach of the information. MHCC product releases tend to be more involved than those for the Public Health Agency of Canada products. MHCC hosted a press conference at the Economic Club for the release of the initial set of indicators, whereas there were no press releases or notifications issued for the Positive Mental Health Infographic. This may have impacted on the awareness of the infographic.

Mental illness surveillance has been undertaken by the Public Health Agency of Canada for a number of years. The 2006 report The Human Face of Mental Health and Mental Illness in Canada continues to be frequently accessed on the Public Health Agency of Canada website. In 2014-15, the report was accessed 25,661 times. This is a significant increase (over 500%) from 2010-11 when the report was viewed 4,483 times. This increase is even more notable when considering that the information is likely outdated at this time. Visits to the Public Health Agency of Canada website did not increase at a similar rate over the five-year period. Stakeholders generally felt that this report had helped increase awareness and knowledge of mental health and mental illness, including the definition of positive mental health.

After completion of development work, in 2010, the Public Health Agency of Canada expanded the CCDSS to include surveillance of mental disorders including mood and/or anxiety disorders. The CCDSS is a network of provincial and territorial surveillance systems that utilizes administrative health data such as physician billing claims and hospital discharge abstract reports. The  Report from the Canadian Chronic Disease Surveillance System: Mental Illness in Canada, 2015 was published in July 2015. The CCDSS data on mental disorders include children and adolescents, as it is based on administrative data rather than self-reported data. The report highlights the high demands placed on the health care system for mental health services.^{Footnote 38} The Report from the Canadian Chronic Disease Surveillance System: Mood and Anxiety Disorders in Canada, 2015 is planned for release by the end of 2015.

The 2014 Survey on Living with Chronic Diseases in Canada (SLCDC) was funded and developed by the Public Health Agency of Canada, and conducted by Statistics Canada. Development of the survey questionnaire began in 2013 with the Public Health Agency of Canada and Statistics Canada working closely on the content.  The survey was a cross-sectional follow-up to the 2013 Canadian Community Health Survey, and provides information on how mood and anxiety disorders impact Canadians and how individuals affected and their health professional(s) manage them. In July 2015, Mood and anxiety disorders in Canada: Fast facts from the 2014 Survey on Living with Chronic Diseases in Canada was released based on data from the 2014 SLCDC. The fact sheet was posted on healthycanadians.ca. There were no accompanying press release to support the posting. At the time of the evaluation, a suite of papers was being developed for publication in the branch journal, Health Promotion and Chronic Disease Prevention, in 2016.

Some internal and external interviewees indicated that the Public Health Agency of Canada could be more responsive in releasing timely information on mental illness. Public Health Agency of Canada staff indicated that various techniques were being explored to share information more rapidly. These efforts include the use of data cubes and fact sheets. It is important to note that in chronic disease surveillance there is an unavoidable time lag between data collection and data release that can be as much as 5 years.

The Public Health Agency of Canada has been working in collaboration with provinces and territories as well as federal partners to advance mental health and mental illness surveillance. The Mental Health and Addictions Information Collaborative includes other national governmental and non-governmental organizations working in the area of mental health and addictions data, including MHCC, Statistics Canada, CIHI, and others. This collaborative was consulted throughout the development of the indicator framework for positive mental health. In addition to advice from the Mental Health and Mental Illness Surveillance Advisory Committee, the CCDSS Mental Illness Working Group and the CCDSS Science Committee informed the expansion of the CCDSS. As well, the Public Health Agency of Canada convenes the Suicide Related Surveillance Data working group that includes federal departments. This work supports data collection related to suicide. Suicide rates are recognized to be underreported, and there are variations across the provinces and territories in the type of information collected and reported.

Although the expansion of mental health, mental illness and suicide surveillance by the Public Health Agency of Canada was appreciated by stakeholders, there were some identified gaps. The unavailability of data on children and youth was identified as a concern, though the recent expansion of the CCDSS is working on addressing this concern. The Public Health Agency of Canada has undertaken work with Statistics Canada to include mental health as a component of the children's health survey that will be piloted in 2016. Another area that was identified by stakeholders was the collection of data from community health centres on mental health services. As mental illness is not always treated in a hospital setting or treated at all, administrative data is likely underreports prevalence and treatment rates. There are also challenges associated with the relevant diagnostic codes being applied in administrative data. The Public Health Agency of Canada works to compare multiple data sources, and interpret the data in a way that will inform public health decision making. An example of this is the publication of  "Comparison of the estimated prevalence of mood and/or anxiety disorders in Canada between self-report and administrative data".^{footnote 39}

Overall, the Public Health Agency of Canada's positive mental health and mental illness surveillance responds to the sixth Strategic Direction of the Mental Health Strategy for Canada, and specifically priority 6.2 "Improve mental health data collection, research and knowledge exchange across Canada." Surveillance and monitoring systems are better able to track mental health, mental illness, suicide and their risk and protective factors in Canada.

Mental health promotion and suicide prevention

The Public Health Agency of Canada's involvement in international and national activities has helped to advance mental health promotion priorities which highlight the value of mental health promotion and positive mental health. There is still work to be done on the common understanding of mental health versus mental illness.

Although there is recognition that mental health is more than the absence of mental illness, the mental health field is still very much associated with the delivery of mental health services to individuals experiencing mental illness. The understanding of the value of mental health promotion as part of the mental health continuum is growing as evidence increases about the role of early intervention in improving mental health and preventing mental illness. Mental health promotion can include increasing coping skills and the resiliency of people in a variety of settings, such as school or workplaces.

The Public Health Agency of Canada is working to build internal and external capacity and expertise in the area of mental health promotion, as well as increase attention on the importance of positive mental health. The Public Health Agency of Canada worked closely with the MHCC on the inclusion of mental health promotion into the Mental Health Strategy for Canada. Strategic Direction 1 focuses on the promotion of mental health across the lifespan. Public Health Agency of Canada staff were also part of the Technical Committee on Psychological Health and Safety in the Workplace engaged in the development of the National Standard. The primary goal of the Standard is promotion of mental health.

International work by the Public Health Agency of Canada has also increased the visibility of mental health promotion. The World Health Organization's Comprehensive Mental Health Action Plan (2013-2020) includes an objective linked to mental health promotion, and broadens its scope beyond a focus on mental illnesses to include mental health and mental health promotion. The work of the Public Health Agency of Canada on positive mental health indicators has been seen as an international best practice and will help to illustrate Canada's progress with regards to the Comprehensive Action Plan.

The Public Health Agency of Canada works with provinces and territories through the Mental Health Promotion Task Group of the HPCSC of the Public Health Network (PHN). The MHPTG developed and delivered on a workplan as approved by the PHN Council.  The objectives of the work plan are to advance mental health promotion across sectors by engaging with health and social services and providing them with tools and resources, and to enhance surveillance and data collection to better understand mental health in Canada.  The collective work of the MHPTG has been challenging in part due to  differing understandings of mental health promotion that made it difficult to arrive at a common approach.  Further, different provincial and territorial governance structures are related to mental health. In some jurisdictions they are linked to addictions, in others mental health promotion is co-located with social services. This can impact the focus and reach of the work undertaken by the Task Group.

Within the Public Health Agency of Canada, mental health promotion has included the development of positive mental health messaging. These messages were developed and pilot tested in 2012 and intended for a larger public awareness campaign. The messages are now available on the Healthy Canadians website.

The Public Health Agency of Canada has also funded a project which focussed on seniors' mental health. This project involved updating mental health guides for seniors and their families, and developing continuing education modules to assist pharmacists in addressing mental health issues with their clients. The Canadian Coalition for Seniors' Mental Health received funding to lead the project, with Shoppers Drug Mart as the initial private sector partner for the distribution of the guides. While the continuing education is voluntary, Shoppers Drug Mart is encouraging all pharmacists in their network of stores to participate in the training and to encourage ongoing distribution of the guides.

Many of the Public Health Agency of Canada's community-based programs can impact the mental health of the participants. Programs that address social isolation or coping skills are strong supporters of mental health promotion. These program areas do not always interact with each other or with those involved in the broader issues of mental health promotion. These programs could be strong levers to promote materials, such as the positive mental health messages, or to explore best practices and innovations in mental health promotion. There is not a clearly understood or consistent definition of mental health promotion or positive mental health among the program areas.

Within the Public Health Agency of Canada, individual documents (e.g., Diagnostique, draft strategic plan) have supported the leveraging of existing programs and resources to promote mental health and prevent mental illness. However, interviewees suggested that the Public Health Agency of Canada and the broader health portfolio does not have a clearly  articulated vision for mental health work. Although activities and priorities can be linked to the Mental Health Strategy for Canada, there is no clear articulation of an overall direction for the health portfolio, or how roles and responsibilities of the organizations  tie to a longer term goal.

The Public Health Agency of Canada's engagement of relevant federal departments and external stakeholders in the development of the Federal Framework for Suicide Prevention was recognized as useful and a strong opportunity for information sharing and will support enhanced collaboration.

The Federal Framework for Suicide Prevention is being developed in accordance with An Act respecting a Federal Framework for Suicide Prevention which became law in 2012. The Act contains six areas of action:

1. Provide guidelines to improve public awareness and knowledge of suicide.
2. Disseminate information about suicide and its prevention.
3. Make existing statistics about suicide and related risk factors publicly available.
4. Promote collaboration and knowledge exchange across domains, sectors, regions and jurisdictions.
5. Define best practices for suicide prevention.
6. Promote the use of research and evidence-based practices for suicide prevention.

Development of the Framework has involved public consultations. There were 376 respondents, with 111 people representing organizations and another 265 responding as individuals. Overall, feedback from the consultation with participants revealed a need for leadership to coordinate the segmented suicide prevention efforts currently taking place across Canada. Provinces and territories were also consulted as part of the Framework development.

Interactions with the relevant federal departments took place through bilateral meetings or discussions. All of the federal departments interviewed for this evaluation felt that the Public Health Agency of Canada had been very collaborative in the work to develop the Framework. Federal partners reported that the preliminary drafts of the Framework reflected their comments and input. Some partners stated that the Public Health Agency of Canada undertook appropriate levels of outreach; however, some external groups (e.g., National Aboriginal organizations) could have been involved earlier in the consultation process. Other organizations representing marginalized populations, such as LGBTQ, were also consulted. Federal partners appreciated the opportunities to share information on ongoing activities and to discuss upcoming research priorities. They expressed interest in continuing to consult with other departments and share information and coordinate research activities to advance work in the area of suicide prevention.

As required by the Act, the Framework does not replace existing strategies or frameworks implemented by provinces, territories, communities, and Aboriginal organizations. Six provinces and one territory currently have their own suicide prevention strategies. The Framework will lay out priorities to align federal activities in suicide prevention and improve collaboration among federal government and other stakeholders, including non-governmental organizations, Aboriginal organization, the private sector, and provinces and territories. While outside the requirements of the Act, many key informants (internal and external) expressed the wish that the Framework be associated with a significant action that will strengthen suicide prevention efforts across Canada. The most often heard suggestion was the implementation of a national suicide hotline that could be accessed anywhere in Canada, and connect a person in distress with the closest distress centre.

The Framework will broadly align with the 2014 WHO's first global report on suicide prevention, Preventing suicide: A global imperative^{Footnote 40}. The Framework will include many of the strategies and approaches recommended by the WHO such as conducting surveillance and improving data quality, raising awareness and reducing stigma, engaging stakeholders, and conducting evaluation and research. Other recommended strategies and approaches that involve health services and access to the means of suicide fall under provincial and territorial jurisdiction.

The Public Health Agency of Canada co-chairs a National Collaborative on Suicide Prevention with the MHCC and CASP. The collaborative brings together non-governmental organizations, researchers, NAOs, departments in the health portfolio and public representatives to increase the capacity for effective suicide prevention by connecting people, ideas, and resources on a pan-Canadian level. Health Canada, CIHR and the Canadian Centre for Substance Abuse are members of the steering committee. The collaborative was not directly linked with the Framework development, but is an opportunity to share ideas and coordinate activities.

The MHCC has also begun a national dialogue on suicide prevention through its #308 Conversations. This initiative aimed to have a community meeting on suicide prevention in all 308 federal ridings by the end of August, 2015. By July 2015, 43 conversations in 8 provinces have been held. These conversations were to increase public awareness, and identify best practices or gaps within each community. The MHCC will be preparing a report on the results of these conversations, and this will inform future community-based suicide prevention work by the MHCC. This work was seen as complementary to the development of the Framework, as the Public Health Agency of Canada is not currently engaging in community level activities.

Overall, the public health focus for suicide prevention was strongly supported by all key informants. The work on the development of the Framework has been seen as a successful collaborative approach.  Ongoing monitoring and reporting on the implementation of the Framework as required by the Act (and on the progress of planned activities) were seen as crucial to ensuring its success. Many key informants contrasted these requirements with the lack of accountability in the Mental Health Strategy for Canada . They felt that the requirement for bi-annual reporting would advance suicide prevention work. Similar to the Mental Health Strategy for Canada ,  no new resources were allocated for the implementation of the Framework.

5.4.4 To what extent have partnerships and collaboration been achieved?

The MHCC and the Public Health Agency of Canada have forged strong partnerships with various departments, organizations and sectors. The two organizations have also acted as successful partners to each other and helped to advance mutual goals.

Part of the MHCC's original objectives was to partner with other groups to educate all Canadians about mental health. The MHCC was designed to act as a facilitator, enabler and supporter for national mental health issues. All of the initiatives reviewed as part of this evaluation involved extensive partnerships. There are different levels of engagement in partnerships, from joint projects with shared objectives to using partners to implement or support program delivery. MHCC uses partners at all levels of engagement. For example, MHFA engages hundreds of partners across the country to deliver the program and monitor the implementation

The MHCC has been recognized by stakeholders as being a strong convenor, and its ability to bring people together was recognized as one of the most important aspects of its creation. As the MHCC has a broader mandate than a specific mental health issue or population, it was seen as being able to bring organizations together and speak for all Canadians. One issue that was raised during data collection was the possibility for there to be competition over funding resources as there are multiple organizations engaged in mental health at the national, regional, and community level. Having the MHCC as a convenor was identified as bringing these different organizations together in a cohesive manner. Another benefit to the MHCC's convenor role was the attention that the MHCC has brought to the issue of mental health and interviewees felt that there was more openness to discuss the issue since the creation of the MHCC. Some of the key informants interviewed felt that without the MHCC other mental health indicatives such as the Bell Let's Talk campaign would not have been possible, as the MHCC created the climate within Canada to discuss mental health publicly.

Through its various initiatives and approaches, the MHCC was able to engage with provincial and territorial governments. Seven of the nineteen members of the MHCC Board of Directors in 2014-15 represented provincial or territorial governments. For the development of the Mental Health Strategy for Canada the MHCC created a P/T Reference Group. This group aided in the consultations and development of the national strategy. A pan-territorial Advisory Group was useful in the development and implementation of the MHFA in the North. This Advisory Group was supported by all three territorial governments. Provincial governments are engaged in the implementation of the National Standard for Psychological Health and Safety in the Workplace. In particular, Manitoba and Nova Scotia are participating in the case study for implementation. Provincial governments have supported the anti-stigma work of Opening Minds. The government of British Columbia co-sponsored a regional workshop in support of HEADSTRONG. These examples of MHCC collaboration and partnership with Provincial and Territorial governments show the connections of the MHCC priorities and those of the other levels of governments.

The MHCC has similar challenges to the Public Health Agency of Canada with regards to the different governances structures related to mental health. Respondents felt that the MHCC dealt directly with the Ministries of Health, which may have excluded other departments focussing on mental health promotion or other elements of mental health and mental illness.

Over the past five years, there have been a few key collaborations with specific foci that have advanced coordination of efforts around mental health. These have included the Mental Health and Addictions Information Collaborative and the National Collaborative on Suicide Prevention. The methodological consultations by the Public Health Agency of Canada on the positive mental health indicators were identified as an excellent example of partnership and collaboration. The Mental Health and Addictions Information Collaborative had the initial goals of joint products and coordinated releases. However, this has not happened to date. As previously mentioned, the releases of indicator work by the Public Health Agency of Canada and the MHCC in the same week in January 2015 was seen as disjointed and confusing.

In terms of data collection, it was noted by key informants that there are often data silos. Organizations collect information for their needs and for their populations of interest but larger data needs are not considered or understood. The introduction of data collaboratives was seen as a positive addition, and will hopefully continue to align and coordinate data collection across jurisdictions. This may be of particular importance for data collection for First Nations communities where there is a paucity of data. Reliable data on deaths by suicide is regarded as an essential element to better understand suicide in Canada. The interdepartmental Suicide-Related Surveillance Working Data working group has membership from the Public Health Agency of Canada (chair), Health Canada, Canadian Institutes of Health Research, Statistics Canada, Canadian Institute for Health Information, Department of National Defence, Veterans Affairs Canada, and Correctional Service Canada. It works collaboratively to share information and expertise to address gaps on suicide surveillance. Federal departments with responsibilities for specific populations have also undertaken various research and monitoring activities related to suicide. For instance, Veterans Affairs Canada, the Department of National Defence and Statistics Canada have collaborated on a number of studies that have contributed to the understanding of suicide in the veteran population, including the Survey on Transition to Civilian Life (2010), the Canadian Forces Cancer and Mortality Study (2011), and the Life After Service Survey (2013).

Since the creation of the MHCC, their partnership with the Public Health Agency of Canada has been evolving, and is currently positive and strong. The Public Health Agency of Canada was involved in the development of the National Strategy and supports the implementation of Strategic Directions related to data collection and mental health promotion. The MHCC and the Public Health Agency of Canada both reported positive working relationships at all levels. There have been some instances of lack of communication, particularly regarding the release of data reports or the timing of news releases. MHCC has used its broader reach to support the dissemination of Public Health Agency of Canada surveillance products, in particular the Positive Mental Health infographic.

The Strategic Policy Branch within Health Canada is the main federal liaison with the MHCC. This relationship has evolved over the course of the MHCC's original mandate and in recent years Health Canada has taken a more proactive role in their interactions with the MHCC. The MHCC is an arms-length organization with reporting requirements outlined in the funding arrangement. The requirements are updated on a regular basis and support ministerial engagement where appropriate. In addition, SPB supports the Health Canada representative,  Associate Deputy Minister Paul Glover, on the MHCC Board of Directors.

Other federal departments also interact with the MHCC, Health Canada and the Public Health Agency of Canada on mental health and mental illness. Some of the key departments are: the Department of National Defence and Veterans Affairs Canada on issues related to current and former members of the Armed Forces; Employment and Social Development Canada on issues related to seniors (the Public Health Agency of Canada) and homelessness and workplace mental health (MHCC), Correctional Services Canada, as well as the Treasury Board Secretariat for issues on workplace mental health.

In accordance with the Act respecting a Federal Framework for Suicide Prevention, the Public Health Agency of Canada consulted with relevant federal departments, provincial and territorial governments, and non-governmental organizations, to inform the development of the Framework. Government departments engaged in the development of the Framework included National Defence, Veterans Affairs, Correctional Services, Citizenship and Immigration, the Royal Canadian Mounted Police, Statistics Canada, Transport Canada, Justice Canada, Employment and Social Development, Treasury Board Secretariat, Health Canada and the Canadian Institutes of Health Research. Key informants indicated that the partnerships had advanced mutual goals and were beneficial to their organizations.

FNIHB within Health Canada works with the MHCC and the Public Health Agency of Canada on Aboriginal mental health or mental wellness. The MHCC recognizes that there is more they could be doing with First Nations, Inuit and Métis organizations and communities. They are making efforts to improve outreach to NAOs. This has been supported by FNIHB. FNIHB was also involved in the development of Strategic Direction 5 regarding First Nations and Inuit mental health priorities. The Public Health Agency of Canada partners with FNIHB on suicide related data, and has also engaged NAOs in the development of the Federal Framework for Suicide Prevention.

Within the health portfolio, the Canadian Institutes of Health Research (CIHR) is the main funder of research in the area of mental health. CIHR provides funds in excess of $50 M annually. Interviewees indicated that priorities for this funding are not specifically coordinated with the other health partners; however, they do generally align with the Strategic Directions from the Mental Health Strategy for Canada.

Overall, external and internal stakeholders were not always clear of the roles and responsibilities within the Health Portfolio related to mental health and mental illness. Many had individual relationships with the various departments but suggested that there could be greater coordination across the health portfolio and the federal government. There was concern that there was a potential for the duplication of efforts, and that a greater sharing of best practices would be beneficial.

Multiple federal departments supported the creation of a community of practice or hub for departments working on mental health issues. It was seen as important to bring together policy makers and researchers to share information, resources, as well as best practices. While federal departments deal with specific populations with unique needs, there are sufficient similarities to make such a hub useful to the overall federal community.

Within the Public Health Agency of Canada, the roles between the two main Centres engaged in mental health are clearly defined. It is well understood by staff that the CCDP is responsible for surveillance and data, while the CHP is responsible for mental health promotion and suicide prevention. Internally, there was good communication and collaboration between the Centres. The importance of coordinating research priorities was identified by staff and management to reduce the potential for overlap on mental health topics.

The overlap of responsibilities between the Public Health Agency of Canada and Health Canada was identified by stakeholders as an area requiring attention. There were certain topics, particularly new and emerging issues (such as eating disorders), which did not have a clear lead department. The role of the Public Health Agency of Canada of Canada with regards to mental illness, beyond surveillance, has not been clarified or documented. The Public Health Agency of Canada is the lead on mental health promotion. Some stakeholders were not clear on the overall goals for the mental health work of the Public Health Agency of Canada or Health Canada, which they linked to the lack of clarity around roles and responsibilities. Stakeholders agreed that having CIHR, Health Canada, the Public Health Agency of Canada, and the external MHCC engaged in mental health activities was important as there are multiple areas that need attention, from research to mental health promotion, and inclusion into overall health promotion. The broad range of involvement in mental health across the health portfolio supports the concept of "no health without mental health" supported by the WHO.

5.5 Performance: Issue #5 - Demonstration of Economy and Efficiency

The Treasury Board of Canada's Policy on Evaluation (2009) and guidance document, Assessing Program Resource Utilization When Evaluating Federal Programs (2013), defines the demonstration of economy and efficiency as an assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes. This assessment is based on the assumption that departments have standardized performance measurement systems and that financial systems link information about program costs to specific inputs, activities, outputs and expected results.

The data structure of the detailed financial information provided did not facilitate the assessment of whether outputs were produced efficiently, or whether expected outcomes were produced economically. Specifically, the lack of output/outcome-specific costing data limited the ability to use cost-comparative approaches. In terms of assessing economy, challenges in tracking funding within the broader program envelopes limited the assessment. Considering these issues, the evaluation provided observations on economy and efficiency based on findings from the literature review, key informant interviews and available relevant financial data.

Economic burden

The federal government has acknowledged that "mental health-related illnesses impact the lives of many Canadians, at a great social and economic cost." The direct cost of mental illness was estimated as close to $8 billion in 2008."^{Footnote 41} It is estimated that poor mental health and mental illness costs the Canadian economy between $11 and $50 billion a year.^{Footnote 42} The Public Health Agency of Canada estimated that, in 2008, mental illness had the second highest direct care costs and total care costs (including mortality) when compared to five other major health conditions (including musculoskeletal, cardiovascular digestive, respiratory diseases, and cancer).^{Footnote 43} (Table 4 details information from the Economic Burden of Illness online tool hosted by the Public Health Agency of Canada for 2008)

Table 4. Costs associated with major health conditions 2008

Type of Cost	Mental IllnessTable 4 footnote 1	Cardiovascular Diseases	Digestive Diseases	Cancers	Musculoskeletal Diseases	Respiratory Diseases
Drug	$2,876,755,400	$4,272,675,500	$1,434,019,700	$467,077,300	$1,982,500,200	$1,197,195,800
Hospital	$4,568,980,400	$5,068,039,500	$2,839,407,600	$2,329,384,600	$1,795,850,600	$1,818,541,100
Physician	$2,112,470,800	$2,352,012,100	$1,232,587,100	$1,031,721,700	$2,002,466,300	$632,562,900
Total Direct	$9,558,206,600	$11,692,727,100	$5,506,014,400	$3,828,183,600	$5,780,817,100	$3,648,299,800
Mortality	$12,438,800	$92,660,600	$24,560,700	$166,485,800	$2,493,900	$11,410,400
All Costs	$9,570,645,400	$11,785,387,700	$5,530,575,100	$3,994,669,400	$5,783,311,000	$3,659,710,200
Table 4 footnote 1 Mental Illness includes the following items listed under the heading Neuropsychiatric conditions: Alcohol and drug use disorders, anxiety disorders, mood disorders, nonorganic sleep disorders, PTSD, Schizophrenia and "other" neuropsychiatric disorders. Return to Table 4 footnote 1 referrer

Information regarding the effectiveness of mental health promotion and mental illness prevention interventions has existed for quite some time. However, there has been less work completed assessing a return on investment. Judging economic impact is often complex due to the diversity of programming that may influence the targeted outcomes and the realisation of benefits outside of the sector offering the intervention.  With the data that are available, economic benefits have been noted in areas of anti-stigma, suicide awareness and prevention, health promotion in school activities and primary health care screening for depression^{Footnote 44}.

Funding

Health Canada and the Public Health Agency of Canada spent $85.3 million on mental health and mental illness activities, covered by this evaluation, between 2010-2011 and 2014-2015. The majority of this money was allocated from Health Canada and directed to the MHCC (87.5%) and CDRIN (4.7%). The remaining funding was allocated from the Public Health Agency of Canada to the CHP (4.8%) and CCDP (3%) for surveillance and mental health promotion activities. The overall annual budget has increased 15.7% in this time from $16M to $18.5M and peaked in 2013-2014 at $19M.

No additional funds were provided for priority initiatives within the Public Health Agency of Canada, such as the positive mental health surveillance and the Federal Framework for Suicide Prevention.  Budget 2013 committed "the Public Health Agency of Canada to reallocating $2 million to improve data collection and reporting of mental illness and mental health as recommended in the Mental Health Strategy for Canada, to improve knowledge and foster collaboration." This internal branch reallocation of funds has allowed the Public Health Agency of Canada's mental health and mental illness surveillance work to continue. The Act respecting a Federal Framework for Suicide Prevention received Royal Assent on December 14, 2012.  No new resources were associated with the passage of this legislation. Internal reallocations have provided resources for the development of the Federal Framework for Suicide Prevention.

The MHCC is funded by Health Canada with a 10-year mandate (2007-2017). In the Economic Action Plan 2015^{Footnote 45}, the federal government acknowledged that the advancement of research in this area, particularly research aimed at developing more effective diagnostic and treatment tools, is critical to improving the lives of these individuals.^{Footnote 46} To this end, in April 2015 the government signalled its intention to renew the MHCC for another 10-year period beginning in 2017 to continue its important work to promote mental health in Canada and foster change in the delivery of mental health services, including suicide prevention^{Footnote 47}.

Financial reporting from the MHCC was not linked directly to specific priority areas therefore it was not possible to determine the proportion of funds that were directed toward key deliverables and management.

The MHCC priority areas are governed by a grant agreement; however, the MHCC has also undertaken several activities that were not explicitly identified in the grant agreement (e.g., Mental Health First Aid training program, National Standard for Psychological Health and Safety in the Workplace, suicide prevention). These activities were developed through both informal and formal consultations with the Public Health Agency of Canada and Health Canada and further outlined in their annual business plans.

Efficiencies

Within the activities of Health Canada and the Public Health Agency of Canada efficiencies were created with regards to the ability to leverage funds and partnerships. The complementarity of various initiatives allowed for an efficiencies. For example:

• There were several synergies noted in the work undertaken by the Public Health Agency of Canada and MHCC to develop mental health surveillance and indicators. The Public Health Agency of Canada's focus on a public health perspective was viewed by the MHCC as complementary to the organization's focus on health services and evaluations. These projects were under development at the same time, providing the opportunity for both organizations to work collaboratively.
• The Commission has been able to leverage Health Canada funding with over 350 partnerships to bring in additional funding, resources and expertise in-kind. It is able to take advantage of its position outside of the federal, provincial, territorial framework to develop partnerships across many different sectors including: the justice system, primary healthcare, workplace, housing and several other areas. Interviewees reported partnerships with the MHCC were complementary, with the MHCC acting as a catalyst for their work and a means to distribute research.
• The MHCC's KEC serves as a national repository for information regarding mental health issues. It synthesizes evidence-based knowledge, best and promising practices, and innovations in mental health, facilitates the sharing of this knowledge and builds the capacity of stakeholders to adopt this knowledge. Interviewees reported that the KEC stood out as a single resource window for important program information and best practices. It was frequently cited as the MHCC's most significant contribution.
• CDRIN hubs have been individually successful in leveraging in-kind and matching funding support from Provincial agencies/institutions.   Outside investments leveraged by CDRIN totaled $970,879 over their operating period.

Despite the efficiencies gained, there are areas that could be improved.

• Interviewees noted that there was no overarching vision for mental health activities within the health portfolio. Clearly articulating existing goals  would clarify the important difference between mental health and mental illness and present the intended direction and lead to better resource allocation. Interviewees expect that this would facilitate communication within the Health Portfolio and with the public, as well as assist in the clarification of roles and responsibilities.
• Despite clearly identified priority areas and efforts to collaborate on surveillance and data (the Public Health Agency of Canada engaging in the MHCC consultative process and MHCC contribution to the Public Health Agency of Canada's Mental Health/Mental Illness Surveillance Advisory Committee), a number of concerns were raised by interviewees regarding the effectiveness and efficiency of the process. These concerns included poor coordination of resources, overlapping indicators and poor communication regarding released information.

Performance measurement

The Public Health Agency of Canada's mental health and mental illness surveillance activities are assessed by CCDP Program Performance Section as part of the overall Centre Performance Measurement Plan. Performance measurement data include citation analysis, web analytics, and stakeholder survey. Standardized tools have been developed to assess use and stakeholder satisfaction. The mental health promotion work does not yet have a detailed performance measurement strategy. However, the Act respecting the Federal Framework for Suicide Prevention requires the Government of Canada to report on its progress under the Framework in 2016 and every two years thereafter.

The MHCC has utilized performance measurement data to support decision-making. The MHCC submits a yearly business plan to Health Canada detailing the stated key activities that were proposed in the previous year and the degree to which these activities have been accomplished. It then proposes key deliverables and activities for the upcoming year(s) and proposes indicators, in terms of both outputs and outcomes, against which these should be measured. The MHCC also describes a risk management approach that supports the accountability and operational effectiveness through decision-making, planning and budget preparation.

Since the MHCC was established in 2007, there have been eight evaluative activities, five conducted internally and three conducted externally. Undertaking and reporting on MHCC activities to address recommendations demonstrates the MHCC's commitment to operational excellence. Key recommendations that the MHCC has adopted as a result of these evaluations include a renewed focus on knowledge exchange and measurement of the impact and implementation of the Mental Health Strategy for Canada (recommendation from the MHCC Formative Evaluation, 2011), development of an online engagement platform for the KEC  (recommendation from the Knowledge Exchange Centre Interim Report, 2014) and continued monitoring of the implementation of key recommendations from the Mental Health Strategy .  

Performance measures for the Canadian Depression Research and Intervention Network is under development. All of reporting requirements of the existing grant have been met.

International Comparisons

Many countries around the world have developed and implemented national mental health strategies. In the  countries examined, the majority of strategies were developed with the department or ministry of health. Australia has a separate Mental Health Commission that is distinct from the Ministry of Health, but the Commission reports directly to the Minister of Health. Canada appears unique in having the MHCC as an arm's length organization with no parliamentary reporting requirements. New Zealand deestablished its Mental Health Commission as a separate crown entity in 2012, and included a Mental Health Commissioner within the Office of the Health and Disability Commissioner.

With the majority of mental health strategies being developed within the governments, these strategies have the endorsement of their respective governments. In Norway and Sweden, the mental health strategies have been adopted by parliament. The Fourth Mental Health Plan of Australia was supported by all of the Health Ministers, both federally and at the state and territory level. In Canada, while the federal government funds the MHCC and all provinces, except for Quebec, endorsed the creation of the MHCC, the Mental Health Strategy for Canada has not been adopted or endorsed by the Government of Canada. Alignment with the Mental Health Strategy for Canada is currently voluntary, and while Provinces and Territories have stated their support, there is no monitoring of the implementation. In New Zealand, the Mental Health Commissioner is responsible for monitoring the implementation of the National Mental Health Strategy, and in Australia periodic mental health reports provide progress updates on the implementation of their plan to the government and the public. There is no comparable actions in Canada.

The high-level and full government support of the mental health strategies in other countries and the requirements to monitor implementation have allowed these countries to detail the progress made in the areas of mental health and mental illness.

6.0 Conclusions

6.1 Relevance Conclusions

6.1.1 Continued Need

Mental health and mental illness continue to be important issues and the needs remains consistent. A significant number of Canadians are directly and indirectly affected by poor mental health and mental illness. Every year, one in ten Canadians will experience a mental illness and one in three will experience a mental illness or substance use disorder at some point in their lifetime. In 2012, 40% of adults in Canada had a family member with substance abuse, emotional or mental health problems. Close to 4,000 Canadians die every year as a result of suicide.

Certain segments of the Canadian population have higher rates of mental illness, suicide or suicide ideation. Suicide is the second leading cause of death among youth, and the prevalence of suicide is a significant problem in some First Nations and Inuit communities.

The direct cost of poor mental health and mental illness was estimated as close to $8 billion in 2008. The indirect annual costs have been estimated to range between $11 billion and $50 billion.

6.1.2 Alignment with Government Priorities

Health Canada's mental health and mental illness activities align with its strategic outcome of a health system responsive to the needs of Canadians. The Public Health Agency of Canada's mental health and mental illness activities align with its strategic outcome of protecting Canadians and empowering them to improve their health. Additionally, these activities meet other domestic and international priorities. Various federal announcements and international declarations have shown the importance of mental health as a federal government priority. Since the 2006 publication of Out of the Shadows at Last, mental health has been a government priority. The creation of the MHCC and the intent to support a renewed mandate in Budget 2015 shows the support for focussed attention on mental health. Additional funding has also been provided to the MDSC to focus on depression and PTSD.

In May 2013, members of the World Health Assembly, including Canada, formalized mental health as a priority by adopting the WHO's Comprehensive Mental Health Action Plan 2013-2020. This supports the belief that "there is no health without mental health"^{Footnote 48}. The United Nations has recently adopted Sustainable Development Goals which include the promotion of mental health and well-being^{Footnote 49}.

The passing of An Act respecting a Federal Framework for Suicide Prevention in 2012, and the resulting development of a Framework has highlighted the priority the federal government is placing on suicide prevention.

6.1.3 Alignment with Federal Roles and Responsibilities

The roles and responsibilities for Canada's health care system is a federal and provincial-territorial joint responsibility. Mental health services are under the jurisdiction of provinces and territories for the majority of Canadians. However, the federal government has a role in coordination and collaboration on mental health and mental illness activities, in conducting and reporting on mental health, mental illness and suicide surveillance and research, as well as being a resource for information on best practices and innovation. The Canada Health Act details Health Canada's responsibilities for the promotion and restoration of the mental well-being of Canadians. The Public Health Agency Act outlines the Public Health Agency of Canada's role in health promotion and health surveillance. The federal government also has responsibilities for mental health services for specific federal populations.

6.2 Performance Conclusions

6.2.1 Achievement of Expected Outcomes (Effectiveness)

The MHCC has been successful in addressing the three specific elements of its original mandate. The original mandate called for the development of a national strategy, an anti-stigma campaign and the development of a knowledge exchange centre. Changing Lives, Changing Directions: A Mental Health Strategy for Canada  was released in 2012, after extensive national consultations. Work has been undertaken across the country to respond to the six Strategic Directions, the 26 priorities and the 109 recommendations. Provincial and territorial governments along with NGOs and federal departments have been working to implement the Strategy. The lack of clear action plans and accountability for recommendations has been seen as a hindrance to full implementation of the Strategy.

The other aspects of the MHCC's original mandate, the Anti-Stigma campaign (Opening Minds) and the Knowledge Exchange Centre, have been successful in their initial efforts. Opening Minds is showing success in reducing stigma amongst target populations. Expansion and reiteration of messaging will be important to continue reducing stigma. The KEC has developed partnerships and increased the accessibility of research and guidelines related to mental health. The consistent implementation of dissemination plans for research has increased the reach and availability of reliable information.

The MDSC through CDRIN has set up multiple Depression Hubs across Canada and has successfully incorporated the voice of people with lived experiences into the research process, including the identification of research priorities.

The MHCC has been recognized as a convenor and has succeeded  together to advance the priorities of the Mental Health Strategy for Canada. The Public Health Agency of Canada was also seen as successfully creating partnerships, in respect to data initiatives and the work on the Federal Framework for Suicide Prevention. As there are multiple federal departments, levels of governments and NGOs engaged in mental health, partnerships were seen as crucial to share best practices, prevent duplication of efforts, and coordinate activities to achieve the overall vision for mental health.

Concerns were raised that the roles and responsibilities across the health portfolio related to mental health, mental health promotion, mental illness and suicide prevention were not clearly defined or understood. The lack of an overall vision was also noted. Furthermore, there are numerous federal departments with activities or responsibilities in these areas. There was interest in the development of a community of practice or hub for mental health within the federal government.

The Public Health Agency of Canada's work on positive mental health surveillance and mental illness surveillance have resulted in new knowledge products and have expanded and improved surveillance systems. Publications such as the Report from the Canadian Chronic Disease Surveillance System: Mental Illness in Canada, 2015 and the infographic on positive mental health have expanded the available knowledge on mental illness and positive mental health.

Mental health promotion work by the Public Health Agency of Canada has advanced the understanding of the fundamental difference between mental health and mental illness, which is at the core of mental health promotion. This has been accomplished through support to international and national efforts.  The Public Health Agency of Canada's community-based programs address elements of positive mental health and could be better leveraged by stakeholders to share consistent messaging regarding mental health, and promote mental health across the lifespan. The lack of consistent terminology (e.g., mental wellness, positive mental health, mental health and mental illness) is seen as an impediment to full collaboration on work in the area of mental health promotion.

The Public Health Agency of Canada's work to develop the Federal Framework for Suicide Prevention was recognized as a strong collaborative effort.  The Framework will outline federal guiding principles, strategic objectives and priorities in suicide prevention.  Ongoing monitoring and reporting on the implementation of the Framework (and on the progress of planned activities), were seen as crucial to ensuring accountability, decision making and continuous improvement of federal efforts in suicide prevention.

6.2.2 Demonstration of Economy and Efficiency

The mental health and mental illness activities undertaken by Health Canada and the Public Health Agency of Canada were administered economically; however, efficiency may be improved with clearly articulated roles and responsibilities, related to the mental health vision.

The funding arrangements with MHCC and MDSC have been actively managed by Health Canada. MHCC was able to leverage over 350 partnerships to further their work on anti-stigma, suicide prevention, workplace mental health and Mental Health First Aid. Further work with Aboriginal and non-traditional organisations was identified as an area for future opportunity.

No additional funds were provided for priority initiatives within the Public Health Agency of Canada, such as the positive mental health surveillance and the Federal Framework for Suicide Prevention. Internal branch reallocations of funds have allowed the Public Health Agency of Canada's mental health and mental illness work to continue.

The Public Health Agency of Canada priority areas for mental health and mental illness have been defined; however the evaluation noted a lack of common understanding of roles and responsibilities and the  consistent use of terminology throughout the Health Portfolio. It is expected that clearly articulated roles and responsibilities and the use of common language would facilitate communication within the Health Portfolio and with the public.

7.0 Recommendations

Recommendation 1

Recognizing the continued significant need across Canada in the area of mental health and mental illness, a mental health vision for the Health Portfolio should be articulated, including clarification and communication of roles and responsibilities regarding mental health, mental health promotion, mental illness, suicide prevention, and surveillance of mental illness and mental health.

Currently, Health Canada, the Public Health Agency of Canada and the Canadian Institutes of Health Research engage in activities related to mental health and mental illness. Activities within these organizations are complementary and areas of intersection exist. However, the vision or overall goals for mental health and mental illness has not been clearly articulated or communicated. By articulating the vision and detailing the relationship between the activities of organizations and the vision, there will be opportunities to ensure that efforts are aligned and the potential for duplication is minimized. Having a clearly articulated vision will aid in communication with external and internal partners.

Many federal departments and external partners working in the broad area of mental health and mental illness were unsure of who to contact regarding various initiatives. Fluid communication between the Health Portfolio and these partners is important; however these relationships are frequently managed on a project basis. There was a desire for a more coordinated community of practice to bring together those working on mental health within the federal government. The feasibility of this could be explored.

Recommendation 2

Encourage MHCC, for the balance of its mandate, to expand outreach to stakeholders, particularly organizations that the Commission has not traditionally worked closely with, such as organizations addressing substance abuse and National Aboriginal Organizations.

The Mental Health Commission of Canada has strong partnerships across various sectors, but there are areas where partnerships and collaborations could be strengthened. Evidence of co-morbidities are becoming more apparent as data become available. Co-morbidity between substance abuse and mental health has garnered particular attention with several provinces and territories combining mental health programming with addictions and substance abuse. The MHCC and the Canadian Centre on Substance Abuse could work more closely on issues of joint interest.

Another area that could be strengthened by further outreach includes Indigenous populations. While some work has been undertaken on Strategic Directions 4 and 5 of the Mental Health Strategy, there has not been a strong focus on Aboriginal mental health by the Mental Health Commission of Canada. Stronger partnerships and relationships with National Aboriginal organizations, Aboriginal communities and Health Canada's First Nations and Inuit Health Branch could improve efforts in this area. It will be important for the MHCC's activities to include these important populations.

Appendix 1 - Logic Model

Appendix 2 - Summary of Findings

Rating of Findings

Ratings have been provided to indicate the degree to which each evaluation issue and question have been addressed.

Relevance Rating Symbols and Significance

A summary of Relevance ratings is presented in Table 1 below. A description of the Relevance Ratings Symbols and Significance can be found in the Legend.

Table 1: Relevance Rating Symbols and Significance

Evaluation Issue	Indicators	Overall Rating	Summary
Continued need for the program
What are the health and societal needs addressed by Health Canada and the Public Health Agency of Canada's mental health and mental illness activities?	Evidence of: current societal and economic burden (national, international) projected societal and economic burden (national, international) new environment/trends (including new players, national, international)	High	A significant number of Canadians are affected directly and indirectly by poor mental health and mental illness . Every year, one in ten Canadians will experience a mental illness, and one in three Canadians will experience a mental illness or substance use disorder sometime in their lives. Of the close to 4,000 Canadians who die every year as a result of suicide, most were confronting a mental health problem or illness.  In 2012, 40% of adults in Canada had a family member with drug, alcohol, emotional or mental health problems. The projected burden of mental illness in Canada is expected to increase.
Alignment with Government Priorities
What are the federal priorities related to mental health and mental illness in Canada? Are Health Canada and Public Health Agency of Canada's current activities aligned with federal priorities?	Current Health Canada and Public Health Agency of Canada activities correspond to federal priorities	High	Federal support for mental health and mental illness includes funding for the Mental Health Commission of Canada's first mandate in 2007, and a commitment to renew funding for a second mandate starting in 2017. Other federal support for mental health and mental illness includes:  Act respecting a Federal Framework for Suicide in 2012, funding of $5.2 million to support the Canadian Depression Research and Intervention Network (CDRIN), and an internal reallocation of $2 million to improve data collection and reporting on mental health and mental illness at the Public Health Agency of Canada.
Are current activities aligned with Health Canada and Public Health Agency of Canada priorities?	Current Public Health Agency of Canada activities correspond to Public Health Agency of Canada priorities Current Health Canada activities correspond to Health Canada priorities	High	Activities and funding commitments at the Public Health Agency of Canada and Health Canada align with strategic priorities related to mental health and mental illness.
Alignment with Federal Roles and Responsibilities
What is the federal role related to mental health and mental illness?	Evidence of federal role in mental health and mental illness, including  international responsibilities	High	The federal government has a clear role in the areas of national coordination and collaboration on mental health and mental illness, as well as the conduct of mental health and mental illness surveillance and research. Activities of Health Canada and Public Health Agency of Canada are clearly aligned with the federal roles in the area of mental health and mental illness policy coordination and research.
Does the federal role and current activities duplicate the role of stakeholders? Are there overlaps?	Evidence of duplication/overlap/complementary role between the federal role and stakeholders role Perception of gaps between federal role and stakeholders role	Partial	Key informants also noted the federal role was not always clear. Although key informants often indicated that the role of their own organization was clear, they also indicated that the role of other organizations, including the federal government, was less clear. Informants reported that aspects of this complexity stems from inconsistent use of language regarding mental health and mental illness.
Legend - Relevance Rating Symbols and Significance: High There is a demonstrable need for program activities; there is a demonstrated link between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes; role and responsibilities for the federal government in delivering the program are clear. Partial There is a partial need for program activities; there is some direct or indirect link between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes; role and responsibilities for the federal government in delivering the program are partially clear. Low There is no demonstrable need for program activities; there is no clear link between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes; role and responsibilities for the federal government in delivering the program have not clearly been articulated.

Performance Rating Symbols and Significance

A summary of Performance Ratings is presented in Table 2 below. A description of the Performance Ratings Symbols and Significance can be found in the Legend.

Table 2: Performance Rating Symbols and Significance

Issues	Indicators	Overall Rating	Summary
Achievement of Expected Outcomes (Effectiveness)
To what extent have the outcomes been achieved by the Mental Health Commission of Canada? Outcome #1: Uptake and Use of the Mental Health Strategy of Canada Outcome #2: Anti-Stigma Campaign Outcome #3: Knowledge Exchange Centre	Evidence of availability and uptake of Mental Health Strategy, including by P/T mental health strategies Level of awareness of mental health and mental illness issues, including the impact of stigma Evidence of availability, accessibility, timeliness of knowledge products	Achieved	The Mental Health Strategy for Canada and the preceding Framework have been identified as important reference points for organizations and governments in shaping their own mental health strategies and programs. Work has been undertaken across the country to respond to the Strategic Directions and priority areas identified in Changing Directions, Changing Lives: The Mental Health Strategy for Canada. The Opening Minds Anti-Stigma Campaign has demonstrated progress in addressing stigma with targeted populations. The MHCC Knowledge Exchange Centre has developed and shared key research material related to the Strategic Directions outlined in the Mental Health Strategy for Canada. This has led to an increase in access to mental health and mental illness knowledge products.
To what extent have the outcomes from CDRIN been achieved?	Evidence of sustained, coordinated research efforts on depression and PTSD Improved targeted mental health research, tools and strategies to prevent/reduce, diagnose and treat depression and PTSD Strengthened collaborative networks and information sharing among researchers and stakeholders	Achieved	The Canadian Depression Research and Intervention Network has made progress in developing research hubs across the country. These hubs undertaking research projects and benefit from the inclusion of the experiences of people with lived experience into the research process. Although there is limited performance measurement data regarding CDRIN's strengthening of networks and information sharing, interviews and survey responses indicated that stakeholders were very positive about CDRIN's contribution thus far.
To what extent have the outcomes from Public Health Agency of Canada been achieved? Outcome #5: Evidence of enhanced mental health and mental illness surveillance systems Outcome #6: Mental health promotion work	Evidence of current and accurate surveillance reports; Evidence of enhanced MHMI surveillance system Evidence of availability, accessibility of knowledge products Level of awareness of mental health and mental illness issues	Achieved	The recent work on positive mental health surveillance by Public Health Agency of Canada and supported by the Mental Health and Addictions Information Collaborative has filled a data gap and complements the expanded surveillance of mental illness and suicide. Public Health Agency of Canada's involvement in international and national activities has helped to advance mental health promotion priorities and showcased the value of mental health promotion and positive mental health. There is still work to be done on the common understanding of positive mental health.
To what extent have partnerships and collaboration been achieved?	Evidence of current collaboration and coordination between internal and external partners	Progress Made; Further Work Warranted	The MHCC and Public Health Agency of Canada have forged strong partnerships with various departments, organizations and sectors. They have also created a successful and mutually beneficial partnership between themselves. The MHCC could improve partnerships with First Nations, Inuit and Métis organizations and communities, as well as with organizations that relate to co-morbid illnesses.
Demonstration of Economy and Efficiency
Have HC and Public Health Agency of Canada undertaken their activities in the most economical manner?	Variance between planned and actual expenditures, and implications Views appropriateness of distribution of funding Views on alternative delivery methods/approaches Views on factors facilitating/hindering efficiency	Achieved	The funding arrangements with MHCC and MDSC have been managed successfully. MHCC has managed to leverage over 350 partnerships to further their work on anti-stigma, suicide prevention, workplace mental health and Mental Health First Aid. No additional funds were provided for priority initiatives within Public Health Agency of Canada, such as the positive mental health surveillance and the federal framework for suicide prevention. Internal branch reallocations of funds have allowed Public Health Agency of Canada's mental health and mental illness work to continue.
Have HC and Public Health Agency of Canada undertaken their activities in the most efficient manner?	Views on whether products have been produced in an economical fashion	Progress Made; Further Work Warranted	Public Health Agency of Canada priority areas for Mental Health and Mental Illness have been defined; however the evaluation noted a lack of common understanding of roles and responsibilities and consistent use of terminology throughout Public Health Agency of Canada programs. Gaps were identified in terms of information sharing to all relevant program areas. It is expected that a clearly articulated roles and responsibilities would facilitate communication within the Health Portfolio and with the public.
Is there a performance measurement culture in place? To what extent is the information used to inform decision-making?	Evidence and use of performance information	Progress Made; Further Work Warranted	Public Health Agency of Canada's mental health surveillance activities are assessed by CCDP Performance Measurement Section. Mental health promotion work does not have a detailed performance measurement strategy. Indicators for the Federal Framework for Suicide Prevention are under development, and bi-annual reporting is a requirement. The Mental Health Commission of Canada has utilized performance measurement data to support decision-making.
Legend - Performance Rating Symbols and Significance: Achieved The intended outcomes or goals have been achieved or met. Progress Made; Further Work Warranted Considerable progress has been made to meet the intended outcomes or goals, but attention is still needed. Little Progress; Priority for Attention Little progress has been made to meet the intended outcomes or goals and attention is needed on a priority basis.

Appendix 3 - Evaluation Description

Evaluation Scope

The scope of the evaluation covered the period of April 2010 to March 2015, and included an examination of the grants to MHMC and MDSC, as well as the activities of the Mental Health Promotion Unit in the Public Health Agency of Canada and the mental health and illness related surveillance systems (Mental Illness, Positive Mental Health, and Self-inflicted injuries).

The following activities were considered out of scope for this evaluation: investments in mental health promotion through the Innovation Strategy; child and parent mental health activities funded through the Community Action Program for Children (CAPC), the Canadian Prenatal Nutrition Program (CPNP) or CAPC/CPNP National Projects Fund; Child maltreatment surveillance; neurological conditions activities; and Mental Wellness programming in the First Nations and Inuit Heath Branch of Health Canada.

Evaluation Issues

The specific evaluation questions used in this evaluation were based on the five core issues prescribed in the Treasury Board of Canada's Policy on Evaluation (2009). These are noted in the table below. Corresponding to each of the core issues, evaluation questions were tailored to the program and guided the evaluation process.

Data Collection and Analysis Methods

Data Collection and Analysis Methods

Evaluators collected and analyzed data from multiple sources. Sources of information used in this evaluation included:

Document review - approximately 30 documents pertinent to mental health and mental illness were reviewed for information regarding the relevance (priorities, roles and responsibilities) of the activities.

File review - approximately 419 files held by the Mental Health Commission of Canada, the Centre for Health Promotion and the Centre for Chronic Disease Prevention were reviewed to obtain information regarding all aspects of the activities related to mental health mental illness and in particular the performance (achievement of outcomes, economy and efficiency) of the activities.

Financial data review - a review of financial data from 2010-2011 to 2014-2015, including budgeted and actual expenditures.

Key informant interviews - interviews were conducted with 76 stakeholders (Mental Health Commission of Canada (n=10); Public Health Agency of Canada (n=31); Health Canada (n=8) other government departments or agencies (n=5); Provincial/ Territorial stakeholders (n=3); external partners (n=6) and academics (n=5); Mood Disorders Society of Canada (n=5).  Interviews were conducted by 2 evaluation team members, one with a primary responsibility for taking notes. Notes and transcripts were analysed with NVIVO.

International analysis - a review of mental health and mental illness activities conducted by the United States, Australia and the United Kingdom was carried out by scanning the countries' public health websites.

Literature review - a search for Canadian and international literature from the past five years using search terms of "Canada", "mental illness" and "mental health". After examining documents to ensure relevance, 62 articles were reviewed.

Performance data review - a review of data on performance of cluster area activities between 2010-2011 and 2014-2015 (stakeholder satisfaction survey, knowledge uptake survey, web analytics, and project-level performance and evaluation reports).

Stakeholder web survey -An online survey was in the field between April 22, 2015 and May 5, 2015; it was sent to 130 stakeholders proposed by the Centre for Health Promotion and the Centre for Chronic Disease Prevention. There were 25 fully complete surveys for a response rate of 22%. A survey of CDRIN conference attendees was conducted between February 20, 2015 and March 10, 2015. Respondents had the option to complete the survey online or on paper. There were a total of 94 respondents.  It was estimated that the conference was attended by 300 researchers, clinicians, policy makers and persons with lived-experience.

Data were analyzed by triangulating information gathered from the different sources and methods listed above. This included: systematic compilation, review and summarization of data to illustrate key findings; statistical analysis of quantitative data from databases; thematic analysis of qualitative data; and comparative analysis of data from disparate sources to validate summary findings.

List of Acronyms