Evaluation of the Community Action Program for Children, Canada Prenatal Nutrition Program and Associated Activities 2010-2011 to 2014-2015

List of Tables

• Table 1: Planned Budget for All Program Components ($)
• Table 2: Limitations and Mitigation Strategies
• Table 3: Percentage of People in Canada Living in Conditions of Risk
• Table 4: Prevalence of Pre- and Post-natal Behavior in the General Canadian Population and those Living in Conditions of Risk
• Table 5: Types of Multi-sectoral Project Site Partners
• Table 6: Number of Projects by Region in 2013-2014
• Table 7: Areas Served by the CAPC and the CPNP in 2013-2014
• Table 8: Number of Participants Reached in 2013-2014 - CAPC Funded Sites
• Table 9: Number of Participants Reached in 2013-2014 - CPNP Funded Sites
• Table 10: Risk Profile of Respondents by Program and General Population
• Table 11: Percentage of Multiple Self-identified Risks by Survey Respondents
• Table 12: Impact on Birth Outcomes and Breastfeeding (CPNP)
• Table 13: Variance: Planned Spending vs Expenditures 2010-2011 and 2014-2015 ($M)

Executive summary

This evaluation covered the Community Action Program for Children (CAPC), Canada Prenatal Nutrition Program (CPNP) and associated activities for the period from April 2010 to March 2015. The evaluation was undertaken in fulfillment of the requirements of the Financial Administration Act (FAA) and the Treasury Board of Canada's Policy on Evaluation (2009).

Evaluation Purpose and Scope

The purpose of the evaluation was to assess the relevance and performance of the CAPC, the CPNP and associated activities within the Public Health Agency of Canada (Agency).

The evaluation focussed on activities funded under the CAPC and the CPNP as well as maternal and child health knowledge development and exchange activities under the Centre for Health Promotion. It also covered the Agency's role in associated activities including: the Health Behaviour in School-Aged Children (HBSC) Survey, the Joint Consortium for School Health (JCSH), the Canadian Perinatal Surveillance System (CPSS), and the Canadian Paediatric Surveillance Program (CPSP). It excluded other maternal and child health programs within the Health Promotion and Chronic Disease Prevention Branch, including: Fetal Alcohol Spectrum Disorder, Family Violence Initiative, Aboriginal Head Start in Urban and Northern Communities program (AHSUNC), Oral Health, Cancer in Young People in Canada Surveillance Program, and Child Maltreatment Surveillance.

Program Description

The Agency, through the Health Promotion and Chronic Disease Prevention Branch, conducts a range of maternal and child health activities which focus on upstream prevention looking at common risk factors related to health disparities. The Agency provides national leadership on key maternal and child health issues, with particular emphasis on addressing the needs of vulnerable populations through the following activities: community-based programming, knowledge development and exchange, and surveillance.

All activities are undertaken in collaboration with partners in other government departments, the provincial and territorial governments, the academic community, non-governmental organizations, and local community-based organizations.

The Agency allocated approximately $90 million for these activities in 2014-2015. Over the five year period covered by this evaluation (April 2010 to March 2015), expenditures for all program activities totalled approximately $450 million.

CONCLUSIONS - RELEVANCE

Continued Need

The CAPC, CPNP and associated activities continue to be relevant programs and activities in Canada as many threats to maternal and child health in Canada persist. Conditions of risk (such as low income, low education, Indigenous status, single parents, recent immigrants and teen parents) can negatively impact the health and well-being of families. The Agency investments in the CAPC, CPNP and associated activities support the services/programs, and associated partnerships, and knowledge development and exchange, related to the needs of these at-risk families. Research in population health indicates that these types of upstream investments in children early on in life have the greatest potential for improving their health and well-being in the future.

Alignment with Government Priorities

The CAPC, CPNP and associated activities are relevant in their contribution to the priorities of the Government of Canada and the Agency. These investments reduce health disparities for at-risk mothers and children and strengthen public health, thereby contributing to the overall Government of Canada priority to safeguard Canadian families and their communities. This priority is mirrored in various Agency planning and reporting documents. The CAPC and CPNP are part of the means by which the Government of Canada fulfills its international commitments to the United Nations Convention on the Rights of the Child and the Millennium Development Goals.

Alignment with Federal Roles and Responsibilities

It is appropriate for the federal government and the Agency to administer its maternal and child health programs and associated activities. Existing legislative and program authorities speak to the federal role and provide the Agency with a mandate to provide national leadership, engage in surveillance, and share knowledge and best practices. While provinces and territories have the primary responsibility for maternal and child health, the Agency's role is complementary as the Government of Canada is playing a supporting role where there is a demand to: address an issue of national scope, fill gaps for a vulnerable population, and complement provincial and territorial directions.

CONCLUSIONS - PERFORMANCE

Achievement of Expected Outcomes (Effectiveness)

For CAPC and CPNP funded organizations, community partnerships have led to enhancements in program delivery with participants having greater access to local health and social service networks. Investments in maternal and child health have led to multi-sectoral and multi-jurisdictional partnerships which, in turn, have facilitated the design and dissemination of various valuable knowledge development and exchange activities. In collaboration with stakeholders, a variety of knowledge products to address gaps in community-based resources have been designed and developed through the Division of Children and Youth or DCY (formerly known as the Division of Children, Seniors and Healthy Development). There is an opportunity to formalize a strategic plan to continue to ensure complementarity with resources for health professionals and parents produced by other stakeholders and to optimize limited knowledge development and exchange resources.

A significant proportion of families reached by the CAPC and CPNP funded organizations are living in conditions of risk and many are affected by multiple risks. The proportion of vulnerable individuals represented among participants exceeds the rate at which these populations are represented in the general population. An analysis of qualitative data collected from participants and funded organization staff indicates that the CAPC and CPNP funding has contributed to organizations having a positive impact on the short term health and well-being of participants and their families. However, it was noted that the mix of funded organizations for the CAPC and CPNP has been stable over the last twenty years or so and there are potential gaps in terms of program reach (geographic distribution, wait lists, etc.). There may be further opportunities to analyse a variety of data sources to explore gaps and optimize program reach.

Demonstration of Economy and Efficiency

An assessment of the economic impact of the CAPC and CPNP demonstrated that funded organizations have successfully acquired additional funding and in-kind resources through their relationships with other partners. Through relatively small Agency investments to engage strategic partnerships with external stakeholders, the JCSH, HBSC survey, the CPSS and the CPSP have produced significant knowledge development and exchange products.

Recent centralization of the administration and management of the CAPC and CPNP has allowed the Agency to gain efficiencies. Realigning the management of the programs has led to the streamlining of governance structures and internal processes, such as performance measurement. There is a current performance measurement strategy for the CAPC and the CPNP and efforts have been made to collect performance data across all program areas. Data collection tools and approaches have been refined over time to reduce the reporting burden on the stakeholders.

RECOMMENDATIONS

Recommendation 1

Recognizing that many players have a role in developing and disseminating knowledge products on maternal and child health in Canada, formalize and implement a knowledge development and exchange strategic plan to ensure complementarity and optimization of Agency resources.

There is currently no DCY strategic plan in place to guide decision-making on their knowledge development and exchange priorities. While the Agency has collaborated with stakeholders to develop and disseminate knowledge products to support its community-based programs, it is recognized that provinces/territories and non-governmental organizations also produce knowledge products for health professionals and parents on topics related to maternal and child health. Key informants highlighted the importance of collaborating with stakeholders to ensure that resources are complementary and to raise awareness to expand the reach of products. Key informants also indicated a need to ensure that new and existing products address knowledge gaps and remain current. With the sun-setting of the National Projects Fund (NPF) in 2014-2015, it will be critical to review priorities and formalize a way forward for the DCY to: (a) best address current and emerging needs in Canada for community-based health information products on maternal and child health to support vulnerable populations and (b) identify and share best practices nationally across funded organizations.

Recommendation 2

Review demographic data (including GIS data) to better understand population trends and changes and explore opportunities to optimize program reach.

Management Response and Action Plan for Health Canada

1.0 Evaluation Purpose

The purpose of the evaluation was to assess the relevance and performance of the Community Action Program for Children (CAPC), the Canada Prenatal Nutrition Program (CPNP), maternal and child health knowledge development and exchange activities related to CAPC and CPNP, and other associated activities, including: the Health Behaviour in School-Aged Children (HBSC) Survey, the Joint Consortium for School Health (JCSH), the Canadian Perinatal Surveillance System (CPSS), and the Canadian Paediatric Surveillance Program (CPSP). The period covered by the evaluation was April 2010 to March 2015.

The evaluation was undertaken in fulfillment of the Financial Administration Act (FAA) and the Treasury Board of Canada's Policy on Evaluation (2009), and was conducted by the Health Canada and the Agency's Office of Audit and Evaluation in accordance with the Five-Year Evaluation Plan 2015-2016 to 2019-2020.

2.0 Program Description

2.1 Program Context

The Agency's maternal and child health program activities are managed within the Health Promotion and Chronic Disease Prevention (HPCDP) Branch. The CAPC and CPNP activities are led by the Centre for Health Promotion (CHP) in collaboration with Regional Operations. The CHP also leads the JCSH as well as the HBSC survey contract. These health promotion activities are complemented by surveillance activities, namely the CPSS and the CPSP, managed within the Centre for Chronic Disease Prevention.

Literature suggests that children are particularly susceptible both to positive and negative experiences. Children are particularly sensitive to social determinants, especially in the very early years. Living in conditions of risk has been shown to have adverse impacts on the health and social development of children. It can impact a child's ability to become a healthy and productive adult. Research in population health indicates that upstream investments, where the aim is to prevent negative consequences from occurring in the future by addressing the root causes early on in life, has the greatest potential for improving health and well-being.

The foundation for the Agency's activities in the area of maternal and child health link back to the 1990 Government of Canada commitment made to invest in the well-being of children at the United Nations World Summit for Children. In 1991, Canada ratified the United Nations Convention on the Rights of the Child. In 1992, it launched Brighter Futures: Canada's Action Plan for Children, which in turn led to the launch of the CAPC in 1993. In 1995, support for early childhood development activities was extended into the prenatal period with the establishment of CPNP. Further demonstrating its commitment, funding was extended for the CAPC and CPNP in 1997 and again in 1999 for CPNP. These programs remain the Agency's grants and contributions programs with the largest materiality.

Since 2012, the administration of the CAPC and the CPNP has been in transition, moving from a decentralized business model to a centralized one. Under the previous decentralized model, the Division of Children and Youth (DCY) in the Centre for Health Promotion set the overall policy direction for the programs. Day to day program management and administration were conducted by Regional Operations. As well since 2012, the Agency has been transforming the administration of grants and contributions with the creation of the Centre for Grants and Contributions to manage the administration and resourcing of all grants and contributions. The Agency has also designated to the National Office (CHP) full accountability and responsibility for the CAPC and the CPNP (e.g., activities related to program development, financial tracking and reporting, payment signatories and the non-financial aspects of program/project administration including performance measurement). The Agency's Regional Transformation agenda shifted the Regional Operations program delivery role. It is now a focal point in the Agency for implementing and supporting many national programs and services.

Previous national evaluations of both the CAPC and the CPNP were conducted in 2009. Specific evaluations of activities within each of the regions have been conducted over the last 20 years. Analyses from many of these regional evaluation reports have been included (and are referenced) in this current national evaluation.

2.2 Program Profile

The Agency conducts a range of activities to promote maternal and child health.

Community-based Programming

The intent of both the CAPC and the CPNP is to promote public health by contributing long-term, stable funding to sponsor organizations to improve the health and well-being of vulnerable families in communities across Canada. More specifically, the programs are intended to target those facing the following conditions of risk: low-income families, those who have achieved a low level of education, teenage parents, lone-parent families, Indigenous status peoples living off-reserves, recent immigrants, those living in remote or isolated communities and children experiencing or at risk of developmental delays, and/or social, emotional or behavioural problems. There are total of 425 CAPC and 280 CPNP projects supported though contribution agreements.

Community Action Program for Children (CAPC)

The sponsor organizations that receive funding through the CAPC develop and deliver comprehensive, culturally appropriate early intervention and prevention programs to improve the health and social development of children and their families, as well as increase the recognition and support for communities at-risk, and their needs and interests. The program is based on the principle that communities are well-positioned to recognize the needs of their children and have the capacity to draw together the resources to address those needs. Examples of activities funded through CAPC include programming focussed on: violence and child maltreatment prevention, parenting support and coping skills, social/emotional resiliency, drop-in programs, school readiness programs, outreach and home visiting, nutritional support and collective kitchens, healthy physical activities, and literacy development.

Canada Prenatal Nutrition Program (CPNP)

The sponsor organizations that receive funding through the CPNP develop or enhance existing prenatal services, to address the needs of at-risk pregnant women, their families and children. These organizations seek to improve the health of pregnant women and their infants, promote and support the initiation of breastfeeding and reduce the incidence of babies born with birth weights that are too high or too low. Examples of activities funded through CPNP include: nutrition and lifestyle counselling, food supplements, prenatal and breastfeeding support, infant attachment and child development education, and referral to health and social services.

Knowledge Development and Exchange Activities

Knowledge development and exchange activities take place to varying degrees in all programs and activities within scope of this evaluation. In most cases, these activities include surveillance or research and the development of evidence-based knowledge products, but they may also include the exchange of knowledge through committees and forums.

Maternal and Child Health Knowledge Development and Exchange Activities

The DCY in the Centre for Health Promotion conducts activities that support the development and dissemination of information to people and organizations that can use it to improve health choices, practices, policies and ultimately health outcomes.

The target audiences for these products vary, but they include parents, caregivers and the general public, as well as public health practitioners, front-line workers, academics, and other organizations. While some products are disseminated through the CAPC and the CPNP funded organizations directly, and through health and allied professionals affiliated with these programs, others are promoted and made available to a broader audience through professional associations, web-based promotion activities, webinars and other knowledge exchange events.

These knowledge development activities have been funded through internal allocations including the National Projects Fund (NPF). The NPF, which sunset in March 2014, provided contributions funding to support short-term projects of national scope focused on knowledge development and exchange, including the development of tools and resources, and training on delivery of knowledge products. The resources were aimed at supporting the staff at the CAPC and the CPNP funded organizations, as well as other individuals, programs and organizations working with families to support healthy child development.

Joint Consortium for School Health (JCSH)

Established in 2005, the JCSH is a partnership of federal, provincial and territorial governments from across Canada, which works to promote the wellness and achievement of children and youth in schools. Its objective is to provide leadership and facilitate a comprehensive approach to school health by building the capacity of the education and health systems to work together. It is governed by two Deputy Ministers' committees: the Advisory Committee of Deputy Ministers of Education and the Conference of Deputy Ministers of Health. The Agency is the federal government representative on the Consortium and unlike the provincial and territorial representatives, has observer status and is not a voting member of the Consortium. The Agency contributes $250,000 per year in the form of a grant to support the secretariat and provides advice and guidance from a federal perspective.

Health Behaviour in School-aged Children (HBSC) Survey

The DCY is responsible for leading the $1.1 M, four-year contract with the Canadian HBSC research team located at Queen's University. The Agency, in collaboration with Queen's University, oversees ethics approval, provides input into questionnaire development and national report writing, manages publishing activities, and is responsible for the dissemination and knowledge translation of survey findings.

The HBSC survey was adopted by the World Health Organization (WHO) Regional Office for Europe as a collaborative study in 1983, and is now a four year, cross-national study conducted by an international network of research teams from 44 countries. Survey data is collected from a representative sample of children aged 11 to 15 years, in over 400 schools across Canada, and provides the Agency and Canadians with ongoing data on mental health and healthy relationships, healthy eating and physical activity, bullying, injuries, risky behaviours, and social settings (e.g., home, school and peers). The HBSC survey is Canada's only national school-based health promotion database pertaining to children of this age.

Canadian Perinatal Surveillance System (CPSS)

The CPSS is managed within the Centre for Chronic Disease Prevention (CCDP). Established in 1995, it is an ongoing national health surveillance system, which monitors and reports on key indicators of maternal, fetal and infant health and conducts targeted analyses to advance organizational priorities and mandate. The CPSS products are distributed internally within the Agency and Health Canada, as well as externally to key stakeholders interested in maternal and child health information such as health care professionals, public health officials, administrators, provinces and territories as well as academics. The CPSS costs approximately $900,000 annually.

Canadian Paediatric Surveillance Program (CPSP)

The CPSP is also managed within the Centre for Chronic Disease Prevention. Established in 1996, it monitors diseases and conditions in Canadian children that are relatively low in frequency but are of public health importance, with high disability, morbidity, mortality and economic cost to society. The CPSP gathers data from over 2,500 paediatricians and paediatric subspecialists each month to monitor rare diseases and conditions in Canadian children. The Agency contracts the Canadian Paediatric Society ($520,000 per year) to conduct surveillance and provide reports on issues of paediatric and public health interest such as non-type 1 diabetes mellitus, depressive disorders, sudden infant death, the safety of wheeled baby walkers (now banned in Canada), liquid detergent packets, Fragile X syndrome, congenital rubella syndrome and tuberculosis.

2.3 Program Narrative

According to the Agency's 2014-2015 Performance Measurement Framework (PMF), the expected result of the healthy child development program area is "Participation in Agency funded interventions is positively associated with protective factors for healthy child development." The expected results for the chronic disease and injury prevention program are: "Chronic disease prevention priorities for Canada are identified and advanced" and "Chronic disease prevention practice, programs and policies for Canadians are informed by evidence."

A logic model^{Footnote i}, approved by Treasury Board, has been developed for the CAPC, the CPNP and related knowledge development and exchange activities. For the purposes of this evaluation, modifications were made to highlight the additional activities that were included within scope, more specifically the JCSH, the HBSC survey and the other knowledge dissemination and surveillance activities (CPSS, CPSP). As well, partnerships and collaborations were added as an overarching area of exploration, as they support all programs and activities in accomplishing their outcomes. No changes were made to the program outcomes. This model is referred to as a 'visual representation of activities' (see Appendix 1) to create a clear separation from the approved logic model.

Two main activity areas lead to the achievement of outcomes: community-based programming and knowledge development and exchange.

• The community-based programming activity stream reflects the contribution funding aspect of the CAPC and the CPNP. It is expected to result in the following immediate outcome: parents/caregivers and their children facing conditions of risk participate; and organizations from various sectors collaborate with the CAPC and the CPNP projects to support the needs of the participants (e.g. sharing of staff and financial resources). The at-risk population's participation is expected to lead to two intermediate outcomes: parents/caregivers and their children gain knowledge and build skills to support maternal, child and family health; and children experience developmental benefits while parents/caregivers adopt healthy practices for themselves and their families.
• The knowledge development and exchange activity stream reflects the Agency's information-sharing activities within the Division, as well as the JCSH, the HBSC survey and surveillance activities within scope for this evaluation. The Agency funds and collaborates with other stakeholders to develop and disseminate evidence-based knowledge products (e.g. the Mother's Mental Health Toolkit, the surveillance products from the CPSS and the CPSP, and the HBSC survey reports). Knowledge development and exchange activities are expected to result in an immediate outcome of target groups (e.g., early child development and public health practitioners, academics and parents/caregivers) access knowledge products, which leads to the intermediate outcome of the target groups using the products to support maternal, child and family health.
• As illustrated by the bi-directional arrow between the two activity streams in the visual representation (Appendix 1), community-based collaborations and programs inform and support the dissemination of knowledge development and exchange products. Partners and funded organizations at the community level identify new and emerging needs for information, resources, best practices and interventions that help inform knowledge development and exchange products.

The CAPC and the CPNP, to varying extents, work with other community partners to meet the needs of children and families in a holistic manner. This means that the Agency has direct influence, but not direct control, over these immediate outcomes. Similarly, the Agency has control over the types of knowledge development and exchange activities that are conducted, but only a direct influence over the extent to which target populations (e.g. public health practitioners, academics, parents and caregivers) access or use these knowledge products.

The ultimate outcome is that children and their mothers and families experience improved health (e.g. healthy birth weights, increased initiation and duration of breastfeeding, positive parent-child interactions, improved family functioning). This is intended and expected to occur in the longer-term through the achievement of the shared outcomes.

The connection between these activity areas and the expected outcomes is depicted in the visual representation (Appendix 1). The evaluation assessed the degree to which the defined outputs and outcomes were being achieved over the evaluation timeframe.

2.4 Program Alignment and Resources

The programs are part of the Agency's Program Alignment Architecture (PAA) 1.2.2.1: Healthy Child Development, with the exception of the CPSS and the CPSP which fall under the PAA sub-activity 1.2.3 Chronic Disease and Injury Prevention.

The combined programs' planned budgets for the years 2010-2011 through 2014-2015 are presented below (Table 1). Overall, the CAPC, the CPNP and associated activities had a planned budget of approximately $450 million over these five years.  The variance between program budgets and expenditures are found in Table 13. A more detailed breakdown (by program/activity area) of planned budgets and expenditures can be found in Appendix 7.

In 2014-2015, there was a total of 64 staff in the Health Promotion and Chronic Disease Prevention Branch supporting the CAPC, the CPNP and associated activities.

• Centre for Health Promotion and Regional Operations: Administration and management of CAPC, the CPNP and the associated knowledge development and exchange activities consisted of both National Office and Regional Operations staff.  The JCSH and the HBSC survey were supported through National Office staff.  The staff complement of 53 full time equivalents (18 national and 35 regional) included: national and regional program managers, policy analysts, performance measurement analysts and program consultants.
• Centre for Chronic Disease Prevention: Support for the CPSP and the CPSS, consisted of staff from the Maternal and Infant Health Section within the Surveillance and Epidemiology Division. The staff complement of 11 full time equivalents included: a chief, a manager, a research scientist, a project coordinator, research/data management analysts, and epidemiologists.

3.0 Evaluation Description

3.1 Evaluation Scope, Approach and Design

The scope of the evaluation covered the period from April 2010 to March 2015, and included maternal and child health activities funded within the Health Promotion and Chronic Disease Prevention Branch. Considering the long-standing nature and materiality of these community-based programs, there was a focus on the CAPC, the CPNP and their related knowledge development and exchange activities. It also included the following associated activities which are most closely related to the knowledge development and exchange stream^{Footnote ii} of this evaluation: the HBSC survey, the JCSH, the CPSS and the CPSP. As well, partnerships and collaboration was an overarching area of exploration, as they support the programs and activities in accomplishing their outcomes.

A number of other program and activity areas in the Branch contribute to improving the health of children, however these were out of scope of this evaluation as they will be or have recently been evaluated separately. These programs and activities include: Fetal Alcohol Spectrum Disorder, Family Violence Initiative, AHSUNC program, Oral Health, Cancer in Young People in Canada Surveillance Program, and Child Maltreatment Surveillance.

The evaluation issues were aligned with the Treasury Board of Canada's Policy on Evaluation (2009) and considered the five core issues under the two themes of relevance and performance, as shown in Appendix 3. Corresponding to each of the core issues, specific questions were developed based on program considerations and these guided the evaluation process.

An outcome-based evaluation approach was used for the conduct of the evaluation to assess the progress made towards the achievement of the expected outcomes, whether there were any unintended consequences and what lessons were learned. The Treasury Board's Policy on Evaluation (2009) also guided the identification of the evaluation design and data collection methods so that the evaluation would meet the objectives and requirements of the policy. A non-experimental design was used based on the evaluation plan, which detailed the evaluation strategy for this program and provided consistency in the collection of data to support the evaluation. As a non-experimental design, the evaluation relied on correlation to demonstrate effect, without implying causation. As such, the evaluation has been designed to demonstrate the likely contributions of the programs to the expected outcomes, rather than demonstrate direct causal links between the programs and outcomes.

Data collection started in April 2015 and concluded in November 2015. Data for the evaluation was collected using various methods, which were: document review, financial data review, key internal and external informant interviews, literature review, case studies and performance data review. Performance measurement data collected by the program, including Children's Programs Performance Measurement Tool (CPPMT 2013-2014) and participant surveys (2015), were further analyzed. More specific detail on the data collection and analysis methods is provided in Appendix 3. In addition, data were analyzed by triangulating information gathered from the different methods listed above. The use of multiple lines of evidence and triangulation were intended to increase the reliability and credibility of the evaluation findings and conclusions.

3.2 Limitations and Mitigation Strategies

Most evaluations face constraints that may have implications for the validity and reliability of evaluation findings and conclusions. The following table outlines the limitations encountered during the implementation of the selected methods for this evaluation. Also noted are the mitigation strategies put in place to ensure that the evaluation findings can be used with confidence to guide program planning and decision-making.

4.0 Findings

This section provides a summary of the findings organized under two broad headings:

• Relevance: the need, priorities and federal public health role in maternal and child health, in particular with respect to vulnerable populations
• Performance: the effectiveness, efficiency and economy of the Agency's activities in this area.

4.1 Relevance: Issue #1 - Continued Need for the Program

There are particular risk groups within the population that are more likely to experience poor maternal and child health and well-being. Evidence exists to support continued investment in these areas to close the gap for children and their families in these risk groups.

As discussed in Section 2 - Program Description, the CAPC and the CPNP were created to help improve the health and well-being of Canadian children, particularly those considered at-risk and therefore more likely to experience poor health outcomes, developmental difficulties, and social, cognitive and behavioural problems. These experiences not only impact them as children but also impact their health, socioeconomic status and developmental opportunities as they age. While most children in Canada experience conditions necessary for healthy outcomes, there are children who live in conditions of risk.^{Footnote 1Footnote 2Footnote 3Footnote 4}

Children and Families Living in Conditions of Risk

Research on the social determinants of health indicates that certain groups of people are more likely to experience poor health outcomes.^{Footnote 5} These disparities are not necessarily caused by lack of medical treatments or lifestyle choices, but rather by living conditions and other factors that impact these individuals and their experiences. The social determinants of health include income and income distribution, education, unemployment and job security, employment and working conditions, early childhood development, food insecurity, housing, social exclusion, social safety network, access to health services, Indigenous status, gender, race, and disability.^{Footnote 6} All of these determinants, to varying degrees, can play a role in how well children develop, the experiences they have, and the likelihood that they become well-adjusted adults.^{Footnote 7}

Highlighted below using data from Statistics Canada - particularly the National Longitudinal Survey of Children and Youth or NLSCY (2011), the Canadian Community Health Survey or CCHS (2011-2012) and the Vital Statistics Birth Database (2011) - are the percentage of people in Canada impacted by the particular conditions of risk (or social determinants of health) that the CAPC and the CPNP are currently measuring using the 2015 participant surveys to better understand their target populations - in particular income, education, Indigenous status, marital status, recent immigrants, teenage parents, and food insecurity (Table 3). These at-risk individuals are the focus of the community-based programs being evaluated.

Impact of Conditions of Risk on Health and Well-being

When families and individuals are impacted by the conditions of risk discussed above, research shows that they are at increased risk of an unhealthy lifestyle that may negatively impact their health and the health of their children.^{Footnote 8} Setting children up for a healthy, successful life begins before they are born.^{Footnote 9} A mother's unhealthy lifestyle during pregnancy and immediately following birth (e.g., prenatal nutrition, smoking, alcohol consumption, breastfeeding) impact the health of her baby, in terms of gestational age, birth weight and early infant nutrition, all factors that can impact the health and well-being of an individual through their lifespan.^{Footnote 10}

Highlighted below, using data from Statistics Canada - particularly the 2014 Canadian Community Health Survey - is the prevalence of some of these behaviors within the general population, as well as within populations particularly impacted by conditions of risk (Table 4). Rates of breast feeding are lower than those of the general population for populations with low income, low education, single parents and teen parents. Rates of smoking during pregnancy are higher for low income, low education, Indigenous status, single parents and teen parents. Teen parents also drank more during pregnancy.

Numerous studies have shown a strong link between low income and poverty leading to an increased risk for preterm births, low birth weight, and small-for-gestational-age babies. Experts have hypothesized that preterm birth and/or intrauterine growth restriction may, to a significant extent, be explained by greater exposure of low income pregnant women to accumulated chronic stressors, including crowded home environments, unemployment, single-parent households, less social support and financial problems.^{Footnote 11Footnote 12Footnote 13Footnote 14}

After birth, the environment of the child continues to influence their development. Children do best in environments with nurturing caregivers, positive learning opportunities, good nutrition and opportunities for social interaction with other children.^{Footnote 15} Research indicates that children living in conditions of risk are more likely than other children to experience challenges such as food insecurity, lack of stimulation and play, family violence and unintentional injuries.^{Footnote 16}

These experiences, which are more likely to occur for families and children living in conditions of risk, can result in a variety of negative effects that impact a family's ability to be healthy and productive, and a child's ability to develop into a healthy and productive adult. Some of these effects include chronic illness, depression, chronic stress, unemployment, and in particular for children, difficulties with behavior and socialization, and increased odds of disengagement from school,^{Footnote 17Footnote 18Footnote 19Footnote 20Footnote 21Footnote 22Footnote 23} resulting in children who grow into adults with lower educational attainment, weaker literacy and communication skills, fewer employment opportunities and poorer overall physical and mental health.^{Footnote 24Footnote 25} Conversely, parents who understand how children develop and who use effective parenting techniques raise healthier and happier children.^{Footnote 26}

Investments in Children Yield Benefits throughout the Lifespan

Research in population health indicates that investment early on in life has the greatest potential for improving health and well-being. The aim of this upstream investment is to prevent negative consequences from occurring in the future by addressing the root causes.^{Footnote 27} Early childhood development plays a crucial role in ensuring that children are physically, socially, cognitively and emotionally set-up to succeed in life.^{Footnote 28Footnote 29} Experiences during this time, and even before birth, can influence health, education and economic prospects throughout a lifetime. Although it is possible to mitigate negative events and impacts in the later stages of the life course, the greatest impact can be made during the early years of childhood.^{Footnote 30Footnote 31}

Early childhood development interventions, such as education and care, parenting support, and initiatives that attempt to reduce the impacts of poverty, yield benefits throughout life that are worth many times the original investment.^{Footnote 32} In fact it has been found that those societies, either rich or poor, that invest in children and families in the early years, have more literate and numerate populations, as well as increased health status and low levels of health inequality.^{Footnote 33}

As detailed in section 4.5 - Economy and Efficiency, the greatest impacts of investment in early childhood are seen as children become adults. It has been estimated that for every dollar spent in the early years, a savings of between three dollars and nine dollars can be expected in terms of health, social, and justice services in the future.^{Footnote 34Footnote 35} It is clear that by investing in children and their well-being during the early stages of life, societies can have a tremendous influence in promoting positive outcomes and minimizing or mitigating the impact of adverse childhood experiences and events.^{Footnote 36Footnote 37}

4.2 Relevance: Issue #2 - Alignment with Government Priorities

The promotion of maternal and child health is a priority for the Government of Canada and the Agency.

Government of Canada

The CAPC, the CPNP and other associated activities linked to maternal and child health are aligned with the broader Government of Canada priority to safeguard Canadian families and their communities.

This priority has been highlighted in a number of previous Speeches from the Throne. In 2013, in particular, the Government of Canada committed to developing universal legislation aimed at protecting "the most vulnerable of all victims, our children" and stated that "Families are the cornerstone of our society. Families raise our children and build our communities. As our families succeed, Canada succeeds." This document also underlined that "Canadian families expect safe and healthy communities in which to raise their children. They want to address poverty and other persistent social problems, access safe and reliable infrastructure, and enjoy a clean and healthy environment".^{Footnote 38}

Similarly, with respect to Indigenous health programs, these commitments are also echoed in recent federal budgets. Budget 2015 provides for a number of supports to assist Indigenous populations, including children. Budget 2010^{Footnote 39} renewed funding for a number of Indigenous health programs, including maternal and child health programs.

These programs and activities are aligned with the Government of Canada's priorities in relation to the promotion of maternal and child health, as articulated in numerous previous key federal policy documents including: the Early Childhood Development Agreement (2000), A Canada Fit for Children (2004), the Pan-Canadian Healthy Living Strategy (2005), and Curbing Childhood Obesity: The Federal, Provincial and Territorial Framework for Action to Promote Healthy Weights (2010).

Internationally, maternal, newborn and child health remains an international development priority and is one of the means by which the Government of Canada fulfills its international commitments to the United Nations Convention on the Rights of the Child (UNCRC) ^{Footnote 40} and the Millennium Development Goals.^{Footnote 41}

As noted in Section 2.1 - Program Context, in 1991 Canada ratified the United Nations Convention on the Rights of the Child to become bound by the provisions of this Convention. Since 1993 and 1995, respectively, the CAPC and the CPNP have provided a platform for the Government of Canada to fulfil many of its strategic commitments and priorities related to the promotion of maternal and child health. For example, CPNP is aligned with the principles of Article 18 of the UNCRC, which affirms the role of parents as the primary caregivers who should be provided with the supports necessary to care for their children.^{Footnote 42}

Public Health Agency of Canada

The Agency has identified addressing health inequities through a variety of maternal and child health activities as a key priority in many corporate planning documents.

Numerous Agency Reports on Plans and Priorities within the scope of this evaluation (2010-2015) highlight how the Agency works to improve population health and well-being and reduce health inequalities. To address this priority, the Agency plans to "…help vulnerable children get a good head start by supporting programs that build and strengthen positive social, emotional and mental health in early childhood and throughout their life."^{Footnote 43} Also, the need to "…scale up best practices in healthy childhood development to improve the overall social, mental, and physical health of children (aged 0-6) and their parents who participate in the Agency's community-based approaches"^{Footnote 44} is also highlighted.

The Agency's Corporate Risk Profile^{Footnote 45} highlights the importance of building and strengthening protective factors in early childhood contributing to lifelong resiliency by focusing the Agency's activities towards the upstream prevention interventions. The Corporate Risk Profile also underlines the importance of leveraging multi-sectoral partnerships to increase the reach and impact of programs.

The objectives of maternal and child health activities align with the Agency's key priorities outlined in its Strategic Horizons 2013-18^{Footnote 46} including "strengthened public health capacity and science leadership" through "…enhancing public health surveillance of non-communicable (chronic) disease risk factors, maternal and child health, injuries and infectious diseases…" and its role as "an evidence-based organization that creates, translates and shares knowledge for the use and benefit of decision-makers and stakeholders." Similarly, the Agency's Strategic Plan 2007-2012^{Footnote 47} sets out strategic objectives, including: "addressing major public health challenges, as well as key determinants of health, health disparities among Indigenous status peoples, children, and seniors, and gaps in public health capacity."

Finally, the 2009 Chief Public Health Officer (CPHO) Report on the State of Public Health in Canada underscores the Government of Canada's and the Agency's commitments to the well-being of children. Specifically, this report emphasizes issues of concern that affect the health of Canadian children, including socioeconomic status, developmental opportunities and prenatal risks.

Alignment between Priorities and Current Activities

To address these priorities, the Agency, Branch and Centre have developed a number of plans and strategies.

At the Agency level, the Agency Plan to Advance Health Equity (2013-2016) was developed with a priority to strengthen the evidence base through consistent collection, analysis and reporting on social determinants of health and health inequalities. It contributes to Canada's commitment for the implementation of social determinants of health approach, as outlined in the Rio Political Declaration on Social Determinants of Health (2011). The Agency also developed a Surveillance Strategic Plan 2013-2016, which sets out, among other priorities, the Agency's surveillance transformation priorities.

Branch and centre strategic and operational plans reflect priorities set out by the Agency and the Government of Canada. One of the strategic directions in the Health Promotion Chronic Disease Prevention (HPCDP) Branch Strategic Framework 2010-2015 is "…providing a stronger evidence base for taking on strategically important health issues and their determinants. Application of these approaches includes assessing how health status varies across the population, understanding that the determinants of health have direct and indirect consequences on health outcomes and utilizing this knowledge to develop and implement policies and actions to improve population health and well being and reduce health inequalities." Further, the Centre for Health Promotion's Strategic Plan (2014/15-2016/17) states that addressing maternal, child and family health is a key strategic direction.

4.3 Relevance: Issue #3 - Alignment with Federal Roles and Responsibilities

Various legislative and policy and program authorities identify that the federal government has a role in, and a responsibility to conduct, activities that target at-risk populations through maternal and child health promotion and disease prevention initiatives, as well as activities related to public health surveillance, providing leadership, and sharing knowledge and best practices.

The Agency addresses the federal government's broad role in health promotion that is outlined in foundational legislation. The Canada Health Act describes one of the main objectives of Canadian health care policy as being to "protect, promote, and restore the physical and mental well-being of residents of Canada". Within this mandate, the Department of Health Act gives the Minister of Health powers, duties, and functions relating to "the promotion and preservation of the physical, mental, and social well-being of the people of Canada". Finally, the Public Health Agency of Canada Act specifies that some of the activities the Government of Canada can undertake include health protection and promotion, population health assessment, and health surveillance.

Expert studies that helped identify gaps in the public health system and ultimately helped shape the Agency's mandate (i.e., Lalonde 1974, Kirby 2002, Naylor 2003, Ottawa Charter 1986) also clearly comment on the federal government's role in health promotion, indicating that the federal government should have a specific role to ensure that health promotion activities are sensitive to health equity considerations. In addition to these broader roles, the Government and the Agency have roles in health promotion and disease prevention relating specifically to women and their children. The Agency also has specific roles related to public health surveillance, leadership and the sharing of knowledge and best practices.

Health Promotion and Disease Prevention - Targeting At-risk Populations

In 1993 and 1995, respectively, program authorities for the CAPC and the CPNP were established along with a number of other programs to improve the health and lives of children, particularly at-risk children. As part of these authorities, the Agency has the role to help local community groups in high-risk communities address the health and developmental needs of at-risk children in the pre-natal, postnatal, infant, and early childhood stages.

As discussed in Section 4.1 - Continued Need, those who are living in conditions of risk often require more support to ensure that their children have a healthy start to life. Authorities indicate that those at the greatest risk have the highest needs, and therefore they have been established as the target population for these programs. As the issues facing those at risk are of a challenging and complex nature, program authorities establish ongoing funding in order to ensure these issues are sufficiently addressed. The program authorities also specified that the programs would be flexible, allowing communities to address the needs of the at-risk populations located in their areas.

Surveillance

Legislation provides the primary basis for the Agency to conduct surveillance activities. It is clearly laid out in the Department of Health Act that the Minister has powers, duties, and functions in matters of investigation and research into public health. The Public Health Agency of Canada Act reinforces this role by giving the Governor in Council the power to make regulations regarding the collection, analysis, interpretation, publication, and distribution of information relating to public health.

As confirmed in policy and program authorities for the CAPC and the CPNP, the Agency has a role to conduct national surveillance on the perinatal health of women and infants. These activities occur through the CPSS, which is an ongoing national health surveillance system that aims to contribute to improved health for mothers and infants in Canada.

Other surveillance activities reflect the Agency's legislated role in surveillance. These include:

• In 1996, a contribution agreement was established with the Canadian Paediatric Society to create the CPSP.
• Since 1989, Canada has participated in the HBSC survey.  The Agency has a contract with the Canadian HBSC research team currently located at Queen's University.

Leadership

There is a legislative basis for the Agency's leadership role in the health promotion of mothers and their children. Under the Department of Health Act, the Minister has powers, duties, and functions to cooperate with provincial authorities in coordinating efforts. The Agency was established under the Public Health Agency of Canada Act to assist the Minister and also to foster collaboration regarding public health with provincial and territorial governments, other countries, and international organizations.

Foundational policy and program authorities for the CAPC and the CPNP further detail the Agency's leadership role in establishing and maintaining joint protocol agreements with provincial and territorial ministries. These protocol agreements provide the mechanism for the federal government to be involved in the delivery of services under provincial and territorial jurisdiction. They outline how the federal government will work with the provincial and territorial governments to ensure complementarity as opposed to duplication of programming.

The Agency also plays a leadership role through their contributions to the JCSH, for which specific program authorities exist. The JCSH, established in 2005, is a partnership of federal, provincial and territorial governments from across Canada, which works to promote the wellness and achievement of children and youth in schools. The Agency is the federal government representative on the JCSH and serves in a funding and advisory capacity. The Agency's contribution facilitates the bringing together of key representatives from provincial and territorial government departments or ministries responsible for health and education to build partnerships and exchange knowledge on activities related to school health.

Sharing of Knowledge and Best Practices

A number of program authorities highlight the sharing of knowledge and best practices as a key role to be carried out by the federal government in the context of maternal and child health.

In particular, original program authorities for the CAPC and the CPNP indicate that the Agency has a role to ensure that new knowledge gained through these programs will be nationally distributed, that a body of knowledge about effective programming will be developed, and that an exchange of information among local and regional projects will be facilitated. As part of these program authorities, the National Projects Fund was established in 1997 to provide national, regional, and local organizations with financial support to conduct specific, short-term activities by providing them with tools, resources, and training on specific issues. While the National Project Fund was sunset in 2014-2015, the knowledge products developed continue to be disseminated and used. More information about the NPF can be found in Section 4.4.2 - Knowledge Development and Exchange.

The Centre for Health Promotion facilitates the sharing of knowledge and best practices through their work on the JCSH, the HBSC survey, and priority issues related to maternal and child health. Within the Centre for Chronic Disease Prevention, the activities of the CPSS and the CPSP contribute to the body of knowledge and the dissemination of best practices on maternal and child health.

Alignment between Role and Current Activities

Over the last 20 years, the roles and activities of the CAPC and the CPNP and associated activities have remained consistent with their original authorities. The Agency's current activities related to the programs are aligned with defined program roles in the area of health promotion and disease prevention for children and mothers, as well as the related areas of surveillance, leadership, and sharing of knowledge and best practices.

• In the area of promoting the health and well-being of mothers and children, the Agency continues to support the CAPC and the CPNP funded organizations.
• The Agency also manages the CPSS and the CPSP.
• With regards to the sharing of knowledge and best practices, the Agency supported the development of a number of knowledge products including the Mother's Mental Health Toolkit, a breastfeeding workbook, a parent brochure and video on infant safe sleep, and the Nobody's Perfect parenting program. The Agency also facilitated further knowledge dissemination activities through the JCSH, HBSC survey, the CPSS and the CPSP.
• The Agency continues to demonstrate its leadership role through the joint management of the CAPC and the CPNP, as well as its work on the JCSH. More details about these activities can be found in Sections 4.4.1 - Partnerships and 4.4.2 - Knowledge Development and Exchange.

The Agency's role in promoting the health of children and mothers is complementary to that of provinces and territories and other federal government departments.

The evaluation assessed duplication, complementarity and gaps for both the CAPC and the CPNP, as well as for associated activities, particularly surveillance.

Provinces and Territories, Non-Governmental and Community Organizations

Many stakeholders across Canada are involved in promoting health and well-being and preventing diseases among mothers and their children. When the CAPC and the CPNP were established in the 1990s, one goal of these programs was to mobilize support of other jurisdictions through partnerships on behalf of children. With organizations at all levels of government, as well as non-governmental and community organizations, working to help children and mothers, some overlap in activities is occurring; however, due to the vastness and complexity of the issue, it is appropriate that many organizations are working in tandem with these populations.

A review of available provincial and territorial, non-governmental organizations, and community program websites, strategic plans and other key documents, as well as key informant interviews and case studies, indicated that the CAPC and the CPNP funded site programming is generally targeted to at-risk populations, whereas programs and services offered by other organizations are more often universal. For example:

• Strategic plans developed by the provinces and territories, such as the Healthy Child Manitoba Strategy and Ontario's Healthy Kids Strategy, are the governments' ongoing commitment to support early childhood development in the general population (further examples can be found in Appendix 4).
• Provinces and territories create knowledge development and exchange products that are directed at the general public through promotional videos and online on topics such as breastfeeding, safe sleep, parenting tips, and facilitating early childhood learning.
• Non-governmental organizations, such as UNICEF and the YMCA/YWCA, have strategic plans and knowledge development and exchange products that are universal in nature, focusing on general parenting tips and overviews of early childhood development.
• Community programs reviewed by the evaluation, particularly the Ontario Early Years and British Columbia Early Years programs, were found to be targeted toward the general population, irrespective of socio-economic status.

While the issue of maternal and child health is vast and complex and some overlap is inevitable, efforts should be made to collaborate where possible to ensure that unnecessary duplication is not occurring and that efficiencies are maximized. When the CAPC (1993) and the CPNP (1995) were established, protocol agreements with the provinces and territories were developed to clearly outline roles and responsibilities. The intention of these protocols was to ensure that the services provided by the federal government were complementary to existing services available in the communities, instead of creating duplication.

At the same time, Joint Management Committees (JMC) were also established between the federal government and each of the provinces and territories to reduce duplication and ensure that organizations were working collaboratively within the complex environment of maternal and child health. While still active, a recent Internal Audit highlighted that the role of the JMCs has shifted over time and further exploration of their current role is warranted. Opportunities to enhance the efficiency of these governance structures are discussed in Section 4.5 - Economy and Efficiency.

Federal Government Departments

A review of strategic plans, websites and initiatives^{Footnote vi} of other federal government departments (as available) in the area of children and mothers indicate that the role played by other federal government departments is complementary to that of the Agency.

The work being done in other government departments tends to target specific populations or specific niches that are outside the scope of the CAPC and the CPNP. In particular, Health Canada's First Nations and Inuit Health Branch conducts similar activities to the CAPC and the CPNP, in most cases with the authorities for their programs having been established in the same documents as the CAPC and the CPNP; however, their target population is Indigenous individuals on reserve, where the CAPC and the CPNP serve Indigenous populations living off-reserve. Key informants indicated that opportunities for integration are limited, but that representatives from both groups sit on a steering committee that provides an opportunity for information sharing.

A limited review of available strategic plans and websites of other government departments such as Indigenous and Northern Affairs Canada, Immigration, Refugees and Citizenship Canada, and the Canadian Institutes for Health Research, reveal that while they work in the area of maternal and child health, their activities appear to be focussed in areas that are different from the CAPC and the CPNP. More information about the work of these other government departments can be found in Appendix 4. Case study key informants indicated that there may be some value in further collaboration and coordination among federal government departments as other government departments (e.g., Immigration, Refugees and Citizenship Canada) offer services at the same sites as the CAPC and the CPNP.

The CPSS conducts surveillance activities through collaboration with Statistics Canada. Internal and external key informants agree that this collaboration is complementary due to the nature of their relationship. The CPSS accesses vital statistics data and hospitalization data through existing corporate-level agreements with Statistics Canada and the Canadian Institute of Health Information, respectively. The CPSS uses Inter-department Letters of Agreement to obtain customized files with optimal maternal and infant health surveillance data. While both groups produce reports on topics of relevance to maternal and child health, Statistics Canada is more focused on the statistical analysis of available data, whereas the CPSS uses the Statistics Canada data but provides the public health context.

Within the Agency, there are also a number of other program areas that conduct activities in the area of maternal and child health, including the Aboriginal Head Start in Urban and Northern Communities (AHSUNC) program, the Family Violence Initiative, and work in the area of fetal alcohol spectrum disorders and injury prevention. Of particular interest is the AHSUNC program as it is co-located with the CAPC and the CPNP at a number of sites and conducts similar activities with Indigenous children living-off reserve in the same age range as the CAPC, but with a focus on school readiness in a preschool-based setting. There are indications that the CAPC, the CPNP and the AHSUNC work together at the national level. For example, some performance measurement tools (i.e., the CPPMT) are combined for all programs and internal key informants indicated that the establishment of program governance structures (i.e., Program Governance Committee and the Program Management Committee) in July 2014 provides strategic policy direction for CAPC, CPNP, AHSUNC and Fetal Alcohol Spectrum Disorders activities, to ensure that the programs are working towards the same goals.

The CAPC, CPNP, AHSUNC and other maternal and child health activities are located in the Centre for Health Promotion in the Health Promotion and Chronic Disease Prevention Branch. Key objectives and activities for these program areas are integrated in the Branch's 2014-2015 to 2016-2017 Strategic Plan. There may be opportunities for these other program areas to leverage the resources of the CAPC and the CPNP to advance their initiatives. As key informants have indicated, it may be worth exploring further ways to use the CAPC and the CPNP as a platform to advance other Agency priorities (see Section 4.5.1 - Economy and Efficiency).

4.4 Performance: Issue #4 - Achievement of Expected Outcomes (Effectiveness)

4.4.1 Partnerships - To what extent did organizations from various sectors collaborate?

CAPC and CPNP funded organizations have created, maintained and expanded multi-sectoral and multi-jurisdictional collaboration. These partnerships have led to enhancements in program delivery.

Partnerships are part of the foundation upon which the CAPC and the CPNP were established in the 1990s. More recently, the Centre for Health Promotion embodied this principle in its 2014 Strategic Plan: "Partnerships sustain community capacity to reach vulnerable populations and promote maternal, child and family health."

Partnerships and Collaboration at the CAPC and the CPNP Project Site Level

In general, the CAPC and the CPNP funded organizations have been able to create, maintain and expand on multi-sectoral partnerships.^{Footnote vii} These partnerships form part of a comprehensive network of community supports that connect families and children in need with additional health and social services. Partner contributions can range from financial support, to joint programming, to in-kind resources (see Section 4.5 - Economy and Efficiency).

The 2013-2014 CPPMT survey highlighted that 90% of the CAPC and the CPNP funded organizations have worked with more than three different types of partners in their community. Table 5 below provides a list of the five most frequent types of partner organizations with whom the CAPC and the CPNP funded organizations engage. The table shows that health organizations (including community health centres, clinics, public health units) are the most common type of partner for 80% of the CAPC and 92% of the CPNP funded organizations respectively. On average the CAPC funded organizations had 24 community partners while CPNP funded organizations had 26 community partners.

Key informants and case studies indicated that maintaining and expanding partnerships has been critical for the sustainability of many funded organizations. Key informants at funded organizations underlined that this is particularly true since the amount of funding from sponsoring programs, such as the CAPC and the CPNP, has remained largely unchanged over many years.

Benefits of Partnerships at the CAPC and the CPNP Project Site Level

Partners support the CAPC and the CPNP funded organizations in providing a complementary continuum of services addressing child, maternal and family health needs. Key informants have indicated that partnerships have allowed the funded organizations to create programs and structures that enhance their ability to deliver their programming in support of maternal and child health and secure additional resources (for more on leveraging, see Section 4.5 - Economy and Efficiency).

In all seven case studies conducted, project site staff spoke to the benefits of partnerships in providing a seamless service to clients and connecting them with other social services as they may be required, thus removing the barriers and "wrapping" services around clients' needs. Multiple partners indicated being very satisfied and related that the CAPC and the CPNP delivery sites are essential to the community's well-being.

Previous regional evaluations spoke to the benefits of partnerships. A regional evaluation of the CAPC (Saskatchewan, 2012) found that community partners were very satisfied with their partnership with the CAPC funded organizations and found them of great value. It encouraged the CAPC funded organizations to maintain strong community partnerships and to work to build additional partnerships. In Alberta (2013-14), The CAPC and the CPNP monitoring report indicated that many funded organizations were involved with other community groups and organizations, benefitted from loaned staff and donations of physical space and resources, and learned through educational workshops provided by partner organizations. It concluded that partnerships may be the single most important factor in the successful delivery of the CAPC and the CPNP.

Document reviews and key informant interviews highlighted that, with Agency support, national and regional linkages have been established among the CAPC and the CPNP funded organizations. Examples of these linkages include: the Ontario Network of the CAPC and the CPNP Projects, the Alberta CAPC and CPNP Coalition, the BC CAPC Society, the North Eastern Network for Children and Youth in Cape Breton, and the National Network of the CAPC and the CPNP Projects.

Valuable partnerships were established and maintained through the associated activities in maternal and child health.

Other associated maternal and child health activities have been effective in developing and leveraging key partnerships and collaborations to support effective program implementation.

Joint Consortium on School Health (JCSH)

The Agency supports the JCSH. This initiative is a unique partnership comprised of policymakers in federal, provincial and territorial government departments or ministries in two sectors - health and education - which work together to promote the wellness and achievement of children and youth in the school setting.

Internal and external key informants indicated that the various activities undertaken by the JCSH have increased pan-Canadian and inter-sectoral collaboration. For example, at the working level, each province or territory has named a School Health Coordinator. The JCSH School Health Coordinators' Committee has served as a pan-Canadian forum to advance comprehensive school health initiatives across Canada. The JCSH provides a unique opportunity for the Agency to interact with the provincial and territorial departments of education.

Key informants indicated that JCSH members and staff are seen as "connectors" that help organizations locate and work collaboratively with other organizations. The Consortium has been engaged in many initiatives with many partners.

Partners have included:

• National: Canadian Education Association, Canadian Partnership Against Cancer, Canadian Teachers Federation, Chronic Disease Prevention Alliance of Canada, ParticipACTION, Physical and Health Education Canada, and Promoting Relationships and Eliminating Violence Network (PREVnet).
• International, including other national governments and multi-national organizations: Australia, United States, England, Scotland, New Zealand, Pan-American Health Organization, Schools for Health in Europe, United Nations and the WHO.

Health Behaviour in School-Aged Children (HBSC) Survey

The HBSC survey is an international school-based survey that is carried out in Canada as well as in 43 other countries in Europe and North America. In Canada, the multi-university research team involves researchers, staff and students and is currently housed at Queen's University in the Social Program Evaluation Group as well as McGill University in Montreal. The researchers, policymakers, and collaborators involved in the HBSC survey have partnered to increase understanding of young people's health and well-being, including the behaviours that influence health.

One of the activities of the HBSC survey is to develop partnerships with other organizations that deal with adolescent health, in order to support health promotion efforts with populations of school-aged children. Some key partnerships have included: other academic institutions (University of Waterloo, McGill University, and University of British Columbia), the WHO, and the Student Commission of Canada.

A key partnership for the HBSC survey has been with the provincial and territorial ministries of health and education, through the JCSH. Collaboration included:

• The JSCH members provide advice and support to the HBSC researchers through the HBSC Research Advisory Committee. Key informants from the HBSC team indicated that this advice and support helps keep the HBSC survey grounded in terms of serving the needs of health and education professionals.
• The JCSH members promote the HBSC survey in their respective jurisdictions so that HBSC researchers can gain access to school jurisdictions and individual schools. Key informants have indicated that this collaboration resulted in the expansion of the 2010 (9,000 to 26,000) and 2014 (26,000 to almost 30,000) HBSC samples. Ultimately, HBSC researchers have been better able to collect provincial or territorial level data on the basis of provinces or territories opting in to this survey thus increasing the survey sample size and geographic distribution. In cases where a province or a territory has a particular area of interest or a focus for a report, or they would prefer a much larger sample so they could look at regions within a province, for a little additional investment to oversample, they can obtain relatively low cost quality data on current health issues.
• Further, once a report on the HBSC data has been produced, the JCSH supports the dissemination of the HBSC information by ensuring that these reports get back to their jurisdictions so that they may serve as an evidence base for policy direction and decision-making.

Canadian Perinatal Surveillance System (CPSS)

The CPSS, managed within the Centre for Chronic Disease Prevention, is a continuous and systematic process of data collection, analysis and interpretation of information that is carried out in partnership with Statistics Canada, the Canadian Institute for Health Information (CIHI), provincial and territorial governments, health professional organizations and university-based researchers.

The nature of these partnerships is highlighted below:

• The CPSS extensively uses vital statistics data and hospitalization data available to the Agency through existing corporate-level agreements with Statistics Canada and CIHI, respectively. It also uses Interdepartmental Letters of Agreement (ILAs) with Statistics Canada to obtain customized files to conduct optimal maternal and infant health surveillance (i.e. linked birth-death file, linked maternal mortality file).
• Particularly for congenital anomalies surveillance, the CPSS implements Memoranda of Agreement (MoAs) with provincial and territorial jurisdictions across the country for data sharing and strengthening surveillance capacity nationally.
• Since its inception in 1995, a Steering Committee has provided ongoing expert advice relevant to perinatal health surveillance to the Agency. The multidisciplinary and multi-sectoral Steering Committee members include expert representatives of national health professional associations, the provincial and territorial governments, consumer and advocacy groups and federal government departments, as well as Canadian and international experts in perinatal health and epidemiology.

Internal key informants indicated that the CPSS complements the community-based programs (CAPC and CPNP) by providing the evidence base for the development of health policies and programs inside the Agency. It similarly supports other maternal and child health programs within the Centre for Health Promotion by providing data to support the Fetal Alcohol Spectrum Disorder Initiative, and using specific reports (e.g. Maternity Experiences Survey) to update national guidelines, such as the Family-Centred Maternity and Newborn Care Guidelines. In addition, perinatal health surveillance informs prevention efforts in other areas such as chronic disease given that health events occurring as early as during fetal development (e.g., weight gain and diabetes during pregnancy) can lead to ill health later on, in both the mother and baby.

The CPSS also collaborates on special studies and initiatives with key stakeholders outside of the Agency. For example, it conducted analyses of infant deaths to determine the SIDS rates in Canada which supported the Joint Statement on Safe Sleep: Preventing Sudden Infant Deaths in Canada (2011). This statement was developed in collaboration with North American experts in the field of sudden infant deaths, the Canadian Paediatric Society, the Canadian Foundation for the Study of Infant Deaths, the Canadian Institute of Child Health, and Health Canada, with input from provincial/territorial, national, and regional public health stakeholders from across the country.

Canadian Paediatric Surveillance Program (CPSP)

The CPSP is a joint project between the Agency and the Canadian Paediatric Society (CPS). Through this partnership, the CPSP is uniquely able to reach out to the paediatric community to get information and, in turn, address the information needs of both Agency and external epidemiologists and scientists.

Priorities of public medical health importance are determined jointly by the CPS and the Agency, with support from the Steering Committee (which oversees the program and meets bi-annually to review proposals) as well as a Working Group. The Steering Committee consists of issue experts from hospitals, communities as well as different subspecialties within paediatrics.

In terms of partnerships within the broader Health Portfolio, the Agency has an annual Interdepartmental Letter of Agreement (ILA) with Health Canada related to the use of the CPSP to monitor and report on adverse drug reactions in children. As well, there is internal collaboration with related Health Canada and Agency program areas on issues such as injury prevention.

The CPSP is a member of the International Network of Paediatric Surveillance Units (INoPSU) which promotes enhanced collaboration between 12 national paediatric surveillance units worldwide that are full members: Australia, Britain, Canada, Cyprus/Greece, Germany, Ireland, Latvia, Netherlands, New Zealand, Portugal, Switzerland and Wales. Through participation in the INoPSU, the CPSP contributes to the advancement of knowledge of uncommon childhood infections and disorders and the participation of paediatricians in surveillance on a national and international basis.

4.4.2 Knowledge Development and Exchange - To what extent did early child childhood development and public health practitioners, and academics access and use knowledge products to support maternal, child and family health?

The Centre for Health Promotion has collaborated with stakeholders to create evidence-based knowledge products and disseminate them to target audiences to promote maternal and child health. Opportunities exist to develop strategic knowledge development and exchange priorities to maximize the impact of investments.

Between April 2010 and March 2015, the DCY, often in collaboration with Regional Operations, developed a range of knowledge products to support the delivery of the CAPC and the CPNP and to provide Canadian families with information to enhance their health and well-being. Products were targeted at parents, caregivers, health practitioners, pregnant women or those planning to get pregnant, the CAPC and the CPNP community project staff, volunteers, and others involved in community-based programs. See Appendix 5 for a listing of knowledge products created or revised between 2010-2011 and 2014-2015.

Funding Knowledge Development and Exchange within DCY

From 2010-2014, the NPF was one source of funding for knowledge development and exchange work within the Division. This fund was established in 1997 at $1.9 million/year (G&C), to identify and support time-limited initiatives with the intent to develop tools, resources and support for the CAPC and the CPNP to respond to common and emerging issues identified by funded organizations across Canada.

Products created under the NPF were completed in partnership with external organizations, through contribution agreements, and structured to include representation from the research, academic and practice communities including mandatory representation of the CAPC and the CPNP funded organizations. Some of the partners involved in the development of these NPF products include the IWK Health Centre, Pauktuutit, Parachute, Canadian Collaborating Centres for Injury Prevention (CCCIP), and others. Funding for the NPF was reduced in 2013-2014 and eliminated altogether in 2014-2015. During the period in scope for this evaluation, there were 10 active NPF contribution agreements.

The Centre for Health Promotion also utilized other sources of internal funding to support the development of knowledge products from 2010-2015. Some of the partners involved in the development of these products include the Canadian Paediatric Society, the Breastfeeding Committee for Canada, Health Canada and the Canadian Institute of Child Health. With the elimination of the NPF, while there are now no fixed financial resources dedicated to knowledge development and exchange, knowledge development and exchange efforts are funded through internal O&M resources.

Examples of Key Funded Projects

Mothers' Mental Health Toolkit

The Mothers' Mental Health Toolkit was one of the key products developed through the NPF between 2010 and 2015. It is a public resource for education, advocacy, and treatment support for mothers with mental health problems and is intended to support those who work with vulnerable mothers and families. The Toolkit was developed in 2010 by health care professionals from the IWK Reproductive Mental Health Service (IWK). The Agency has provided funding of $777,490 over 2 years to fund the creation of these materials.

Since 2013-2014, training to support implementation of the Toolkit has been provided via workshops in English and French to 980 people across Canada, including all the CAPC and the CPNP funded organizations, some AHSUNC participants and select regional partners. The majority of funded organizations visited as part of case studies for this evaluation highlighted the value of this training.

In a stakeholder survey administered by the program in 2015with a group of the CAPC and the CPNP funding recipients, 51% indicated being familiar with the Mother's Mental Health Tool Kit. Based on web analytics, awareness rates were higher in the Atlantic Provinces. This higher rate is likely due to the fact that it was developed there and suggests that there may be opportunities to expand reach in other regions. Of those respondents that reported being aware of the tool, 74% had used it at least once since March 2014. Respondents indicated that it was used to share health information with clients, students, parents and program participants (89%) or with other colleagues and health professionals. Moderate levels of use were reported to establish or update professional practice/procedure at the organizational level (38%) and to develop health promotion messages or initiatives (24%). Satisfaction with the Toolkit was very high, with at least 96% of respondents indicating that they agree/strongly agree with all categories of satisfaction, including that the product has addressed a knowledge gap that existed on the issue of mother's mental health.

Given that this product is owned by IWK, not the Agency, the Toolkit is currently only made available on the IWK website. According to internal key informants, this approach limits the Agency's ability to ensure that documentation is kept current and that training associated with their delivery is ongoing. It also restricts their ability to regularly monitor uptake or impact.

Various Safe Sleep and Breastfeeding Products

Beyond the NPF, the Agency has also invested resources to develop and disseminate products related to safe sleep and breastfeeding, including:

• "Protecting, Promoting And Supporting Breastfeeding: A Practical Workbook For Community-based Programs - 2nd Edition" (2014);
• Safe Sleep Products [Safe Sleep for Your Baby - Parent Brochure (2010, revised 2014); Safe Sleep for Your Baby - Video (2012); and Joint Statement on Safe Sleep: Preventing Sudden Infant Deaths in Canada (2011); and
• Brochure "Ten Valuable Tips for Successful Breastfeeding" (2009) and Brochure "Ten Great Reasons to Breastfeed your Baby" (2009)^{Footnote viii}.

Information collected via the 2013-2014 CPPMT survey indicates that a significant number of funded CAPC and CPNP organizations across Canada use at least one of these products to either share information with clients, students, parents, or participants; other colleagues or health practitioners; family and friends. The highest rates of usage were reported for The Safe Sleep for Your Baby - Parent Brochure (45% among CAPC respondents and 76% among CPNP respondents) and breastfeeding brochures, including 10 Great Reasons to Breastfeed your Baby (37% for CAPC and 80% for CPNP respondents respectively) and 10 Valuable Tips (34% for CAPC and 77% for CPNP).

To assess knowledge use/satisfaction with safe sleep and breastfeeding products, a stakeholder survey was sent to 588 CAPC and/or CPNP funding recipients and a list of over 1500 known recipients of KD products (with a response rate of 23%, or 479 respondents).

• Survey results show that among those familiar with these resources, 98% had read the Safe Sleep for Your Baby - Parent Brochure, 84% had viewed the video and 97% had read the Joint Statement.
• This survey also demonstrated that for "Protecting, Promoting And Supporting Breastfeeding: A Practical Workbook For Community-based Programs - 2nd Edition"(2014), 28 % of individuals were familiar with the Workbook, many of whom had read it (85%), understood it (90%) and used it at least once since its release in March of 2014 (67%). The Workbook was most frequently used to share health information with clients, students, parents and program participants (72%) and with other colleagues and health professionals (52%). Other reported uses of the Workbook were to establish professional practice/procedures (36%), to develop health promotion messaging/initiatives (27%), and to inform policy development or revision in their organization (23%).

An examination of printing and distribution data related to safe sleep and breastfeeding products indicates that, at various times, demand has exceeded supply for all three brochures which indicates that demand for printed knowledge products persist despite recent trends toward promoting online access to health information.

Another related prenatal product is The Sensible Guide to a Healthy Pregnancy (2008, revised 2012) which is targeted at pregnant women or those planning to get pregnant. It was distributed to the CAPC and CPNP funded organizations. Data collected through the 2013-2014 CPPMT survey illustrates that 48% of CAPC and 72% of CPNP respondents were aware of it, and 29% of CAPC and 68% of CPNP respondents indicated using it.

Nobody's Perfect

A key product developed outside the period in scope for this evaluation, but which continues to be relevant to both the Agency and the target audience, is Nobody's Perfect. Nobody's Perfect is a community-based parenting education and support program, developed specifically for vulnerable parents of children from birth through age five, and is typically offered by a trained facilitator to small groups of parents in weekly sessions over a six to eight week period. It was developed in the early 1980's by the Agency (then Health and Welfare Canada) and the four Atlantic provincial departments of health. It was introduced nationally in 1987.

Nobody's Perfect is owned by the Agency and continues to be delivered across Canada by the provinces and territories. Delivery rates of Nobody's Perfect vary across the country. According to a 2014-2015 assessment of knowledge development and exchange activities using the 2013-2014 CPPMT survey, 40% of the CAPC funded organizations and 30% of the CPNP funded organizations deliver the program, with delivery rates ranging from 49% in Manitoba/Saskatchewan to 21% in the Atlantic Region. There may be opportunities to further understand this variation and to potentially expand reach in certain regions.

The Agency supports training and learning opportunities for facilitators and trainers, aimed at improving the knowledge and skills of those delivering the Nobody's Perfect program. Overall satisfaction with in-person learning events was high. For example, feedback obtained from a program-led stakeholder survey (2015) demonstrates that the current approach to training and the material provided are highly effective and have a moderate impact on personal and professional practice. Most frequently reported uses of the events and resources among trainers/facilitators were: to share information with other health practitioners, clients, parents or program participants; to inform/adjust professional practice; and to facilitate the program.

Key informant interviews and case studies, as well as a 2009 impact study of Nobody's Perfect, demonstrated that the program continues to achieve considerable success in enhancing the capacity of parents to raise healthy children. More specifically, the 2009 impact study found that the program contributed to improvement in a number of parental outcomes that are consistently associated with superior child outcomes, such as parents' confidence and self-image and coping skills.

Establishing Priorities Moving Forward

Based on the 2014-2015 assessment of knowledge development and exchange activities, products developed to date have reached the target population and have demonstrated positive results. Recently, the DCY has been working on documenting needs and priorities for knowledge development and exchange activities. They recognize the need to put in place a plan to ensure a strategic approach is taken to knowledge development and exchange activities planning and reporting within the DCY moving forward. Previous DCY evaluations have noted the need to review knowledge development and exchange activities through strategic planning. This gap is particularly significant in light of the elimination of the NPF funding in 2014-15 and other reductions in funding for knowledge development and exchange work for maternal and child health within the Division.

Key informants identified a number of challenges that could be addressed through better knowledge development and exchange strategic planning horizontally within the DCY to identify priorities. They indicated that there is a need to:

• identify knowledge gaps and develop new and/or adapt existing knowledge products to respond to needs, including adaptation for other audiences as appropriate;
• ensure that product content remains current through regular review and revision;
• raise awareness of knowledge products beyond the CAPC and the CPNP funded organizations to the broader public health and social service systems;
• consult with stakeholders to ensure that efforts are complementary; and
• work with partners to expand reach of knowledge products.

Further, a 2014-2015 Assessment of Knowledge Development and Exchange activities undertaken through CAPC, CPNP and associated activities^{Footnote ix} was completed in support of this evaluation. This assessment provided performance metrics for documents deemed of significant value to the Agency according to project site staff. To support future strategic knowledge development and exchange planning and reporting, there is a need to ensure that knowledge development and exchange efforts continue to be monitored and measured in a systematic way, perhaps through the CCPMT, and in consultation with partners.

Significant knowledge development and exchange successes were achieved through associated activities in maternal and child health.

Other associated maternal and child health activities have been effective in supporting and conducting research and knowledge exchange to promote the uptake of research results.

Joint Consortium on School Health (JCSH)

One of the key activities of the JCSH is to facilitate the development and dissemination of better practices and information promoting comprehensive school health approaches. Key informants indicated that JCSH participants have collectively learned from one another and built top notch resources together.

Below are a few examples of key knowledge products produced and disseminated through the activities and multi-sectoral and multi-jurisdictional networks of the JCSH over the last five years. Knowledge products are also available through the JCSH website: http://www.jcsh-cces.ca.

• Youth Engagement Toolkit (2014) shows the importance of youth engagement for learning.
• Schools as a Setting for Positive Mental Health: Better Practices and Perspectives, 2nd Edition (2013) looks at best practice approaches in the school setting for mental health.
• Articles on the Healthy School Planner were publish in the Canadian Association of Principals Journal on Comprehensive School Health (Summer 2013) such as "The JCSH Healthy School Planner: Learning how to assess the health of your school", "Comprehensive School Health in six priority areas: The work of an Education and Health partnership" and "National Perspectives on Comprehensive School Health: What's the Value of Your School?"
• Between 2009 and 2015, numerous other toolkits and fact sheets on topics such as substance use and physical activity were produced and continue to be disseminated.

Health Behaviours of School Aged Children (HBSC) Survey

The HBSC survey aims to increase knowledge and understanding of the health and well-being of young people (aged 11 to 15) and the social context of their health attitudes and behaviours. The HBSC survey is the main source of bullying data in Canada, and also produces data on mental and emotional health, physical activity, sedentary activities, eating habits, injury and substance use. HBSC survey findings are published in international and national reports, as well as peer-reviewed journals. Knowledge products are also available through the HBSC website: http://www.hbsc.org.

The national reports, published on the Agency and the HBSC websites, contain findings on the health of Canadian youth. Examples of national reports include: Healthy Behaviour in School-aged Children: Trends Report 1990-2010 (2014) and The Health of Canada's Young People: a mental health focus, Canadian Report (2011). PREVNet, an active national network of Canadian researchers, non-governmental organizations (NGOs) and governments working together to stop bullying in Canada, was built using HBSC national reports as a key platform. HBSC national data has been referenced in the annual Chief Public Health Officer's Reports on the State of Public Health in Canada.

Key informants indicated that eight provincial or territorial specific HBSC reports were shared by the Agency with respective ministries of Health and Education. Some of these reports, in turn, have been key documents in shaping provincial or territorial wellness strategies, e.g. On the Path Together - Wellness Plan for Yukon's Children and Families (2014).

International reports collect the findings of participating member countries for cross-comparison and identification of global trends. Reports are produced as part of the World Health Organization Policy series 'Health Policy for Children and Adolescents'. An example report published on the international HBSC survey website is: Social determinants of health and well- being among young people. Health Behaviour in School-aged Children (HBSC) study: international report from the 2009/2010 survey (2012). Another example, in 2013 UNICEF launched its 11th Report Card 'Child well-being in rich countries', using HBSC data, among others, to rank 29 of the world's richest countries in terms of child well-being.

Key informants indicated that HBSC findings are also used for secondary analysis in other publications. The HBSC research team have published numerous peer reviewed articles, sometimes with the Agency's collaborators, on topics such as substance use, injury and physical activity.

Canadian Perinatal Surveillance System (CPSS)

The CPSS surveillance activities provide information and evidence that allow monitoring and reporting on key indicators of maternal, fetal and infant health in Canada. This information is used to improve the effectiveness and efficiency of clinical care and guide the development of public health policies and programs for maternal and infant health.

Program documents detail how the CPSS key indicators address both determinants (i.e., risk/protective factors) and outcomes (i.e., health conditions) of perinatal health. Outcome indicators capture health events that are serious and frequent in the target population. Several of these indicators, such as infant mortality and maternal mortality, are used domestically and internationally as well-established measures of the overall health of the population and health system performance. Indicators such as multiple and preterm births provide information of conditions with high impact on the health of babies and on the Canadian health care system due to the costs associated with their management. Other indicators such as maternal smoking, maternal alcohol consumption and breastfeeding are used to inform prevention efforts from a public health perspective.

A document review and key informant interviews indicate that various information products have been produced and are disseminated broadly.

• A review of documents shows that information products have varied according to the target audience (which includes policy makers, health care providers, researchers, and the public) and include national reports, topic-driven fact sheets, and peer-reviewed publications. Recent CPSS publications include: Perinatal Health Indicators for Canada (2013), Congenital Anomalies in Canada 2013: A Perinatal Health Surveillance Report (2013), Canadian Hospitals Maternity Policies and Practices Survey (2012), Maternal Mortality in Canada Fact Sheet (2011), and Severe Maternal Morbidity in Canada Fact Sheet (2011). A series of fact sheets published in 2014 are also available: Maternal Diabetes in Canada, Maternal Hypertension in Canada, Sudden Infant Death Syndrome (SIDS), Folic Acid Use among Pregnant Women in Canada, and Pregnancy and Women's Mental Health in Canada.
• Key informants indicated that these products have been disseminated within the Agency, including targeted distribution to internal program staff and more broadly through electronic tools such as Just the PHACS and Broadcast News. External distribution has been extensive, including responding to email requests, and proactive distribution online and through various networks including: CPSS Steering Committee and former study groups, provincial and territorial contacts, hospitals, and various national and international paediatric stakeholders (Canadian Association of Midwives, Canadian Association of Perinatal and Women's Health Nurses, Canadian Institute of Child Health, Canadian Paediatric Society, Canadian Perinatal Programs Coalition, Canadian Public Health Association, College of Family Physicians of Canada, Canadian College of Medical Geneticists, National Birth Defects Prevention Network, International Clearinghouse for Birth Defects Surveillance and Research, Society of Obstetricians and Gynaecologists of Canada).

Canadian Paediatric Surveillance Program (CPSP)

The CPSP is a uniquely flexible and responsive program that provides information on a wide range of issues that would not otherwise be available. Program documents indicate that, through the Canadian Paediatric Society, the CPSP collects data directly from over 2,500 paediatricians and paediatric subspecialists (participants) through monthly survey submissions (80% response rate in 2014). Other participants, such as coroners, paediatric surgeons and adult endocrinologists are enrolled in the program when research studies indicate their participation. These physicians cover a paediatric population of over seven million Canadian children and youth.

Key informants indicated that the CPSP provides an infrastructure for rapid, inexpensive and efficient collection and transmission of information. It allows the Agency to quickly respond to public health emergencies, e.g. H1N1 and reactions to Tamiflu in the paediatric population. This approach is timelier than passive surveillance using administrative data sources. It also provides the Agency with a network of frontline practitioners who are key to health promotion and disease/injury prevention and provides information on current patterns in clinical practice.

A document review and key informants indicated that the CPSP provides timely epidemiological prospective data collection that has a direct impact on the diagnosis and treatment of patients, regarding the occurrence (or absence) of serious infectious diseases and health outcomes related to toxins or other risks (e.g., consumer products). Reports and studies are used by paediatricians to inform their practice and by Health Canada to regulate consumer and health products.

Information has been disseminated through a variety of approaches, including published papers and presentations at national and international conferences.

• An annual "Results" report is published by the CPSP that details publications, presentations and studies produced and available to policy and program decision makers.
• A list of academic journals in which 26 CPSP published papers were included in 2014 includes: Child Abuse and Neglect, Brain Injuries, Pediatrics, In Practice: Journal of Allergy and Clinical Immunology, The Canadian Journal of Infectious Diseases & Medical Microbiology, Pediatric Diabetes, Clinical Immunology, and Paediatrics and Child Health.
• Examples of national and international conferences at which the CPSP findings were presented in 2014 include: Canadian Immunization Conference, Canadian Paediatric Society Annual Conference, Histiocyte Society Annual Meeting, American Academy of Child and Adolescent Psychiatry Annual Meeting, and Pediatric Rheumatology Symposium.

Results of the CPSP surveillance activities are also disseminated directly through the Canadian Paediatric Society through a variety of approaches, including articles in the Child Health Journal (16,000 subscribers) and through presentations at their annual meeting in June. Also, the "ADR Tip of the Month" is an educational tool on adverse drug reaction topics that is sent to the 2500 survey participants on a monthly basis to build support and awareness of the Adverse Drug Reaction study. All knowledge products, including the annual "Results" report, are also available through the CPSP website: http://www.cpsp.cps.ca.

The CPSP supports Health Canada and the Agency in meeting international reporting requirements and participation in international studies. Health Canada contributes to international surveillance efforts, e.g. Canada's commitment to the global polio (acute flaccid paralysis) eradication initiative through the WHO and the elimination of measles. Canada has also been a contributor to many multi-country studies (e.g. diabetes, adverse drug reactions, acute rheumatic fever, and congenital rubella syndrome) that provide the basis for international comparisons. By including international comparisons with information from the CPSP, and other child health data sources, the Agency has the opportunity to enhance the scope and quality of its analysis and interpretation of child health issues.

4.4.3 Reach (CAPC and CPNP) - To what extent did parents/caregivers and their children facing conditions of risk participate in programs?

Over the last 5 years, the CAPC and CPNP funded organizations have consistently reached parents/caregivers and children facing conditions of risk across Canada. There are opportunities to conduct additional analyses to identify and address potential gaps in terms of program reach.

Numbers of Site Locations and Participants

Internal key informant interviews, case studies and a document review indicate that since the inception of both programs, in 1993 (CAPC) and 1995 (CPNP), the site locations and numbers of program participants for both programs across Canada has remained consistent. Results collected through the 2013-2014 CPPMT survey with funded organizations provide insight into the number of funded organizations, project sites and participants across Canada (Tables 6, 7, 8 and 9). The numbers provided below are an underrepresentation of the complete population served by CAPC and CPNP (see Appendix 3 for details on exemptions to 2013-2014 CPPMT survey data, including sites in the North and Quebec).

• Based on 2013-2014 CPPMT data, a total of 1,531 CAPC sites and 654 CPNP sites were supported through contribution agreements (Table 6). Organizations were funded in all Canadian provinces and territories. Further, Appendix 6 provides a map of CAPC and CPNP project sites (as of 2010) as a visual depiction of the distribution of sites across Canada.

Source: 2013-2014 CPPMT

• Results from CPPMT data collected for the years 2013-2014 illustrate that the CAPC and the CPNP funded organizations served a comprehensive mix of urban, rural and remote or isolated areas (Table 7). Some of the funded organizations provided services in both urban and rural areas. Although only 19% of the Canadian population lives in rural, remote, or isolated areas or on reserves (Statistics Canada, Census 2011)^{Footnote xi}, more than one third (38.8%) of CAPC projects and half (50%) of CPNP projects were focussed on reaching communities in rural, remote, or isolated areas or on reserves. The programs are increasing accessibility to health system supports and potentially reducing inequities in these often underserved communities.

Source: 2013-2014 CPPMT survey

Based on 2013-2014 CPPMT reporting, there were a total of 223,340 participants (112,074 parents/caregivers, and 109,745 children age 0-6 years) in CAPC funded organizations. An additional 1,521 participants were served by projects in Nunavut, Northwest Territories and Yukon. There were a total of 51,228 participants reported by CPNP funded organizations (24,605 pregnant women, 4,169 postnatal women, and 5918 caregivers). Additionally 15,977 participants in Quebec^{Footnote xii} and 559 participants from Nunavut, Northwest Territories and Yukon^{Footnote xiii} were served.

Source: 2013-2014 CPPMT survey
Note: Does not include participant numbers from Nunavut, Northwest Territories or Yukon

Source: 2013-2014 CPPMT survey
Note: Does not include participant numbers from Nunavut, Northwest Territories, Yukon or Quebec.

At-risk Populations Reached

Both CAPC and CPNP were designed to support priority populations, especially children and families living in conditions of risk. The criteria in the CAPC and the CPNP 2015 participant surveys to measure the at-risk target population include: low income, low education, Indigenous status, single parents, recent immigrants (10 years or less in Canada) and teen parents. CPNP also includes food insecurity.

Results from the participant surveys have been compared to 2011 National Population Health Survey (NPHS) data to provide an indication of CAPC's and CPNP's reach to various groups of people living in conditions of risk (Table 10). In general, across all risk criteria, program reach exceeds levels of risk criteria found in the general population.

Low income families: Overall, more than half (51%) of parents and caregivers at CAPC funded sites and 65% of participants at CPNP funded sites who responded to the survey reported family incomes below the program's assessment of low income. Although direct comparisons are difficult, 15% of Canadian families had a low income based on the after-tax low income measure included in the National Population Health Survey, 2011.

Low education: Almost one in five (19%) parents and caregivers at CAPC funded sites and one in four (26%) participants at CPNP funded sites who responded to the survey had not graduated from high school or equivalent. In comparison, 13% of the Canadian adult population (between ages 25-54) reported that they had not obtained a high school diploma in the National Population Health Survey, 2011.

Indigenous status: Almost one in four (22%) parents and caregivers at CAPC funded sites and one in three (33%) participants at CPNP funded sites who responded to the survey identified themselves as an Indigenous person. In comparison, 4% of the Canadian population self-identified as Indigenous in the National Population Health Survey, 2011.

Single parents: Almost one in four (23%) parents and caregivers at CAPC funded sites and more than one in four (27%) participants at CPNP funded sites who responded to the survey were single parents. By comparison, 14% of the general population were single parents in the National Population Health Survey, 2011.

Recent immigrants: About 13% of parents and caregivers at CAPC funded sites and 16% of participants at CPNP funded sites who responded to the survey had immigrated to Canada within the last 10 years. Overall, the proportion of new immigrant CAPC and CPNP funded site participants is about double that of general Canadian population (7%) from the National Population Health Survey, 2011.

Multiple Conditions of Risk

In should be noted that these conditions of risk are not mutually exclusive. People living in conditions of risk are generally impacted by multiple conditions of risk. An assessment of risks identified in the CAPC and CPNP 2015 participant surveys indicated that multiple risks were often identified (Table 11) by survey respondents. According to the CAPC and CPNP 2015 participant surveys, at least 62% of CAPC and 80% of CPNP survey respondents self-identified as having one or more of these risk factors.^{Footnote xv}

While it was noted that about one third of parents and caregivers at CAPC funded sites and one fifth of participants at CPNP funded sites who responded to the survey did not identify any conditions of risk, internal key informants indicate it is possible that: broad-based community-centred organizations provide services to other individuals, who may not identify themselves as at risk; and some organizations received funding from other sources that do not specify the need to target monies to at-risk individuals.

In summary, a significant proportion of CAPC and CPNP families live in conditions of risk, and many families reached by CAPC and CPNP are affected by multiple risk factors. For all risk factors, both CAPC and CPNP families exhibit conditions of risk greater than the general population. This early intervention is important as research has shown that children growing up in conditions of risk often have higher rates of poor health, developmental difficulties, and social, cognitive and behavioral problems (see Section 4.1 - Continued Need).

The 2011 CAPC national study highlighted a number of resource related challenges for parents and caregivers at CAPC funded sites. The report states that nearly 39% of participants reported challenges with employment and approximately 25% reported challenges with food and 24% with housing. Other challenges identified in the 2011 CAPC national study included: stress (59%), child care (46%), transportation (36%), parenting (31%), isolation (28%), family conflict (25%), health (22%), and mental health (17%).

The report also details that the participants at CAPC funded sites experienced greater exposure to the following risk factors:

• approximately 25% of caregivers at CAPC funded sites reported some food insecurity compared with 14% in the matched Canadian Community Health Survey (CCHS) subsample;
• approximately 37% of mother participants at CAPC funded sites reported that they were depressed based on the Centre for Epidemiologic Studies Depression scale (CES-D) compared with 16% in the matched National Longitudinal Survey of Children and Youth (NLSCY) subsample; and
• approximately 14% of caregivers at CAPC funded sites reported that their child had special needs such as difficulty with hearing, seeing, speech, learning, moving or behaviour. This was significantly higher than the proportion (6%) in the NLSCY matched subsample.

Enablers to Optimize Participation of At-risk Participants

Case studies and program documents highlighted that funded organizations work strategically to optimize participation of at-risk participants in their CAPC and CPNP sites. Incentives associated with accessibility include: providing transportation (e.g. bus tickets) or the ability to walk to project site, offering child care with a focus on child development, and offering a multiplicity of services available at one point of entry. Other enablers linked to optimizing the participation of at-risk populations, include: providing a non-threatening environment, offering continued support, allowing parents to identify their own needs, and offering universal or mixed (vs. targeted) programs because they can reduce the sigma of attending for the at-risk participants.

Of particular note, offering activities that include access to or the preparation of food was noted in a number of case studies and previous program reviews as being particularly impactful. Food security is an ever growing issue for many of the families served through CAPC and CPNP funded sites. Program activities related to food, such as community kitchens, helped families meet their nutritional needs by providing access to healthy food and learn more about healthier eating.

Program Reach and Gaps in Analyses

In terms of understanding and optimizing program reach, there are opportunities to enhance the performance measurement data collected and analyzed to better understand the vulnerabilities of the program participants reached at the project sites funded through the CAPC and the CPNP.

As highlighted earlier in this section, many of the families participating in both the CAPC and CPNP funded activities exhibit conditions of risk. While funded activities are typically designed to reach an at-risk population, other community members may frequent the organizations and attend funded activities, given the community-based nature of these organizations. It was noted, in the CAPC and CPNP 2015 participant surveys^{Footnote xvi}, that about one third (37%) of CAPC and one fifth (20%) of CPNP funded site participants self-identified as not having any of the following six risk factors:  low income, low education, Indigenous status, single parents, recent immigrants (10 years or less in Canada) and teen parents. Additional data collection and analyses could provide more insight into other vulnerabilities and needs of program participants that have not been documented.

Demand for CAPC and CPNP activities at times exceeds the capacity of the funded organizations. Many project sites have to turn away clients who seek out their maternal and child health services and have wait lists. The 2013-2014 CPPMT survey indicated that 36% of CAPC and 23% of CPNP funded project sites reported that they had to refuse participants, for various reasons including: lack of funding, space, project staff or transportation. Of those projects that had to refuse participants, approximately half of (52%) of CAPC and one third (33%) of CPNP projects contribution agreement recipients reported that they used wait lists. There were an average of 27 individuals on CAPC funded project site wait lists and 15 individuals on CPNP funded project site wait lists. A clearer picture of the locations of the funded project sites in comparison with demographics related to vulnerable populations would provide a better understanding of gaps and opportunities to optimize program reach.

The document review and key informant interviews have confirmed that the mix of project sites for both programs has remained stable since the programs were established more than 20 years ago. A document review indicated that demographics related to conditions of risk may change over time, e.g. settling of new immigrants in communities, income distribution nationally or across regions.

There has been some analysis undertaken by the program on the composition of project sites and relevant changes to demographics. Key informant interviews highlight that a few regions have done some analyses using vulnerability indices and made some adjustments to funding agreements. An internal national GIS study of the CAPC and the CPNP was completed in 2012 which highlighted that a spatial analytical approach is feasible and there are specific communities of potential need.

With a view to assessing if the programs are optimizing reach to the populations that are most vulnerable to conditions of risk, there may be further opportunities to analyse a variety of data sources to explore gaps to optimize program reach.

4.4.4 Health Outcomes (CAPC and CPNP) - To what extent did parents/caregivers and their children gain knowledge and build skills to support maternal and child health?

Organizations receiving CAPC and CPNP funding have contributed to increased participant knowledge and skills to support maternal and child health.

Parents who understand how children develop and use effective parenting techniques raise healthier and happier children (see Section 4.1 - Continued Need). Both programs have enhanced participant knowledge and skills related to supporting both maternal and child health. Data from both CAPC and CPNP participant surveys, case studies, and program document reviews highlight the dimensions of the variety of knowledge and skills gained.

Parental Knowledge and Skills (CAPC)

A number of previous evaluation studies highlight the value of CAPC funded activities in terms of enhancing parenting knowledge and skills. The 2011 CAPC national study related that 60% of project coordinators reported that CAPC funded activities had a strong impact on parenting. A 2012 CAPC evaluation in Saskatchewan indicated that both project staff and community partners recognized improvements in participants' parenting skills. According to staff, participants were better able to support their children, were becoming more involved with their child's education, and were becoming more aware of and monitoring their child's progress. In focus groups undertaken as part of this 2012 evaluation, parents and caregivers discussed at great length how they learned more about parenting skills and gained more knowledge regarding child development.

Also in terms of supporting children's development, participant responses to the 2015 CAPC participant survey highlight how participation in programs funded through the CAPC has increased awareness of positive parenting practices and helped parents and caregiver gain new parenting skills. Eighty-six percent (86%) of respondents strongly agreed or agreed that the program improved their parenting skills. Parents and caregivers also indicated that as a result of participating in these programs they: do more things with my child to help him or her learn (91%); are more aware of how children change as they learn and grow (91%); know more about where I can get answers to my parenting and child development questions (91%); and make time to read to my child more often (81%).

Parents and caregivers had improvements in self-awareness and self-esteem in parenting roles as a result of participation in CAPC funded activities.

• The 2015 CAPC participant survey highlighted that the large majority of parents and caregivers feel more positive in their role as a parent (90%), have more confidence in their parenting skills (88%) and are better able to cope with their stress (78%) as a result of participating in these programs.
• In addition, the 2012 Saskatchewan regional evaluation found that some participants discussed specifically how they have learned more about discipline strategies and how to interact with their children in a more calm and positive way. Staff recognized that participants have increased confidence and self-esteem.

Improved Relationships (CAPC)

After participating in programs at the CAPC funded sites, the parents and caregivers reported having a better relationship with their children. Many participants also described having improved relationships with others such as spouses or partners.

• When compared with a NLSCY matched sample, the 2011 CAPC national study found that the participants at CAPC funded sites were significantly more likely to have a better family functioning score^{Footnote xvii}.
• The 2012 CAPC evaluation in Saskatchewan noted that three areas of improvement were discussed by program participants: improved communication, being more understanding of one another, and being on the same page in terms of parenting techniques.
• The recent 2015 CAPC participant survey highlighted that 87% of parents and caregivers indicated that they have a better relationship with their child since coming to the program.

Knowledge of Nutrition and Healthy Living (CPNP)

The 2015 CPNP participant survey highlighted a number of nutrition and healthy living issues about which knowledge had been enhanced for the large majority of participants in the program. Participants indicated that since coming to the program they: know more about healthy eating during pregnancy (92%); know more about prenatal vitamins or multivitamins (83%); have learned more about healthy weight gain during pregnancy and know more about where I can get food if needed (82%); and have learned more about food budgeting (72%).

Knowledge of Prenatal Health and Infant Care (CPNP)

Knowledge of prenatal care had been enhanced for the majority of participants in CPNP funded activities. Through the 2015 CPNP participant survey, the large majority of participants indicated that since coming to the program they: know more about the stages of pregnancy and birth (91%); have learned more about the signs of baby blues/post-partum depression (87%); have a better understanding of the effects of drinking alcohol during pregnancy on my baby (85%); and have a better understanding of the effects of smoking during pregnancy on my baby (85%).

Knowledge about infant care had also been enhanced for the majority of participants as a result of participating in the CPNP funded activities. Through the 2015 CPNP participant survey, the large majority of participants indicated that they: have learned more about my baby's growth and development (93%); know more about ways of bonding with my baby (93%); have learned more about the importance of breastfeeding (92%); know more about safe sleep for my baby (90%); have learned more about ways to protect my baby from injuries (88%); know more about when to give solid foods to my baby (88%); know more about Shaken Baby Syndrome (85%); and have learned more about how to put my baby safely in a car seat (85%).

Social Supports and Networks (CAPC and CPNP)

Case studies for both CAPC and CPNP funded sites highlighted the value of peer-to-peer support for teens and other at-risk women and children. Project site staff indicted that participants mentor one another and have been able to build friendships in a welcoming and safe environment. They also stated that these supportive relationships continue to last after the program has concluded.

A number of recent program studies have indicated that programs at CAPC funded sites reduced social isolation through helping to build social support networks and increasing social skills.

• The 2011 CAPC national study indicated that 9% of participants can be considered to have poor social support.^{Footnote xviii} However, participants were significantly more likely to have higher social support scores when compared with a matched sub-sample of the NLSCY. Almost 80% of project coordinators reported that their CAPC funded project had a strong impact on improving social support for parents and caregivers.
• The 2012 review of program impacts in Saskatchewan highlighted that receiving support and socializing with others was one of the top reasons participants provided for joining programs at CAPC funded sites. Participants described how attending the project provided an opportunity to socialize with others and alleviated isolation.
• A 2013-2014 Northern Unit highlights report indicated that CAPC funded sites provide a "safe space" for women and children, thus reducing isolation for vulnerable women and creating support networks in the community.
• The recent 2015 CAPC participant survey identified a large majority of parents and caregivers indicated that through the program, they had more people to talk to when they need support (90%).

Knowledge of and Access to Community Services (CAPC and CPNP)

The 2015 CAPC and CPNP participant surveys asked parents and caregivers about their knowledge and use of community services. Respondents indicated that they: know more about who to contact in the community when they need help (90% CAPC and 93% CPNP); are more connected with programs they can use (91% CPNP); are using other programs and services that they had not used before (81% CAPC), and learned more about where people can get help for abuse or family violence (70% CAPC and 79% CPNP).

The 2011 CAPC national study indicates that the rate at which participants at CAPC funded sites seek support from community agencies and programs is 19% higher than among a similar population in the NLSCY.

4.4.5 Health Outcomes (CAPC and CPNP) - To what extent did parents/caregivers and their children adopt healthy practices for themselves and their families?

Organizations receiving the CAPC and the CPNP funding have contributed to participants adopting a variety of healthy practices for themselves and their families.

Many of the socio-demographic characteristics of program participants and their families place them at risk for developing poor health (see Section 4.1 - Continued Need). The 2011 CAPC national study related that there were significant challenges to the health and well-being of parents and caregivers and their families. Overall health comparisons between participants in the program (self-reported) and the NLSCY revealed that:

• 82% of children at CAPC funded sites were reported to be in either excellent or very good health compared to 91% of children in the overall Canadian population; and
• 57% of caregivers at CAPC funded sites reported being in excellent or very good health compared to 73% in the overall (unadjusted) NLSCY data.

The adoption of healthy practices through both CAPC and CPNP relate to both the health and social development of children as well as adult participant health and well-being.

Improvements in the Health and Social Development of Children (CAPC)

A child's ability to interact with other children and adults is an important indicator of social knowledge and competence. Throughout the case studies, regional evaluations and CAPC participant survey, project site staff spoke of children's enhanced social skills and how these improvements contributed to children's school readiness.

In the 2012 Saskatchewan regional evaluation, parents indicated that their children had become more social and less shy when around others. Participants from two of the focus groups discussed how the CAPC funded project helped their children meet developmental milestones including: becoming more independent, learning more about and correctly identifying emotions, learning to sit and listen, and improvements in speech.

The 2015 CAPC participant survey highlighted that the majority of parents and caregivers indicated that the program had enhanced the health and social development of their children. Participants indicated that: my child has more chances to play with other children (95%); my child spends more time in active games, playing outside or doing other physical activities (83%); and my child is more interested in being read stories, or looking at books (80%).

Improvements in Participant Well-being (CAPC and CPNP)

The 2015 CAPC and CPNP participant surveys highlighted that the large majority of parents and caregivers adopted healthier practices. The development of health-related skills included making healthier personal choices and improving personal health, learning about food preparation and overcoming addiction.

Prenatal health (CPNP)/Nutrition and healthy living (CAPC and CPNP)

In terms of prenatal health, survey respondents (CPNP) indicated that since coming to the program: I have limited my exposure to second hand smoke (79%); I am more physically active (69%); and I see a doctor, midwife, and/or nurse practitioner more regularly (66%).

In terms of nutrition and healthy living, survey respondents (CAPC and CPNP) indicated that since coming to the program: I am making healthier food choices (84% CPNP); I prepare healthier meals and snacks for my family (79% CAPC); and I take prenatal or multi vitamins more regularly (78% CPNP).

Smoking and alcohol cessation (CPNP)

Rates of both smoking and drinking were higher for survey respondents (CPNP) when they entered the program than for the general population. It was noted that 3.7% of the general population reported in the 2014 CCHS that they smoked during their pregnancy - compared to 27% for participants at CPNP funded sites entering the program. As well, 0.62% of the general population reported in the 2014 CCHS that they drank alcohol during their pregnancy - compared to 5% for participants at CPNP funded sites entering the program.

Participation in CPNP funded activities contributed to a reduction in both the rates of smoking and drinking during pregnancy for program participants. The CPNP 2015 participant survey results are listed below.

• In terms of smoking behaviors while pregnant, as mentioned, about 27% (448/1666) of 2015 CPNP survey respondents reported smoking since learning they were pregnant. About 80% (343/430) of reported smokers reported that their behavior changed since learning they were pregnant. Most (55%) reported that they had reduced their smoking, while 25% reported that they had quit smoking. In terms of the impact of CPNP on their decisions, about 64% (203/319) of those who changed their behavior indicted that participation in CPNP in some way aided their decision to reduce or quit smoking.
• In terms of consumption of alcohol while pregnant, about 5% (81/1687) of 2015 CPNP survey respondents reported drinking alcohol since learning they were pregnant. About 99% (71/72) of reported drinkers reported that their behavior changed since learning they were pregnant. While many (28%) reported that they had reduced their drinking, 71% reported that they had quit drinking. In terms of the impact of CPNP on their decisions, about 70% (48/69) of those who changed their behavior indicted that participation in CPNP in some way aided their decision to reduce or quit drinking alcohol during their pregnancy.

Further, in terms of exposure to the program, a 2012 CPNP study (Muhajarine) determined that participants with higher exposure were more likely to reduce the number of cigarettes they smoked, to cease drinking and to increase the use of vitamin/mineral supplements from never to daily.

4.4.6 Health Outcomes (CAPC and CPNP) - To what extent did parents/caregivers report that they or their children were impacted as a result of participation in CAPC and/or CPNP?

Overall there are indications that the CAPC and the CPNP funding have contributed to positive short-term health outcomes for program participants and their families.

Based on their program authorities, the CAPC and the CPNP were designed to ultimately contribute to the improved health and reduction of health disparities for participants (parents, caregivers, and pregnant women) and their children who are facing conditions of risk. Through the case studies and various participant surveys, both project site staff and participants have strongly indicated that both CAPC and CPNP have had a positive short-term impact on the health and well-being of participants and their children.

Improvements in the Health and Social Development of Children (CAPC)

The CAPC strives to contribute to the healthy development of children (0 to 6 years) living in conditions of risk. The 2015 CAPC participant survey provides insight into how parents and caregivers noted a marked improvement in their children's physical health and social development. Ninety percent (90%) indicated that participation in CAPC funded activities improved the health and well-being of their children.

In terms of social, emotional and cognitive development of their children, 80-90% of parents and caregivers (survey respondents) reported the following program impacts through the 2015 CAPC participant survey: my child is more comfortable in social settings (90%); my child plays better with other children (89%); my child follows along with or knows more songs and rhymes (87%); my child is better able to express him/herself (83%); my child knows or uses more words (82%); my child recognizes more colours, or shapes, or letters or numbers (82%); and my child plays more with crayons or pencils scribbling or drawing (80%).

Thorough the case studies, project site staff reported many direct benefits to the children who participated in early childhood development activities. Examples included developmental benefits like improvements in speech (knowing and using more words) and school readiness.

Improvements in Participant Well-being (CAPC and CPNP)

Both the CAPC and the CPNP funding provided to project sites contributed to improvements in the physical and mental health of parents and caregivers. Participant surveys, previous program studies, and case studies have highlighted how program participation has impacted parent and caregiver overall health and well-being.

The CAPC and CPNP 2015 participant survey results reflect a strong indication from program participants that the programs have had a significant impact in improving their overall health and well-being (83% CAPC and 92% CPNP). Further, CPNP survey respondents indicated overwhelmingly that the program had a positive influence on their pregnancy (97%). A large proportion of survey respondents indicated that their mental health has improved as a result of participating in the program (82% CAPC and 83% CPNP). Further, CAPC survey respondents also indicated that they are better able to handle the everyday challenges of raising children like sleep, toilet training, food dislikes, etc. (84%).

These results are consistent with the emotional well-being findings from the 2011 CAPC national study. It also highlighted that 55% of project coordinators reported that CAPC had a strong impact on improving parent mental health (e.g. reduce stress, reduce isolation, referrals to counselling).

Case studies illustrated anecdotally, through interviews with program leaders and participants, that program participants have enhanced self-esteem through their participation in the program. For example, participants enhanced their positive parenting knowledge and skills, as well as had opportunities to build friendships and gain group support though the program, which in turn led to enhanced self confidence in participant's parental or caregiver roles.

Impact on Birth Outcomes and Breastfeeding (CPNP)

The impact of the CPNP funding to project sites includes improvements in birth outcomes and breastfeeding initiation and duration. Details are provided in Table 12. These impacts have been noted in the CPNP 2015 participant survey, previous program studies and in case studies. Comparisons are also made with the general population.

Birth Outcomes

A 2012 CPNP study (Muhajarine) determined that participants with overall high exposure to the program tended to have fewer negative birth outcomes than those with lower exposure to the program. They were consistently less likely to have pre-term births or to give birth to an infant that was low birth weight or otherwise in poor health.

Recent performance data collected by the program indicate that birth outcomes for participants are similar to that of the general population. In the 2015 CPNP participant survey, 6.8% respondents indicated that they had a baby with a low birth weight - that rate was very close to that of the general population (6.10%). Similarly, project sites reported in the 2013-2014 CPPMT survey that 8.3% of participants had preterm babies - compared to 7.57% in the general population.

Given that many program participants experience risk factors that are known to increase the likelihood of poor birth outcomes (see Section 4.1 - Continued Need), these results highlight the value of program participation.

Breastfeeding Initiation and Duration

There is a considerable body of evidence demonstrating the benefits of breastfeeding for both infants and their mothers. Research also shows that the duration of breastfeeding yields even greater benefits.

• The 2015 CPNP participant survey results, when compared with the general population (CCHS 2014), highlights that the rate of breastfeeding initiation was equivalent (88.5% and 88.15% respectively).
• A 2012 CPNP study (Muhajarine) determined that participants with overall high exposure to the program tended to have more positive health behavior changes than those with lower exposure to the program. They were consistently more likely to breastfeed their infants and, in particular, to breastfeed for longer.

Given that many program participants experience risk factors that are known to decrease the likelihood of breastfeeding (see Section 4.1 - Continued Need); these results highlight the value of program participation.

Contribution versus Attribution: Longer Term Impacts for Program Participants

As mentioned in Section 3.1 - Evaluation Scope, Approach and Design, the evaluation has been designed to demonstrate the likely contributions of the programs to the expected outcomes, rather than demonstrate direct causal links between the programs and outcomes. It is acknowledged that there are many other influences that may have an impact on changes in an individual's knowledge, skills and health behaviors. As with all community-based programs, the CAPC and CPNP funded organizations work in partnership with other local services, have a multitude of funding sources, and offer a plethora of services.

As detailed in other report sections, the majority of funded organizations work in partnership with (Section 4.4.1 - Partnerships) and receive financial contributions from other stakeholders (Section 4.5 - Economy and Efficiency). There is significant variability in the percentage of overall project funding from the Agency through the CAPC and the CPNP. The percentage of CAPC and CPNP contributions in relation to the overall budget for funded organizations was not available for all projects for this evaluation; however, for those sites selected for case studies, CAPC and CPNP funding accounted for 8% to 40% of their overall budgets.

A limitation in program performance measurement data collected to date relates to a need to better understand program reach and impact across this spectrum of percentages of Agency financial support. Further insight on the impacts of CAPC and CPNP funding may be gained through analyses of reported health outcomes at project sites where the percentage of Agency funding is highest.

Many of the funded organizations provided a wide range of social service and health-related services to their clients. Funding provided through the CAPC and the CPNP is flexible in nature, and therefore the resulting funded activities vary considerably. In those funded organizations visited as part of the case studies conducted for this evaluation, project staff cited a number of ways for which funding from these two programs had been utilized, including: transportation for participants, food vouchers, structured activities, informal drop-in sessions, daycare and the funding of staff positions.

While the evaluation was unable to confirm achievement of longer-term health benefits, reviews of programs with aspects similar to those of the CAPC and the CPNP and literature in the field of child health, suggest that these types of programs may produce health benefits for participants over time.^{Footnote 48Footnote 49Footnote 50Footnote 51} Given, the flexible nature of the community-based programs, coupled with the multitude of delivery partners involved, it is recognized that these longer term health benefits are the result of many influences.

4.5 Performance: Issue #5 - Demonstration of Economy and Efficiency

Investing in children in the early stages of life can improve their outcomes across the life course reducing the economic burden they place on society. Economies and efficiencies have been realized through steps taken to improve program delivery and performance measurement. There are opportunities to use performance data to inform program decision-making.

The Treasury Board of Canada's Policy on Evaluation (2009) and guidance document, Assessing Program Resource Utilization When Evaluating Federal Programs (2013), defines the demonstration of economy and efficiency as an assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes. This assessment is based on the assumption that departments have standardized performance measurement systems and that financial systems link information about program costs to specific inputs, activities, outputs and expected results.

The data structure of the detailed financial information provided for the program did not facilitate the assessment of whether program outputs were produced efficiently, or whether expected outcomes were produced economically. Specifically, the lack of output/outcome-specific costing data limited the ability to use cost-comparative approaches. Considering these issues, the evaluation provided observations on economy and efficiency based on findings from the literature review, key informant interviews and available relevant financial data.

In addition, the findings below provide observations on the adequacy and use of performance measurement information to support economical and efficient program delivery and evaluation.

Funding

When looking at the combined program activities totals, there were slight variances between planned spending and expenditures during the period evaluated. As illustrated in Table 13 below, the fiscal year variances over the last five years were not significant and ranged between minus 1.21% and plus 1.53%. Overall, programs have spent their allocated budgets.

Data verified by the Office of Chief Financial Officer.

Economics of Investing in Children

As detailed in Section 4.1 - Continued Need, investing in children is cost-effective. Upstream preventative health interventions that benefit at-risk children and their families directly and indirectly produce economic benefits for both the health system and society more broadly. While this fact supports the role of most Agency activities linked to maternal and child health (including surveillance), the illustrations below are related to the community-based programs funded by the Agency.

In general, the literature suggests that early childhood development initiatives can influence well-being, obesity, mental health, heart disease, literacy and numeracy skills, criminality and economic participation throughout life.^{Footnote 52Footnote 53} Interventions focusing on the cognitive, language and social skills of children from birth to six years of age can contribute to their general development, school readiness, educational performance and employment prospects later in life. As well, early intervention that involves the participation of parents and caregivers is a critical upstream investment in children's health.^{Footnote 54Footnote 55Footnote 56Footnote 57} In step with these approaches, CAPC funded activities (i.e. play groups, education programs, nutrition classes) focus on the predictors of healthy childhood development by addressing the health and social development of at-risk children.

There is evidence in the literature that prenatal interventions offering social support, breastfeeding education, and nutritional counselling are generating positive impacts on the health system (e.g. increasing rates of breastfeeding initiation, fewer preterm infants, fewer prenatal hospitalizations and fewer infant re-hospitalizations)^{Footnote 58Footnote 59Footnote 60}. CPNP funded sites provide food supplements and/or vouchers, food preparation training, breastfeeding education and support, education and support on infant care and child development and referrals or counselling regarding health and lifestyle issues.

While there currently is no long-term experimental evidence per se to show the economic impact of CAPC and CPNP funding in Canada, various national and regional studies, have shown that CAPC and CPNP funded organizations have had a positive impact on program participants (e.g. healthy child development through improved infant and child nutrition and health; improved social, emotional and motor skills; reduced parent isolation through social support networks and social environments through increased parental knowledge of community services and opportunities to interact with their communities) as detailed in Sections 4.4.4, 4.4.5 and 4.4.6.

Economic Program Delivery

As mentioned in Section 4.4.1 - Program Reach, the funding allocations for both CAPC and CPNP have remained the same since 1997 (for CAPC) and 1999 (for CPNP) and have not been adjusted for inflation. Based on data from the 2013-2014 CPPMT survey with funded organizations, while approximately one-third of funded organizations reported reducing their programming due to a lack of funding, it appears that positive results continue to be achieved within funded organizations. This outcome may be due in part to the funded organizations seeking additional funds from other sources, the use of in-kind donations, and building community partnerships. (For more on the benefits of partnerships, see Section 4.4.1.)

Both internal key informants and case studies noted that inflation has caused organizations to be creative in leveraging funding from other partners. According to the 2013-2014 CPPMT survey data, 72% of CAPC funded organizations and 63% of CPNP funded organization received funding from other sources (e.g. other government departments, other orders of government, not for profit organizations) in the previous year. These funds equate to $38.3M (CAPC) and $21.8M (CPNP) respectively. While CAPC and CPNP funded organizations are often part of an already established community organization, interviewees from case studies indicated that the Agency's three year contribution agreements provide stable funding, which puts organizations in a favourable position to leverage funds. It was also noted that some organizations started off strictly delivering CAPC or CPNP funded activities, and then leveraged funding, and are now able to extend their services.

According to the 2013-2014 CPPMT survey, CAPC and CPNP funded organizations have received in-kind donations from provincial or territorial, municipal and community partners (i.e., the time and skills of public health nurses and dieticians, resources to support programming, lower rental fees). In-kind donations were valued at $7.3M (CAPC) and $9.3M (CPNP) respectively.

As detailed in Section 4.4.1 - Partnerships, funded organizations collaborate well with other community partners. Evidence from case studies and previous evaluations indicates that sites are well integrated into their communities and are working closely with partners to address maternal and child health needs. Data from the 2013-2014 CPPMT survey indicated that 90% of funded organizations had more than three different types of partners, including health and community organizations (e.g., community health centres, clinics and public health units), educational institutions and family resource or early childhood centres.

In addition to economies in program delivery at CAPC and CPNP sites, associated maternal and child health activities (JCSH, HBSC survey, CPSS and CPSP) have also demonstrated how the Agency has been able to leverage significant outputs with relatively modest investments (See Appendix - Detailed Financial Breakdown by Program Area) and strategic partnerships.

• The JCSH produces and disseminates nationally key school-based resources on child health through a unique multi-jurisdictional and multi-sectoral network of health and education leaders in Canada.
• The HBSC survey, implemented through a partnership with academic experts, is a key source of national and international surveillance data and knowledge on the behaviours of school-age children.
• The CPSS gathers surveillance data from reliable sources to make significant contributions to the evidence-base needed for policy and program decisions in maternal and child health.
• The CPSP, delivered in partnership through the Canadian Paediatric Society, benefits from established networks with health professionals to collect timely surveillance data related to a broad range of paediatric health issues.

Internal Efficiencies

Recent organizational changes within the Agency (within the Centre for Grants and Contributions, Centre for Health Promotion, and Regional Operations) have translated into greater program efficiencies.

Over the past three years, administrative efficiencies have been gained through the centralization of the management of contribution agreements for the Agency under the Centre for Grants and Contribution in the NCR. CAPC and CPNP comprise the largest materiality and numbers of all the Agency grants and contribution programs. Administration costs were reduced by 50% with a net reduction of 90 employees involved in direct administration. Where appropriate, multiple contribution agreements with single organizations were consolidated, reducing the number of agreements being administered from approximately 1300 to 850 (34% reduction) for all Agency programs, including consolidated CA's for organizations funded through one or more of CAPC, CPNP and AHSUNC funding. As well, streamlined multi-year agreements were put in place for up to three years. The centralization of contribution agreements also led to efficiencies through standardization of tools to track and report on contribution agreements.

As part of the same Agency budget reallocation exercise in 2012-2013, within the Centre for Health Promotion and Regional Operations, the national and regional roles in the management of CAPC and CPNP were realigned. These decisions have resulted in fewer financial and human resources for CAPC and CPNP. The program has adapted to these reductions in a number of ways, including streamlining performance measurement activities.

The Agency has also established a combined external and internal governance structure for a number of its children's health programs (CAPC, CPNP, AHSUNC, and FASD). The Program Governance Committee (PGC) and the Program Management Committee (PMC) were both created in July 2014. The PGC provides strategic policy direction for the programs, as well as guidance to the PMC and decision-making. The PMC's role is to oversee program delivery and provide operational advice and direction for the four children's program areas. While it is too early to assess the efficiency of these governance structures, these changes are expected to enable the programs to have streamlined decision-making and priority setting mechanisms and processes.

Internal key informant interviews highlighted how the activities of CAPC, CPNP and associated activities in maternal and child health have advanced other Agency priorities. Through the development and dissemination of surveillance data and knowledge products, support for a variety of the Agency's priorities was realized, including in the areas of chronic and infectious disease prevention. Through liaison with long established funded organizations (access points for social services), the community-based programs supported Agency priorities including: fetal alcohol spectrum disorders, family violence, injury prevention, and Indigenous status people's health. It was noted that there is room to further enhance collaboration on other priorities.

Area of Potential Inefficiency

There may be opportunities to enhance external collaboration with the JMCs. As described in Section 4.3 - Program Reach, as part of the governance structure for CAPC (1993) and CPNP (1995), these Committees were established between the federal government and each of the provinces and territories to support engagement on issues related to maternal and child health.

The 2015 internal Audit of Maternal and Child Health Programs found that some Committees are more active than others, with one Committee being inactive, and those that are active play varying roles. Through a review of records of decisions available from JMCs meetings held in fiscal years 2013-2014 and 2014-2015, it was found that meetings have focused primarily on the renewal process of existing CAPC and CPNP funded organizations. These findings indicate that the JMCs have shifted their roles over the last 20 years. The Audit recommends that the JMCs' terms of references be updated to reflect their current roles. With the shift in the JMC's role, there is an opportunity to look for other mechanisms to ensure that the FTE mechanisms are providing strategic, value-added advice.

As the Audit looked in depth at the JMCs, the evaluation did not focus on these committees; however, relevant discussions were held with a number of key informants. Some external key informants emphasized the need for provinces to improve communication with the federal government. The majority of case study key informants indicated that they were not aware of the JMCs and their role in facilitating collaboration with the provincial and territorial governments; however, they noted that the federal government could play a greater role in facilitating strategic coordination of services (e.g. coordination through a triannual planning table) with the provinces and territories.

Further, there may be opportunities to enhance information sharing mechanisms between the Centre for Health Promotion and the JMCs, as well as among the JMCs. A few JMCs members reported that they had not received performance measurement data, but thought that it would enable them to make informed decisions and to better understand the needs of their communities. Another suggestion, to support the JMCs in their work, was to establish national and regional mechanisms to share best practices and lessons learned.

Performance Measurement

For all program components (CAPC, CPNP, JCSH, HBSC survey, CPSS and CPNP), a combined visual representation (evaluation logical model) was developed for this evaluation. Document review and key informant interviews indicated that all the associated activities (JCSH, HBSC survey, CPSS and CPNP) collect performance data (e.g. annual reports, stakeholder surveys, results summaries, and evaluations).

Since April 2013, performance measurement for CAPC and CPNP has been led by the National Office. In May 2014, the Centre for Health Promotion developed a new joint Performance Measurement Strategy for CAPC and CPNP. This evergreen document provides an overview of the programs, a combined CAPC and CPNP logic model, a performance measurement plan and an evaluation strategy. It includes details on the types of performance information that will be collected and used by the programs.

There is evidence that CAPC and CPNP performance measurement data has been collected and refined over the last five years by the program. While there are some limitations, valuable efficiencies have been gained.

• Between 2010 and 2015, three performance measurement tools were used to gather information from funded organizations 2009-2010 (INET), 2010-2011, 2011-2012 (INAT) and 2013-2014 (CPPMT). There were some similarities and differences in the project level information collected by these three tools. The current CPPMT survey collects combined project level data for CAPC, CPNP and AHSUNC. This reduces the burden on funding recipients who often are funded through two or three program streams.
• Most recently (Spring 2015), CAPC and CPNP Participant Surveys have gathered participant data on program outcomes. Information from project participants was collected through streamlined surveys administered to a sample of funded organizations and participants.
• Moving forward, the performance measurement strategy outlines how the program plans to conduct the CPPMT and Participant Surveys at regular intervals. This approach will facilitate gathering comparable data over time to inform program delivery decisions. While performance measurement data was previously collected from funded organizations annually, the intention moving forward is to administer the CPPMT survey every two years.

While streamlining the tools and methods for collecting performance measurement data has resulted in efficiencies, there are limitations in interpreting and comparing the data due to the inherent nature of a flexible, community-based program design. Collecting participant data from a select number of funded organizations further limits the programs ability to attribute the outcomes to CAPC and CPNP funded activities. There are also inherent challenges in understanding the full scope of reach for the funded organizations, as well as in determining whether program outcomes are being met across all sites. Further, data collection does not consistently cover all funded jurisdictions, as organizations in the North do not complete the CPPMT and CPNP projects in Quebec do not complete the CPPMT or the 2015 Participant Surveys.

However, to mitigate these limitations, as highlighted in Section 4.4.6 - Health Outcomes, the program area might re-frame the program outcome story to more accurately illustrate the complexity of CAPC and CPNP's contributions to health outcomes for program participants. A more comprehensive performance story would focus on how the funded organizations (often with contributions from other partners) deliver a mix of programs and how these flexible, community-based programs impact the lives of program participants.

Moving forward, the program intends to continuously monitor and assess the results of programs as well as the efficiency of their management. It plans to use performance measurement information to support decisions around the 2017 renewal process. As mentioned in Section 4.4.3 - Program Reach, there are opportunities to analyse demographic and performance measurement data to optimize program reach and performance for populations that are most vulnerable to conditions of risk. With a view to continuous program improvement, performance data could be used to better understand the successes of high performing funded organizations and to share best practices with all funded organizations.

5.0 Conclusions

5.1 Relevance Conclusions

5.1.1 Continued Need

The CAPC, the CPNP and associated activities continue to be relevant programs and activities in Canada as many threats to maternal and child health in Canada persists. These investments provide programs and services that respond to the needs of these families.

While most children in Canada experience conditions necessary for healthy outcomes, there are children and families that live in conditions that place them at risk for poor health outcomes. Some of the social determinants of health that contribute to these conditions include: low-income families, those who have achieved a low level of education, teenage parents, lone-parent families, Indigenous people living off-reserves, and recent immigrants.

For many of these Canadian families, living in conditions of risk has been shown to have adverse impacts on the health and social development of their children. Parents may have life styles that negatively impact their own health and the health of their children, such as poor prenatal nutrition and smoking or alcohol consumption during pregnancy. After birth the environment of the child continues to influence their development, with a greater risk for issues such as, family violence, lack of stimulation or play, and unintentional injuries. These experiences can impact a child's ability to become a healthy and productive adult. Literature suggests that parents who understand how children develop, and who use effective parenting techniques, raise healthier and happier children.

5.1.2 Alignment with Government Priorities

The CAPC, the CPNP and associated activities are relevant in their contribution to the priorities of the Government of Canada and the Agency. These investments are intended to reduce health disparities and strengthen public health, thereby contributing to the Government of Canada's broader outcome of "healthy Canadians".

Government of Canada commitments at both the domestic and international levels are addressed through the CAPC, CPNP and associated activities. The federal government has made several domestic commitments to address health disparities, in particular as they relate to the well-being of children and their families. For example, recent Speeches from the Throne have underlined the value of families as the cornerstone to our society and noted that families expect safe and healthy communities, in part through addressing poverty and other persistent social problems. These programs and activities also contribute to international commitments, such as fulfilling the Government of Canada's strategic commitments and priories linked to both the United Nations Convention on the Rights of the Child and the Millennium Development Goals.

5.1.3 Alignment with Federal Roles and Responsibilities

It is appropriate for the federal government and the Agency to administer its maternal and child health programs and associated activities.

Existing legislative and/or program authorities speak to the federal role and provide the Agency with a mandate to target at-risk populations, engage in surveillance, provide national leadership, and share knowledge and best practices. While provinces and territories have the primary responsibility to provide health and social services related to early childhood development and maternal health, the Government of Canada plays a supporting role if there is a demand to address an issue of national scope, fill gaps for a vulnerable population and/or complement provincial or territorial investments.

5.2 Performance Conclusions

5.2.1 Achievement of Expected Outcomes (Effectiveness)

Partnerships have been integral to the development and implementation of the Agency's maternal and child health programs since their inception. CAPC and CPNP funded organizations have been able to create, maintain and expand on multi-sectoral partnerships as part a complementary continuum of services addressing child, maternal and family health needs. The benefits of these partnerships include the ability to secure additional program resources and enhance program delivery. In fact, maintaining and expanding partnerships has been critical for the sustainability of many program funded organizations. Other associated activities (JSCH, HBSC survey, CPSS and CPSP) have all been designed and successfully delivered with a broad array of longstanding partnerships, including: other jurisdictions, federal organizations, academic institutions and multi-national organizations.

The DCY, in collaboration with stakeholders, has developed and disseminated a variety of knowledge products to address gaps in community-based resources and advance the Agency's priorities. There is evidence of high use of and satisfaction with these products, although use across jurisdictions and product varies. There is an opportunity to formalize a strategic plan to ensure complementarity with resources for health professionals and parents produced by other stakeholders and to optimize limited knowledge development and exchange resources. Other associated activities (JSCH, HBSC survey, CPSS and CPSP) have been effective in working with partners to conduct research and surveillance, then develop and disseminate leading edge and timely knowledge products to address evidence-based information needs in maternal and child health.

With respect to program reach, the proportion of CAPC and CPNP funded project participants characterized by one or more conditions of risk - such as low income, low education, Indigenous status, new immigrants, teen parents, and food insecurity - is higher than rates reported for these conditions among the general Canadian population, demonstrating that these community-based programs reach pregnant women, parents/caregivers and children in greatest need. The mix of funded organizations for CAPC and CPNP has been very stable over the last twenty years or so. There have been some analyses conducted on the distribution of project sites with respect to any potential shifts in demographics and current community needs. Using demographic and geographic analyses, there may be further opportunities to analyse a variety of data sources to explore gaps and optimize program reach.

In terms of the contribution to health outcomes made by CAPC and CPNP funding, funded organizations have had a positive effect on parents/caregivers and their children. Participants have gained knowledge and built skills, specifically knowledge of nutrition, healthy living, prenatal health and infant care, as well how to access other community services. New skills were linked to coping abilities and competence as a parent, and improved relationships. Parents/caregivers adopted health practices for themselves and their families, in particular related to prenatal health, nutrition and healthy living, socialization, and smoking and alcohol cessation. There was a positive impact on the health and well-being of parents/caregivers and their children, including improvements in parents/caregiver physical and mental health, and the health and social development of children. Of particular note, with respect to prenatal programs, there were positive impacts on birth outcomes and breastfeeding.

5.2.2 Demonstration of Economy and Efficiency

Research indicates that preventing health problems early in life has one of the greatest potentials to reduce health inequities. The value of early upstream prevention is a cost-effective way to promote human health and well-being. An assessment of the economic impact of these community-based programs demonstrated that CAPC and CPNP funded organizations have successfully acquired additional funding and in-kind resources through their relationships with other partners. The associated activities (JCSH, HBSC survey, CPSS and CPSP), through strategic partnerships with external stakeholders, have produced significant knowledge development and exchange products through relatively small Agency investments.

Recent centralization of the administration and management of the CAPC and the CPNP has allowed the Agency to gain efficiencies in the delivery of these community-based programs. Administration costs were reduced through consolidating (and thereby reducing) the number of contribution agreements, reducing the number of employees administering these agreements, and standardizing tools. Realigning the management of the programs has led to the streamlining of governance structure and internal processes, such as performance measurement.

Performance measurement activities are aligned with the Health Promotion and Chronic Disease Prevention Branch's priorities and efforts have been made to collect performance data across program areas. There is a current performance measurement strategy for the CAPC and the CPNP, including knowledge development and exchange activities. Data collection tools and approaches have been refined over time. There are opportunities to continue to refine the performance story for maternal and child health activities, and to use performance measurement and demographic data to support program decision-making.

6.0 Recommendations

The Centre for Health Promotion and the Centre for Chronic Disease Prevention have built long-term relationships with stakeholders in the field of maternal and child health in Canada. In terms of performance, over the past five years, stakeholders have appreciated the Agency's leadership, funding of targeted community-based programs, and surveillance and knowledge products that speak to the needs of at-risk populations. Many improvements have been made to the governance and administration of many of these activities.

Although there have been many achievements, the two recommendations are put forward to assist the Agency in continuously improving its maternal and child health community-based and knowledge development and exchange activities.

Recommendation 1

Recognizing that many players have a role in developing and disseminating knowledge products on maternal and child health in Canada, formalize and implement a knowledge development and exchange strategic plan to ensure complementarity and optimization of Agency resources.

There is currently no DCY strategic plan in place to guide decision-making on their knowledge development and exchange priorities. While the Agency has collaborated with stakeholders to develop and disseminate knowledge products to support its community-based programs, it is recognized that provinces/territories and non-governmental organizations also produce knowledge products for health professionals and parents on topics related to maternal and child health. Key informants highlighted the importance of collaborating with stakeholders to ensure that resources are complementary and to raise awareness to expand the reach of products. Key informants also indicated a need to ensure that new and existing products address knowledge gaps and remain current. With the sun-setting of the National Projects Fund (NPF) in 2014-2015, it will be critical to review priorities and formalize a way forward for the DCY to: (a) best address current and emerging needs in Canada for community-based health information products on maternal and child health to support vulnerable populations and (b) identify and share best practices nationally across funded organizations.

Recommendation 2

Review demographic data (including GIS data) to better understand population trends and changes and explore opportunities to optimize program reach.

The distribution of program funded organizations has remained stable over the last 20 years. Evidence suggests that the CAPC and the CPNP are currently reaching the at-risk populations for which they are intended; however demand for these programs can exceed the capacity of some project sites. While some efforts have been made to review national and regional data to assess potential shifts in the demographic distribution of populations that are most vulnerable to conditions of risk, there are opportunities for further analyses.

Appendix 1 - Visual Representation of Activities

Appendix 2 - Summary of Findings

Rating of Findings
Ratings have been provided to indicate the degree to which each evaluation issue and question have been addressed.

Relevance Rating Symbols and Significance:
A summary of Relevance ratings is presented in Table 1 below. A description of the Relevance Ratings Symbols and Significance can be found in the Legend.

Performance Rating Symbols and Significance:
A summary of Performance Ratings is presented in Table 2 below. A description of the Performance Ratings Symbols and Significance can be found in the Legend.

Appendix 3 - Evaluation Description

Evaluation Scope

The scope of the evaluation covered the period from April 2010 to March 2015, and was focused on the CAPC and the CPNP. It also included the following programs and associated activities: the JCSH, the HBSC survey, the CPSS, and the CPSP.

The evaluation included maternal and child health activities funded within the Health Promotion and Chronic Disease Prevention Branch and excluded a number of other activity areas in the Branch that contribute to improving the health of children: Fetal Alcohol Spectrum Disorder, Family Violence Initiative, AHSUNC program, Oral Health, Cancer in Young People in Canada Surveillance Program, and Child Maltreatment Surveillance.

Evaluation Issues

The specific evaluation questions used in this evaluation were based on the five core issues prescribed in the Treasury Board of Canada's Policy on Evaluation (2009). These are noted in the table below. Corresponding to each of the core issues, evaluation questions were tailored to the program and guided the evaluation process.

Data Collection and Analysis Methods

Evaluators collected and analyzed data from multiple sources. Sources of information used in this evaluation included:

Literature review

• A search for Canadian and international literature from the past five years using search terms of "at-risk mothers and children", "prenatal risk behaviours", "breastfeeding rates in Canada", "early childhood intervention", "investing in children".
• After examining documents to ensure relevance, 45 articles were reviewed.

Document review

• The main purpose of conducting a document review is to obtain a comprehensive understanding of the underlying theory of maternal and child health activities within the Agency, delivery of the activities, and results over the five year period covered by the evaluation.
• Approximately 250 documents pertinent to maternal and child health and related activities held by the Centre for Health Promotion and the Centre for Chronic Disease Prevention as well as external documentation.
• There was a review of 21 prior evaluations conducted both at the regional and national levels.

Financial data review

• An analysis of financial data helps respond to questions of effectiveness, efficiency and economy.
• A review of financial data from 2010-2011 to 2014-2015 focussed on actual expenditures.

Key informant interviews

• Interviews were conducted with 44 stakeholders: Agency (n=23); other federal government departments or agencies (n=4); other non-governmental and provincial/territorial government representatives (n=15); and external issue experts (n=2).
• Interview questionnaires were developed and slightly modified and tailored for each specific stakeholder group. Guides were based on the evaluation issues and questions identified in the evaluation matrix. They were developed using a semi-structured format, including probes where helpful. These semi-structured interviews based on several key questions help to define the areas to be explored, and also allow the interviewer or key informant to diverge in order to pursue an idea or response in more detail. The flexibility of this approach, particularly when compared to structured interviews or focus groups, also allows for the discovery or elaboration of information that is important to participants but may not have been previously thought of as pertinent by the evaluation team.
• Interviews were conducted in person (when possible) or by telephone.
• They were recorded, with participant's consent, and transcribed as necessary. Data was coded and analyzed with the aid of NVIVO software.

Performance data review^{Footnote xix}

• Analyses included a review of data on performance of program activities collected by the DCY between 2010-2011 and 2014-2015, including:
  • CAPC stakeholder survey (2015), CPNP stakeholder survey (2015)
  • Children's Programs Performance Measurement Tool (CPPMT, 2013-2014)
  • Integrated National Analysis Tool (INAT, 2010-2011, 2011-2012)
  • Integrated National Evaluation Tool (INET, 2009-2010)
  • Knowledge Development Exchange Study (Winter-Spring 2014-2015).
• In terms of recent program performance data collected, below are details on three of the primary sources of performance data provided by the program and subsequently analyzed for this evaluation:
  • CAPC Participant Stakeholder Survey (2015): Through stratified sampling (type of community, regional representation, project size), 65 of the 425 (15%) CAPC funded organizations were invited to participate in the survey over a one month period (May 2015). Each funded organization was asked to administer 30 surveys to program participants. 62 (95%) of funded organizations returned surveys. The number of returned surveys included for analyses was 1386 (71%).
  • CPNP Participant Stakeholder Survey (2015): Through stratified sampling (type of community, regional representation), 101 of the 280 (49%) CPNP funded organizations were invited to participate in the survey over a one month period (May 2015). The number of surveys each funded organization was asked to administer to program participants varied according to project size (ranged from 15-35). Ninety-three (93%) of funded organizations returned surveys. The number of returned surveys included for analyses was 1792 (69%).
  • CPPMT (2013-2014): Data collection took place online between April-August 2014. The 2013-2014 survey exempted some funded organizations^{Footnote xx}. All 408 CAPC and 180 CPNP funded organizations required to complete the survey did so (100% response rate). Surveys were completed by funded organizations and reviewed by regional program consultants.
• Numbers reported in this report based on program performance measurement data may be slightly different when compared to the programs reports due to data cleaning and/or inclusion/exclusion criteria.

Case studies

• The intent of the case studies was to collect site-specific information about implementation and impact.
• Seven CAPC and CPNP case studies were completed and each included a document review, on-site tours and interviews with sponsored organization staff and partners, and site observation.  Case studies consisted of interviews with a total of 59 key informants, representing different capacities associated with the funded organization, including staff and partners.
• Case studies included a geographically distributed mix of the CAPC and the CPNP funded organizations across the six regions, from different geographic areas of the country (e.g., rural, urban, northern, remote/isolated), that employ different delivery mechanisms, and serve different populations (e.g., Indigenous status peoples, immigrant communities).
• The following organizations/communities were included: Cape Breton Family Resource Coalition (Sydney, Nova Scotia), La Maison de Famille de Quyon (Quyon, Quebec), St. Mary's Home (Ottawa, Ontario), Growing Up Healthy Downtown (Toronto, Ontario), Healthiest Babies Possible (Regina and Qu'Appelle Region, Saskatchewan), Frog Hollow Neighbourhood House (Vancouver, British Columbia), and Shared Care Child Care Society (Arviat, Nunavut).

Data were analyzed by triangulating information gathered from the different sources and methods listed above. This included: systematic compilation, review and summarization of data to illustrate key findings; thematic analysis of qualitative data; and comparative analysis of data from disparate sources to validate summary findings.

Appendix 4 - Role of Stakeholders

The Broader Canadian Context

Maternal and child health Canada is addressed through collaboration across all levels of government. To help further situate the limited role of the Agency in this response, a brief overview of the broader Canadian context is provided.

Within the federal system of government in Canada, the Constitution Act defines federal and provincial/territorial government responsibilities. In turn, municipal governments and their powers are created by provincial/territorial legislatures. Other groups in Canada that take an active role in the effort to address maternal and child health include a variety of non-governmental organizations, associations serving Indigenous Canadians, professional associations, academic institutions and private sector organizations

Using its spending power, the federal government created various children oriented programs to meet identified need for assistance.  Federal responsibilities include coordinating and collaborating with the provinces/territories and other partners to ensure an effective and efficient pan-Canadian system that supports the needs of all Canadians. The provincial/territorial governments' role is to administer and deliver a variety of services. Federal initiatives play a supportive role to provincial/territorial initiatives. Federal departments, in particular those which work to promote the health and well-being of mothers and children in Canada, support their provincial/territorial counterparts to respond to a wide range of issues, including making investments to address issues of national concern - such as the needs of vulnerable populations.

Along with the delivery of its community-based programs (CAPC and CPNP) for at risk populations, the Agency's role has been to support the coordination of activities with provinces/territories to ensure a collaborative and cohesive approach through the Joint Management Committees. The Agency's role also includes facilitating the development and dissemination of knowledge through its policy and research efforts and surveillance activities.

Other Federal Departments

Along with the Agency, a number of other federal departments work to promote the health and well-being of mothers and children in Canada, specifically:

• Health Canada's First Nations and Inuit Health Branch conducts a number of activities that are similar to the CAPC and the CPNP but are specifically for Indigenous status peoples living on reserve. These programs and activities include the Aboriginal Head Start on Reserve program, the CPNP activities on reserve, and activities related to Fetal Alcohol Syndrome. Many of these programs were established under the same program authorities as the CAPC and the CPNP; however, as the authority for Indigenous status peoples living on reserve, Health Canada conducts these activities. Health Canada also makes recommendations about nutritional needs for infant and breast feeding.
• Indigenous and Northern Affairs Canada mainly conducts programs for children and mothers living on reserve, including their First Nations Child and Family Services Program and the National Child benefit reinvestment program.
• Citizen and Immigration Canada conducts activities directly targeted at new immigrants, most of which focus more on acclimating to new communities, accessing services, and language acquisition.
• The Canadian Institutes for Health Research have a particular institute focused on child and youth health. They prepare publications on topics such as fertility, physical activity, neonatal care, mental health, backpack safety, obstetrics and childhood obesity, which focus on the general population, as opposed to those at-risk.
• Statistics Canada collects data on children through a number of mechanisms including the census and National Household Survey, the Canadian Community Health Survey, the Aboriginal Children's Survey, and formerly the National Longitudinal Survey of Children and Youth (last cycle in 2008-2009). Statistics Canada also publishes information on children related to topics such as children at school, children of older mothers, readiness to learn, mental health, and smoking.
• Also, while not a federal department, the Canadian Institute for Health Information works with stakeholders to create and maintain a broad range of health databases, measurements and standards, and develop reports and analyses from their own and other data, some of which is related to maternal and child health, however, data and reports are focused on the health care system, rather than the more public health focus of the CAPC and the CPNP. (Approved by the F/P/T Ministers of Health in 1992, the CIHI is an independent, not-for-profit organization that provides essential information on Canada's health system and the health of Canadians.)

Provinces and Territories

Provinces and Territories also carry out activities in the area of health promotion and disease prevention for mothers and children. For example:

• The British Columbia Early Years Strategy outlines the government's commitment to supporting young children and their families between birth and age six by using a community development approach which allows local groups to work together to assess, identify and plan for the unique needs of young children in their community.
• Alberta's Approach to Early Childhood Development (2013) titled Together We Raise Tomorrow, is a unified agenda to achieve better outcomes for children now and into the future, which relies on shifting to a more connected approach across government and communities.
• The Healthy Child Manitoba Strategy is the government's prevention and early intervention strategy to achieve the best possible outcomes for Manitoba's children with respect to their physical and emotional health, safety and security, earning success and social engagement and responsibility.
• Led by Ontario's Ministry of Health and Long-Term Care,the Healthy Kids Strategy is a cross-government initiative to promote children's health. It focuses on a healthy start in life, healthy food, and healthy active communities.
• New Brunswick's Child and Youth Strategy is an initiative to improve services to children, youth, and families through greater collaboration between four key government departments (i.e., Health and Wellness, Education, Community Services, and Justice).
• Nunavut's Healthy Children Initiative provides funding for communities to develop or enhance programs and services for children and their families from the prenatal stages until age.

Appendix 5 - Knowledge Development and Exchange Products

Appendix 6 - Mapping of the CAPC and the CPNP Sites (as of 2010)

Appendix 7 - Detailed Financial Breakdown by Program Area

List of Acronyms