CIET conducts community intervention trials to demonstrate the extent to which a given solution to a problem will be effective. At its simplest, this is a comparison of outcomes in one group of communities receiving an intervention with outcomes in a similar group of communities where the solution has not yet been tried.

A baseline survey in all communities’ documents initial levels of the outcomes of interest.  Wherever possible we randomly allocate communities to “intervention” and “control” (delayed intervention) status, often stratifying the sample of communities to ensure similar levels of factors potentially related to the outcome between the intervention and control groups. When they are randomized, community intervention trials are known as a cluster randomized controlled trials (CRCTs). A further survey measures the outcomes after the intervention and allows us to compare between intervention and control communities. If the intervention is shown to be beneficial, our commitment is to extend it to all communities in the sample.

Examples of CIET community intervention trials include:

A study of the impact of evidence-based communication materials about maternal and child health used by Lady Health Workers in Sindh Province, Pakistan.  [ Click here to read more ]

A study of the impact on childhood immunization rates of evidence-based community dialogues in Lasbela district, Sindh Province, Pakistan. [Click here to read more] 

A large CRCT in Mexico and Nicaragua, testing a “green” alternative (camino verde) to conventional dengue control measures through evidence-based community mobilisation. [Click here to read more]

A CRCT, in a stepped-wedge design, to test the impact of universal home visits to pregnant women and their spouses on maternal and infant health and male attitudes. [Click here to read more], [ii], [iii], [iv]

Knowledge synthesis is the aggregation of existing knowledge about a specific question by applying explicit and reproducible methods to identify, appraise, and synthesize studies relevant to that question. Products of knowledge synthesis include scoping reviews, systematic reviews and meta-analyses.

Many scoping and systematic reviews include not only studies published formally but also those in the “grey literature” such as blog posts, reports published on websites, conference publications, and theses. But there is a great deal of knowledge that is not written. Most knowledge related to indigenous medicine, for example, is not available in written, much less in accessible published form. And community knowledge based on lived experience is also a valid form of knowledge.

CIET’s contribution to knowledge synthesis includes refining approaches to documenting stakeholder knowledge through cognitive mapping (see section on fuzzy cognitive mapping) and developing methods for juxtaposing stakeholder and literature knowledge based on creating cognitive maps from literature reviews to compare with stakeholder maps.

• • Fuzzy cognitive mapping: An old tool with new uses in nursing research.
  • Weight of Evidence: Participatory Methods and Bayesian Updating to Contextualize Evidence Synthesis in Stakeholders’ Knowledge.

Some recent knowledge syntheses from CIET include:

• • Systematic review of causes of short birth interval (kunika). 
  • Systematic review of flyout childbirth from Nunavik. 
  • Systematic review of HIV-sensitive social protection. 

Cognitive mapping is frequently used as a decision support tool to better understand causal contributions to outcomes or decisions. It has wide applications in ecological management, decision-analysis, information technology, economics, organizational behaviour, as well as in medicine. Regardless of the application, many cognitive maps start from a person or group’s perspective of what is relevant to a specific question or issue. One person or group’s perspective could be completely different to another’s view, and each set of views is probably partial, incomplete and changing over time. But when we construct fuzzy cognitive maps, the shared language of concepts and space become comparable and sometimes combinable.

Our use of cognitive mapping is a conceptual shift from quite mechanistic artificial intelligence – and the attendant issues of “programming” – to the view that these maps are soft models of how people see things. This conceptual shift also requires that we move beyond depending solely on a regularity or variance-based understandings of causation, to one that also relies on an understanding of the social and structural processes that lead to observable outcomes.

More information and guidance about creating and analysing fuzzy cognitive maps can be found here.

For every behavioural outcome there are intermediate stages on the way to a particular action or practice. Making these stages explicit can help in specifying indicators to be measured in studies of interventions aimed at changing a practice or behaviour. Using concepts from social psychology and decision theory, CIET uses the acronym CASCADA to describe an unpredictable but broadly sequential flow through conscious knowledge, attitudes, subjective norms and positive deviation from subjective norms, the intention to change, the sense of agency (collective and individual) that change is possible, discussing it and, finally, making the behaviour change. Each stage may be illustrated with reference to HIV and AIDS prevention.

Conscious knowledge: In primary prevention, we might be concerned about knowledge about rights (for example, to say no), modes of transmission of sexually transmitted infections, prevention mechanisms including condoms, misconceptions and myths about HIV/AIDS. In secondary prevention, the value and nature of HIV testing and for tertiary prevention, we might focus on knowledge of antiviral therapy.

Attitudes: Attitudes of interest in primary prevention include the belief that, for example, women enjoy sexual abuse or that people with AIDS should live apart from others. In secondary or tertiary prevention, an example might be the belief that precautions like condoms are worth the effort.

Subjective norms: In primary prevention, the belief that friends or neighbours see things in a certain way (people around here believe that women enjoy sexual abuse) can be part of a negative cultural environment or, if modified, part of an incentive to change. The positive deviation from a negative social norm (in secondary prevention, my friends do not believe condoms are worth the effort, but I believe they are) can provide an early positive indicator of a behaviour change strategy.

Intention to change: The intention to go for HIV testing and, in tertiary prevention, intention to use a condom during intercourse, are useful markers of progress towards behaviour change.

Agency: Individual agency or sense of self-efficacy can be measured by the declared ability to insist on using a condom (secondary prevention). Also important is collective efficacy, the perception that a particular issue can be dealt with in the community.

Discussion or ability to talk about it: The ability to talk about an issue (refuse unwanted sex, use a condom, go for HIV testing or take up an ART offer) often precedes the actual practice. This also provides pointers for interventions in primary, secondary or tertiary prevention.

Actions/practices/behaviors: Prevention-related practices include sexual violence, going for HIV testing, using condoms, multiple concurrent partners, and uptake or adherence to ART. Like any prevention outcome, behaviour might be conditioned by factors like age or sex. A general concern is that of the social, cultural or economic factors supporting risk behaviour. The key to understanding preventive impact is to relate individual behaviour change to individual exposure to interventions.

Neither the steps in the cascade nor its slope are fixed. In a given situation, x% will have conscious knowledge of AIDS prevention, y% will have helpful attitudes and z% will have the agency to do something about it. The implication of the CASCADA model is that a positive attitude counts for more than conscious knowledge, and agency counts for more than a positive attitude. We do not know exactly how much more – this is probably situation and subgroup specific.

CIET has used CASCADA in resource-poor settings in studies of health behaviors,[i] [ii] design of interventions,[iii] [iv] and analysis of intervention effects.[v] [vi]

Communication is at the core of any development effort. However, development agencies, governments, researchers, and academics disagree on what to use communication for and how best to communicate with people.

In CIET, we see communication as the way to build the voices of the community into planning, particularly the voices of the most disadvantaged. We partner with communities to help them identify and solve their development challenges in their own terms and in a sustainable manner, based on the participatory search, lay interpretation and public discussion of local evidence. We call this process SEPA: socializing evidence for participatory action.

SEPA is itself an intervention; sharing the evidence with stakeholders allows them to plan and implement actions to improve their situation, based on the evidence. The impact of the intervention can be measured, for example in a randomized controlled trial when the sharing of evidence is randomized to some communities while other communities serve as (delayed intervention) controls.

Recent examples of SEPA as an intervention include:

Camino verde trial for dengue prevention, where evidence about the mosquito life cycle and breeding places was the evidence shared. [Click here to read more]

Universal home visits to pregnant women and their spouses in Bauchi State, Nigeria, shared local evidence about risk factors (actionable at household level) for maternal and child health.

• Protocol.
• Impact on maternal health.
• Impact on child health.
• Impact on male attitudes.

An ongoing project in Bauchi, Nigeria, shares evidence about threats to good adolescent sexual and reproductive health (ASRH)(based on local FCM and a survey) with communities who take actions to protect ASRH.

• Protocol.

A central objective of CIET’s approach – usually geared for planners — is to estimate the change in risk associated with a given potential determinant, usually a programme intervention. A household survey might produce a certain status indicator, such as that 55% of women attend prenatal care. Repeated surveys might even be able to detect a change in the associated health status, say low birth weight. The challenge is to decide whether the change in status is in reality an impact of the programme input. This is rarely a simple exercise; it requires documentation of exposure to programme interventions, and it requires that many other explanations for change be excluded. Even with modern multivariate analysis, survey data may be too shallow to provide much insight into what ought to be done about the status.

Valuable additional insights may be obtained from more qualitative approaches such as direct or participant observation, community meetings, Delphi techniques, focus or nominal group discussions and case studies. Rapid assessment procedures can deepen the knowledge base on the way things work, based on external observer opinions. On their own, these are rarely useful for measuring impact. Since qualitative techniques are based on small samples, repeating the observation or focus group discussion at another time can produce a different result by chance alone.

If the depth of knowledge generated by the qualitative studies is to be generalised, then its representativeness must withstand scrutiny in the manner of epidemiological data. Usually, qualitative techniques provide intense detail on a purposively chosen segment of society. The insights represent what goes on in those chosen segments. And if the segments chosen for qualitative study are not the same as those for quantitative study, linking qualitative appreciation to macro-level data is difficult. Typically the sample survey covers one domain (the district or country); the focus groups, participant observer reports or case studies reflect another domain.

The “sentinel sites” that characterise CIET methods are essentially survey clusters whose size has been increased to offer a representative panel of “mini-universes”.There is no sampling within a site. Each site can be followed over time by returning to it and repeating the enquiry. Like age, sex, or education, “site” can be considered an individual factor or grouped by characteristics (geography, religion, prevailing opinions, or price of basic grains) to provide a link between qualitative and quantitative data, between data from the household and data from the community or local environment.

In the evaluation of the 1998-2003 Health and Population Sector Programme in Bangladesh a detailed institutional review of facilities serving each sentinel site enabled CIET to identify two characteristics of health centres – separate toilets for women and use of screens when examining patients – which were key determinants of client satisfaction. In South Africa’s Free State province  variations in attitudes and performance of health workers in the different facilities providing antiretroviral therapy (ART) proved to be key influences on community engangement with the ART programme. In a baseline CIET survey on social vulnerability in Venezuela in 2003, a composite indicator of environmental conditions in each site was established on the basis of interviews with community leaders. Where these conditions were weakest, the risk of chronic child malnutrition as measured by height for age was more than double that in sites with better environmental conditions. This kind of analysis is not easy with typical random cluster surveys that use much smaller clusters.

Attributes of place, a specific aspect of local service, or the result of a focus group discussion – have a quite different analytical relevance when observations or focus groups are repeated in a comparable manner across a panel of communities chosen to be representative of the geographic area in question. These differences can be related to programmatic input and other factors that might be heterogeneous across different sites. The impact assessment is based on the time sequence and the heterogeneity among sites.

Some of the most disadvantaged people in the world live in geographically or socially isolated small groups with little access to services or opportunities of any kind. Such is the case of indigenous rural populations relegated by society to the most remote and least accessible locations in their countries. In urban centres as well, ethnic minorities often find themselves scattered, marginalized and effectively isolated. Gathering actionable evidence that these people can use to assert their human rights presents special challenges.

CIET has been working to develop epidemiological methods particularly suited to this challenge.

Since 1987 CIETmexico began studying the special needs of working children scattered throughout the city of Acapulco. In some of the most remote rural areas of Guerrero State the project on Microregional Planning assisted communities to identify and carry out their own solutions to pressing community problems.

CIET’s work with Aboriginal Canadians, both in cities and on reserve, began in 1995 and continues to the present. See Indigenous Canada.

A discussion and an example of methods for research among isolated and marginalized groups can be found in a doctoral thesis by Professor Lorenzo Monasta of the Institute for Maternal and Child Health in Trieste, Italy: Macedonian and Kosovan ROMÁ living in “Nomad Camps” in Italy: Health and living conditions of children from birth to five years of age. The full thesis, an English summary, and an Italian summary, are available from the library. The main findings of the thesis are published in: Monasta L, Andersson N, Ledogar RJ, Cockcroft A. Minority Health and Small Numbers Epidemiology: A Case Study of Living Conditions and the Health of Children in 5 Romá Camps in Italy. American Journal of Public Health 2008; 98(11):2035-2041. 

With the explosion of biomedical publishing in the latter half of the 20th century, keeping up with primary research has become an impossible feat for policy makers and practitioners. There has been a proliferation of systematic reviews as one of the key tools for evidence-based health care.  This presents both opportunities and risks. The opportunities are to base decisions on accurate, succinct, credible comprehensive and comprehensible summaries of the best available evidence on a topic.

The goal was to create a valid reliable and useable instrument that will help users differentiate between systematic reviews, based on their quality, and to facilitate the development of high-quality reviews. This exercise led to AMSTAR and was reported in a series of papers. Candidate domain items were identified through a review of the evidence and existing tools. Nominal group consensus methods were used to identify best items and to gauge face validity.  A comparison among the draft instrument and competing instruments was undertaken to establish its measurement properties. AMSTAR, the resulting instrument has good inter-rater agreement, test-retest reliability, face and construct validity, and is easy to use.

We believe that AMSTAR has significantly advanced the practice of assessing the methodological quality of systematic reviews. It has received endorsements from the Canadian Agency for Drugs and Technologies in Health CADTH and several authors and had been cited nearly 400  times as of early 2013. It has been translated into Japanese, French and Spanish. It is used to evaluate reviews, as a guide to conduct of reviews and as an aid to teaching about systematic reviews.

See:

Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology 2007; 7:10. Available from:  http://www.biomedcentral.com/content/pdf/1471-2288-7-10.pdf.

Shea, B. J., Bouter, L. M., Peterson, J., Boers, M., Andersson, N., Ortiz, Z., … & Grimshaw, J. M. External validation of a measurement tool to assess systematic reviews (AMSTAR). PloS one 2007, 2(12), e1350.