In order to carry out a Research, Monitoring or Evaluation project, choosing the right kind of methodology is paramount depending on the research objectives. Irrespective of whether the study is qualitative or quantitative or mixed, it is important to choose the method that best suits the study needs in the context of the overall requirements of the research. DevInsights experienced team is creative and innovative in exploring alternative methods that can be adopted in a research. The decision to adopt a method is based on some of the key questions like, objective of study, target respondents, users of research, geography, who collects data, the analysis plan, ethical issues etc.
The research team at DevInsights are completely hands on and well equipped in utilizing the most appropriate techniques and designs suiting the requirements of the project. Some of the methodologies that we adopt are:
Causal research is conducted to identify and establish the extent and nature of the cause and effect relationships between the variables. Causal research is utilised to determine how the changes to the independent variables influence the dependent variables. It is conducted through different methods, including experimental, quasi-experimental and non-experimental.
RCT is a comparative, experimental study design that is performed under controlled conditions whereby the allocation of participants is random to treatment and control or comparison groups. Considered as the ‘gold standard’, it is a rigorous design that helps to determine the existences of cause-effect relationship between the intervention planned and the outcome. The study design requires that the population receiving the treatment is chosen at random and so is the population for the control or comparison group. Essentially an RCT involves planning at the stages before the intervention is initiated and is thus beneficial for impact evaluation wherein the sample is large, and the intended impacts could be measured readily. The randomization that is undertaken in an RCT is dependent upon the implementation level of the programme and thus can either be an Individual level RCT (IRCT) or Cluster RCT, wherein the randomization occurs at the cluster level.
Similar to the experimental design, the Quasi-Experimental Design is used to establish or test the causal hypothesis; to identify the extent and nature of the cause-effect relationship. As the name suggests, this design is quasi-experimental and thus remains less robust and rigorous than an experimental design. The QED does not include random allocation of the treatment group. In this design, the control or comparison group is identified in terms of being as similar as possible to the treatment group, before the implementation of the intervention. This comparison or control group is utilised to capture the status of population in terms of outcomes, in the absence of such intervention. This helps to establish effectively what would have happened in the absence of the intervention. The QED methods could also be used in retrospective manner.
To establish or generate a valid control or comparison group, various techniques can be adopted, including Regression Discontinuity Design, and Propensity Score Matching (PSM). These techniques help to establish a statistically significant treatment and control sample and to reduce the risk of bias of selection of sample.
The method of matching is utilised to construct a comparison group while relying on matching the observed characteristics of the treatment group, using statistical techniques. The assumption underneath the method of matching remains that based on the unobserved characteristics there is no difference in the treatment and control groups.
One of the matching techniques is the PSM which attempts to estimate the effect of an intervention or treatment by accounting for the covariates that predict being part of the intervention. Essentially PSM is used to estimate the difference in outcomes between the treatment and control group, so the individuals are matched on their propensity score which is calculated as the likelihood that the individual would receive the treatment given their observable characteristics. The samples are restricted to the units whereby the region of common support appears in the propensity score distribution. The individuals from treatment group are matched with similar individuals from comparison group and then the average difference in certain chosen indicators (of interest) is calculated.
Difference-in-differences (DID) is a method utilised at the time of analysis in the quasi-experimental design whereby the changes in the outcomes over time between the treatment and control groups are compared to estimate the impact of the treatment. Also known as the double difference method, DID is used to establish the impact estimate through comparing the difference in outcomes between treatment and comparison group before the intervention (baseline) and following the intervention (end line). The method involves that the indicators of interest are identified and then the difference in the values of these indicators before and after for the treatment groups are compared with the difference in the values before and after for the comparison group. It controls the factors which are constant over time in both the groups.
Conducted to describe the characteristics of a population or phenomenon, descriptive research is utilised to identify variable and hypothetical constructs.
One of the types of descriptive research, cross-sectional study is utilised to establish the prevalence of the outcome of interest for a given population within the universe at a given point of time. The essential aspect of cross-sectional studies is that this design is a one-time measurement of exposure and outcome, and thus is mostly relevant to establish prevalence or comparisons among groups or distribution of variables at any given point of time. A cross sectional study design provides the snapshot of particular situation/intervention/condition/groups at a given moment of time. Furthermore, cross-sectional study can assess several outcome and risk factors due to studying populations at a single point of time. It is due to the aspect of being a one-time measurement, that the cross sectional studies cannot be effectively utilised to derive cause-effect relationships.
Another type of descriptive research, longitudinal study involves measurement of the same sample (which remains fixed) on the same variable repeatedly over a period of time. The essential concept of longitudinal study remains the tracking of the same people along the same variables taking multiple measures, bringing about more acute observations and discerning trends over time. Also known as a panel study, this type of research focusing upon the same sample allows for greater in-depth understanding and thus provides more scope to suggest a causal relationship than a cross-sectional design. The data from longitudinal studies can facilitate in analysing of duration of a specific phenomenon, due to repeated measurement over time.
A type of longitudinal study is the Longitudinal Cohort that involves the sample in the study to constitute cohorts who are examined over the time on same variables. A cohort can be understood to be a group of people sharing a particular defining characteristic, usually those who experience a common event in a given period, such as joining or leaving some institution, or birth. Thus a cohort study being a descriptive research, charts the statistical occurrence in a given sub group, that share similar or same characteristic that is relevant to the research problem. The longitudinal cohort studies have the advantage to establish a particular sequence of events, following a particular change over time and in identifying the relating events to specific exposures (defined in terms of timing, presence and chronicity).
An exploratory research is utilised to address or investigate into a problem or issue where an inadequate knowledgebase exists to provide insights and understanding. This type of research helps to establish familiarity with basic details, and concerns, for generating new assumptions and ideas and developing a tentative theory or hypothesis. There are various methods used in exploratory research such as secondary research, focus group discussions, in –depth interviews, standardized patients.
The focus group discussions help to explore the nuances and depth of opinions regarding an issue and to understand differences in perspectives.
In-depth interviews can be described as qualitative key informant interviews (loosely structured conversation) with the clients as well as people who have knowledge regarding the topic that is being explored. IDIs allow the researcher to explore the subject in depth, examinations of the systems and processes, and to also refine the data collection efforts. This format guided along a certain structure provides the space for detailing along necessary components that require an enhanced focus from the point of view of the study.
Embracing the phenomenon of interdependent relationships between the married couples and taking in account their roles in the family and their dynamics in the family and as a couple, dyad interviews are generally conducted with married couples to garner information related to the intervention of particular aspects of study.
PRA methods are participatory or collaborative in nature, in which the evaluation approach provides hands-on practical training in monitoring and evaluation techniques to the partners and stakeholders involved. Here the program implementers are actively involved in all steps of the evaluation process. PRA Tools are of three types – Space-related tools, Time-related tools and Rationale tools
- Space related: The space-related PRA techniques are useful tools to explore the spatial dimensions of people’s realities. The primary objective is to understand how people perceive and relate to space in their communities. Various types of visual maps are utilized in the application of space-related methods.
- Time related: Time-related PRA methods assist in exploring the temporal dimensions of people’s realities. The originality of these PRA methods is that they allow people to use their own concept of time.
- Rationale related: The related-related PRA techniques are useful tools to explore the rationale behind specific actions and tried mostly to establish cause-effect relationship.
Standardized patient is a healthy person trained to portray the personal history, physical symptoms, emotional characteristics and everyday concerns of an actual patient. A Standardized patient is a person carefully recruited and trained to take on the characteristics of a real patient thereby providing an opportunity for the clinicians to learn and to be evaluated on learned skills in a simulated clinical environment. Such participants provide constructive feedback from the patients unique perspective, to learners at all levels of training.