Can the research studies be replicated elsewhere?

Since the time of Aristotle, philosophers have been engaged in answering the big questions such as; what is the nature of reality? How the knowledge about this reality can be accessed and ascertained? Responses to these questions determine the way we perceive the world and then go on exploring and investigating it. One of the most dominant paradigms in the philosophy of the research has been the positivism. Positivism takes the reality as fixed, which can be broken into pieces. These pieces are put together to make an understanding of the whole. The positivist perspective in social sciences uses principals from natural sciences to establish the truth about reality. Once a truth claim becomes a theory, it can be applied to other contexts. It was Auguste Comte, who first used categories from natural sciences to define a social reality. For him, like natural sciences, the social world could also be categorized, measured and predicated through repeated observations which he called ‘social facts’. It is evident from the following passage where he explains:

“All good intellects have repeated, since Bacon’s time, that there can be no real knowledge but that which is based on observed facts. This is incontestable, in our present advanced stage; but, if we look back to the primitive stage of human knowledge, we shall see that it must have been otherwise then. If it is true that every theory must be based upon observed facts, it is equally true that facts cannot be observed without the guidance of some theory. Without such guidance, our facts would be desultory and fruitless; we could not retain them: for the most part we could not even perceive them.” (The Positive Philosophy, trans. Harriet Martineau (1853), Vol. 1, 3–4)

This worldview dominated the scene in the 19th century when all the social sciences were in their embryonic stage. The primacy of positivist worldview was further reinforced by the use of methods such as experiments, observations, etc. in gathering the information or facts about the reality. Most of the colonial and imperial investigations were informed by positivist worldview where fixed categories were used to define the reality. While these investigators professed about the objectivity and neutrality in their research studies, their categories itself were subjective. The emergence of feminist, anti-race, anti-colonial movements questioned the feasibility of the categories proposed by positivism. The further realization that the reality was not as objective as positivist perceived and the knowledge was not value-free, led the development of post-positivist paradigm. For post-positivists, knowledge was not based on a priori assessment from an objective individual (Taylor, Thomas R.; Lindlof, Bryan C., 2011) but rather upon propositions. The post-positivists took an ontological position where reality could never be known fully, and mistakes in measuring the reality were inevitable. This worldview recognized the importance of contexts while making sense of the reality. Thomas Kuhn, a proponent of post-positivism puts it eloquently:

“Far from being magisterial in its objectivity, science was conditioned by history, society, and the prejudices of scientists.” (Thomas S. Kuhn (2012). “The Structure of Scientific Revolutions: 50th Anniversary Edition”, p.113, University of Chicago Press)

From this vantage point, a theory was only valid if it was relevant to the context of the particular reality. The post-positivist realization that knowledge was a result of social construction and the categories were essentially social constructs, led to the acknowledgment of bias while framing questions, designing methods, or even in analyzing and interpreting the findings.

All sorts of program evaluation/impact studies find their roots in the post-positivist framework. Generalizability assumption in these studies demands that the findings should not be taken at the face-value instead scrutinized for the relevance in a given context. It would not be enough to just cite the evidence. For instance, a policy is effective in one place would not deem it suitable for another place. To say that Odd and Even policy was effective in Delhi, does not itself qualify it for implementation at some other place. It has to be relevant and suited to a given context. The evidence-based policy movement was criticised for apparently these reasons only as it critics argued that it offered a one-size-fits-all policy solution to the problems, without considering the contextual and policy-environmental factors. Realist such as Ray Pawson has argued that the practical evaluation approach to public policy does not take into account the complexities involved in implementing the policy/program somewhere else. For these complexities, he provides a useful ‘checklist’ with easy to remember acronym; VICTORE which stands for volition, implementation, contexts, time, outcomes, rivalry, and emergence. He comments:

“To be sure, no two programmes will be exactly alike but we would be unable to recognise them as programmes unless they contained common elements: resources, implementation chains, stakeholders, inputs, outputs and so on. We would be unable to understand that the problems confronted were indeed wicked without the pre-existing and chequered family history of attempts to overcome health inequalities, underdevelopment, drug abuse, crime victimisation, and so on. On balance, I prefer Mark Twain’s starting point that history does not repeat itself, but it rhymes.” (p.55)

Among all paradigms, only post-positivists give scope for replicability of research study/program/policy. Though the investigators have first to make sure that the given research/program/policy are relevant to the context and it is the best mechanism to achieve the given outcomes.

References:

Pawson, R., Tilley, N., & Tilley, N. (1997). Realistic evaluation. Sage.

Taylor, Thomas R.; Lindlof, Bryan C. (2011). Qualitative communication research methods (3rd ed.). Thousand Oaks, Calif.: SAGE. p. 5–13.

EdTech, Educational Data Sciences, Childhood Studies, History of Education & Policy Sociology, curate @central_ed , Currently @IIMAhmedabad