Reliability and Validity in Research

Reliability and Validity in Research The debate whether qualitative methods are reliable and valid have been contested for a long time between qualitative and quantitative researchers. Quantitative researchers approach research in a positivistic way where they believe there is a single truth, behaviours can be explained by Universal laws and research should be done objectively (Research Methodology, 2016). Most of the studies also involved using statistical methods to analyse. Today, quantitative research is valued by government and policy makers more as they are more reliable, generalizable and easier to analyse (Cannella and Lincoln, 2004 cited in Tracy, 2010). However, qualitative studies study issues in depth hence it should be valued more. In this essay, the definition of reliability and validity in quantitative and qualitative research will be discussed. Then requirements of a good qualitative research will be explored and interview as a data collection method will be evaluated. Reliability in quantitative research is defined by (Joppe, 2000, cited in Golafshani, 2003) as the Extent which results are consistent over time and accurate representation of the total population under study. This means similar results should be replicable at a different time and the sample should represent characteristics of the general population. Validity is whether the research measure what it intends to and how accurate the data is (Golafshani, 2003). Being objective is also important to ensure data are not biased which affects the reliability of the data. Meeting these objectives will ensure good quality quantitative research. In contrary, qualitative research focuses on structures and process behind to have a deeper understanding of a topic through methods such as interviews and discourse analysis. McDowell (1992) called these researchers critical realists and she believes the work they do can help explain a phenomenon whereas quantitative data are only useful in identifying the pattern as correlation does not equal to cause (empirical realists). Qualitative methods also aim to show that the world is complex (Schoenberger, 1991) and messy, therefore reliability does not apply and generalised as diversity is more important (Flyvberg, 2006). The meaning of validity is different in qualitative research as it does not measure anything. Golafshani (2003) defines it as Whether a study appears to be reasonable and appropriate. This is related to the method of data collection and analysis. However, some academics believe that validity should not be applied to qualitative research as the term is defined from a sci entific background. Therefore, alternative criteria are developed as part of a guideline to improve the quality of qualitative research which will be explored in the next section. There is an ongoing tension between rigour and creativity in the qualitative paradigm. To achieve good rigour, a universal standard could be developed to ensure the quality of work is consistent. However, this is rejected by most qualitative researchers as it limits the creativity of their work and results are usually unpredictable which means it will be difficult to meet the standard. Researchers hence need to strike a balance between the two and a guideline will be the most appropriate way to do so (Bailey et al, 1999). The guideline will allow researchers own interpretation of how rigour could be met based on their own circumstances. This will improve the reputation of qualitative research in the society and accepted more widely by the quantitative researchers and government agencies. It also allows researchers from different paradigm to learn from each other and improve their own research methods (Tracy, 2010) One such guideline was proposed by Guba and Lincoln (1985) cited in Baxter and Eyles (1997) who suggested four criteria evaluating qualitative studies to achieve rigour and trustworthiness. They try to apply criteria from quantitative into qualitative research. Firstly, it should be credible so that it is accurately representing the findings and can be trusted. The reader should be able to understand the issue easily by making the whole research and writing process plausible and persuasive (Richardson, 2000 cited in Tracy, 2010). The second criteria is transferability where findings would be useful outside the study. This is similar to generalisability for quantitative research. However, there is less emphasis in qualitative research on that as each case would be different and no clear conclusion would usually be made. It can be transferable to the reader when they make their own interpretation and decide how each specific case could be useful to their own daily life (Tracy, 2010). T he third criteria is dependability developed from the idea of reliability. Although data collected from qualitative methods would not produce consistent results, the way data is transcribed, coded and analysed can be agreed between multiple researchers and participants to ensure interpretation would be consistent. The final criteria confirmability is based on objectiveness. This is The degree which findings are determined by the respondents and not by the motivations of the inquirer. (Lincoln and Guba, 1985. P.290 cited in Baxter and Eyles, 2010). This is needed to reduce the bias in qualitative data collection and results which is one of the main reason why it is less popular than quantitative research. Researcher being reflexive throughout the study is the main way to achieve this which will be explained further later in the essay. The criteria above could now be used to evaluate interview as a data collection method. There are three types of Qualitative interview which vary in the degree of freedom given to the participant when responding. Structured interview involves answering a set number of questions prepared before. A semi-structured interview provides a better interaction between the participant and the researcher, it is a guided conversation with a few predetermined question and prompts used when more information is needed, which is usually written as an interview guide (Longhurst, 2009). The guide improves the credibility (Baxter and Eyles, 1997) as it allows a better answer to be constructed by the participant. An Unstructured interview is rarely used as it is very difficult for the participant to speak for a long period of time on the same topic and could easily go off topic. Semi- structured is the most common qualitative research method as it strikes the balance between the ability for the participant to speak freely and structuring the interview (University of Leicester, Unknown) pro viding a certain degree of credibility. Conducting interview requires choosing participant carefully through sampling. Purposive sampling is usually used in which means choosing participants which fit with researchers criteria who are usually an insider to the topic of research. This means participants will be able to talk in great detail which is the aim of a qualitative interview. Other sampling methods such as snowball (interviewing friends of participants) and convenience are sometimes used due to difficulty in finding insiders. However, this will reduce the credibility of the research as they might have limited to say due to lack of knowledge in the area (Baxter and Eyles, 1997). Interviewing friends will make it harder to achieve credibility as participants might give accounts to satisfy the need of the researcher which might not reflect the real situation. The sample size is seen by quantitative researcher an important factor to produce good quality research. However, in qualitative interview, the number of intervie ws conducted is usually limited to around to 20 due to time and financial constraints (Baker and Edwards, 2012) Moreover, a large sample size is not needed as it is enough when the theoretical saturation is reached (Glaser and Strauss, 1967 cited in Bailey et al, 1999). This is where the themes identified during analysis are repeating and no new knowledge would be gained from interviewing more people. During the interview and analysing process, the language the researcher use is crucial to the outcome of the result as interviews are based on interpretation by different people (McDowell, 2010). The way a question is asked could receive a different response as same words might have different meanings to people. During transcription, the researcher might make his/her own interpretation when something is unclear and the final level of interpretation is made by the reader themselves (Schoenberger, 1991). To make the interview results transferable to the reader, the transcription and analysis should be narrative with thick description and minimal interpretation from the researcher. This should include detailed description and illustration of the situation (Tracy, 2010), for example using a direct quotation from the interview is a good way to achieve it. Methods to improve the credibility of research includes member check and language training. Member check involves sending a copy of the transcript and analysis to the participant (Long and Johnson, 2000) and ask for feedback. The researcher could also understand the language used in an industry before conducting an interview (Schoenberger, 1991). Both of these methods will reduce the chance of misinterpretation. After transcription, coding is done to identify common themes from different accounts. This is done by the researcher who chooses the themes. Due to limited space in journals, some section of the interview is therefore not analysed. As mentioned before, coding needs to be consistent for the research to be dependable through methods such as collaborative coding or the use of a coding book. Triangulation is a common method used to improve the credibility by using multiple sources and methods which give the same conclusion (Denzin, 1978 cited in Tracy, 2010). For example, using multiple quotations to support a statement made would improve the credibility of the research. Most importantly, researchers should be self-reflexive to improve credibility. They should think about their own positionalities and decide how to declare it to the reader (Longhurst, 2009). One way is to be transparent about how decisions are made during the research such as sampling and coding strategy. This could be attached as an appendix (Creswell and Miller, 2000 cited in Tracy, 2010) at the end to help a reader to decide whether they agree with researchers interpretation. Overall, interview as a data collection method can meet all the criteria proposed by Guba and Lincoln (1985) if the researcher is careful about every decision that was made and the language used to communicate with the reader. The guidelines are developed based on the idea of validity hence it can be said interview is also valid as a method but not in a positivistic way in terms of accuracy. They also suggested that When validity is present, it also represent reliability. Therefore, reliability does not have to be considered as it is irrelevant to qualitative studies. Currently, researchers are not being explicit enough in terms of their rationale and how things are done (Baxter and Eyles, 1997), the guideline will therefore help researchers to be more aware of the issue. However, the guideline should not be taken too seriously to the extent that it affects creativity which is the main objective of qualitative research, hence a balance between the two is needed. Bailey, C, White, C, and Pain R (1999) Evaluating qualitative research: dealing with the tension between science and creativity. Area. 31(2): 169-83. http://eprints.ncrm.ac.uk/2273/4/how_many_interviews.pdf Baxter, Jamie and Eyles, John (1997) Evaluating qualitative research in social geography: establishing rigour in interview analysis. Transactions . 22: 505-25. Golafshani, N. (2003). Understanding Reliability and Validity in Qualitative Research. The Qualitative Report, 8(4), 597-606. Retrieved from http://nsuworks.nova.edu/tqr/vol8/iss4/6 Long, T and Johnson, M. (2000). Rigour, reliability and validity in qualitative research Longhurst, R (2009) Interviews: In depth, semi-structured. In international Encyclopaedia of Geography McDowell, L. 1992. Valid games? A response to Erica Schoenberger. The Professional Geographer. 44(2), pp.212-15. McDowell, L. 2012. https://www.dawsonera.com/readonline/9781446206560 Research Methodology, 2016. http://research-methodology.net/research-philosophy/positivism/ Schoenberger, E. 1991. The corporate interview as a research method in economic geography. The Professional Geographer. 43(2), pp.180-9. Tracy, S.J.2010. Qualitative quality: Eight big-tent criteria for excellent qualitative research. Qualitative Inquiry. 16(10), pp.837-51. http://www.le.ac.uk/oerresources/psychology/psa/unit5/page_09.htm