Three of the most prominent criteria for the evaluation of business and management research are reliability, replication, and validity
- Reliability: reliability is concerned with the question of whether the results of a study are repeatable. Reliability is particularly at issue in connection with quantitative research. The quantitative researcher is likely to be concerned with the question of whether a measure is stable or not. Example: IQ test, which are designed as measures of intelligence were found to fluctuate, so that people’s IQ scores were often wildly different when administered on two or more occasions. We would be concerned about it as a measure. We would consider it an unrealistic measure – we could not have faith in its consistency.
- Replication: the idea of reliability is very close to another criterion of research replication and more especially to replicate the findings of others. In order for replication to take place, a study must be capable of replication – it must be replicable. This is a very obvious point: if a researcher does not spell out his or her procedures in great detail, replication is impossible. Similarly in order for us to access reliability of a measure of a concept, the procedures that constitute that, measure must be replicable by someone else. Ironically, replication in business research is not common. In fact, it is probably true to say it is quite rare.
- Validity: the most important criterion of research is validity. Validity is concerned with the integrity of the conclusion that is generated from apiece of research. The main types of types of validity that are typically distinguished.
(A). Measurable validity – this criterion applies primarily to quantitative research and to the search for measures of social scientific concepts. Measurement validity is also often referred to as construct validity. Essentially, it is to do with question of whether or not a measure that is derived of a concept really does reflect the concept that it is supposed to be denoting. Does the IQ test really measure variations in intelligence in order to test the hypotheses: ‘magnitude of consequences ’and‘ issue framing and two context-related concepts that also need to be measured: ‘perceived social consensuses’ and ‘competitive context’ The question then is; do the measures really represent the concepts they are supposed to be tapping? If they do not, the study’s findings will be questionable. It should be appreciated that measurement validity is related to reliability, if a measure of a concept is unstable in that fluctuates and hence is unreliable; it simply cannot be providing a valid measure of the concept in question. In other words, the assessment of measurement validity presupposes that a measure is reliable.
(B) Internal validity – this form of validity relates mainly to the issue of causality. Internal validity is concerned with the question of whether a conclusion that incorporates a casual relationship between two or more variable holds water. If we suggest that x causes y, can we be sure that it is x that is responsible for variation in y and not something else that is producing an apparent casual relationship.
(C) External validity – this is issue concerned with the question of whether the results of a study can be generalized beyond the specific research context. It is in this context that the issue of how people or organizations are selected to participate in research becomes crucial. External validity is one of the main reasons why quantitative researchers are so keen to generate representative samples.
(D) Ecological validity – this criterion is concerned with the question of whether or not social scientific findings are applicable to people’s every day, natural social settings. At times business research may produce findings that may be technically valid but might have little to do with what happens in people’s life. If business researches are ecologically invalid, they are in sense artefacts of the social scientist’s arsenal data collection and analysis tools.