This week, I reviewed a dissertation that did not answer its main research question. Unfortunately, the student had described its design, delimitations, and sampling method incorrectly which made the conclusions incorrect. Luckily it was fixable.
Let me tell you about the study, with its details all changed to respect the student’s confidentiality and anonymity. Its lessons may help you avoid the same pitfalls in your own dissertation.
The study I speak of was designed to assess the clarity of the written messages that South African public hospitals provide to their patients suffering from a particular disease. Let’s say the disease considered in the study was diabetes. The written messages studied were brochures, pamphlets, and posters.
The student worked in the Nelson Mandela Bay Municipality and had selected this municipality as the target area. A team of medical specialists at the hospitals in the area had worked with marketers to develop the communication material – brochures, pamphlets, and posters – on the presentation and management of the disease.
The student approached all the hospitals that serve the population of the municipality for copies of the communication material. Thereafter, the student employed two skilled assistants to rate the clarity of the messages on criteria derived from relevant theory. The inter-rater reliability of the ratings was checked.
Now recall that the main objective of the study was to assess the clarity of the written messages that South African public hospitals provide to their patients with diabetes. The main research question was: How clear are the written messages that South African public hospitals provide to their patients with diabetes?
In the dissertation, the research design was described as a quantitative descriptive research design. The sampling was described as a convenience sample of South African public hospitals. The communication messages were described as having been randomly selected. The results were generalised to the messages on diabetes that South African public hospitals communicate to their patients.
Let’s turn for a moment to some research methodology theory so that we can assess the student’s description.
Correctly used, a convenience sample refers to a sample selected by the researcher out of convenience. As a convenience sample does not involve any form of random selection, it is a nonprobability sample and you cannot apply statistical theory to it. Thus, you cannot calculate standard errors and confidence intervals, nor generalise its results to your target population. A convenience sample always presents a red flag whenever the study objective involves generalization (or at any time, really).
In this dissertation, the student was generalising the findings to the written messages on diabetes in all South African hospitals. This was clearly incorrect.
So, what was really going on in this study?
The research methodology was quantitative, and the research strategy/design was a nonexperimental observational case study of the diabetes messages communicated to patients in the Nelson Mandela Bay Municipality. The research method was content analysis. The sampling of hospitals in the Nelson Mandela Bay Municipality should have been described as a delimitation of the study as the researcher deliberately ringfenced the research to this area. Furthermore, all hospitals in the municipality had been approached and had participated in the study, so a census of the hospitals was obtained rather than a sample. And, as the hospitals had been asked for copies of all their written messages and had been forthcoming, the researcher could have assumed that they had obtained all the written messages on diabetes of these hospitals.
So, could the dissertation fulfil the main research objective of assessing the clarity of the written messages that South African public hospitals provide to their patients with diabetes? Clearly not.
Any conclusions that the researcher could draw given the research approach should have been limited to the hospitals in the Nelson Mandela Bay Municipality as the study was delimited to these hospitals and were specific to these hospitals. This, in itself, was worthwhile research and the student fixed the dissertation by acknowledging this. The bottom line here is that your research methodology, design or strategy, assumptions, delimitations, and sampling methods all need to be considered for you to address your research questions and hence draw the correct conclusions.
