Truth and honesty time. Until I started my Masters this year (in Public Health & Nutrition at University College Dublin), I would definitely say my skills in interpreting and critiquing research, particularly medical research, were lacking. Big time. Largely because it wasn’t taught much during my years at medical school. Now, this post isn’t about me dissing my undergraduate education – it was awesome, made me the doctor I am today, and through it I’ve found my passion in medicine. BUT. The little bit of teaching we did get on the basics of interpreting medical research were delivered as part of our fourth year module in general practice, and being honest, in my opinion was insufficient.
We are, in fairness, taught in medicine about the most evidence-based approaches to patient care, which is evolving constantly, due largely to the rapid pace at which technology advances in modern society. So while what I might have learnt for my medical finals over 3 years ago now was the most up to date at the time, that may not be the case now, depending on how the research in a given field has progressed since. For example, acute stroke management has undergone major changes over the last couple of decades, and the medications we use to manage type 2 diabetes mellitus have also been a big focus of research in recent years. Therefore, staying informed and up to date as a doctor seeing a wide range of patient presentations requires continuous self and senior/peer education. This is done formally and informally – during my intern and senior house officer years, I had structured weekly teaching sessions from various consultants, received teaching from and delivered teaching to my peers, and did my own reading too. But I can’t say I felt confident or in any way proficient in critiquing medical research – being able to identify strengths and limitations of different studies, or even being aware of what the different study designs are.
Fast-forward to Semester 1 of my Masters, and as I expected, it turns out reading, interpreting and critiquing research, and applying this knowledge to develop policies and strategies to optimise population health, is a foundation of public health. I have learnt SO much in the last 4 months alone, and when I shared my thoughts on Instagram on the topic, I got LOTS of feedback from you guys, expressing a similar lack of confidence in research interpretation. So I decided to summarise the basics I have learnt into 6 key points you can take from this article, and apply to the next article you read for work or college! A quick note before I dive into these points is that I’m predominantly focusing on health-related research, so that’s who these tips will be most applicable to. I always start an article by reading the abstract (that’s the summary bit usually on the first page!) before diving into each section separately – as, I’m do most of you, so apologies if that sounds obvious to say, but it might not be for many!
Disclaimer: I am no research expert – like I said above, I’m sharing what I have learnt as the basics to reading research, and of course the questions and tips below are not exhaustive! But I do hope they help those of you who are beginners like me but want to improve their skills and knowledge. And I would love to hear your own tips and thoughts on this topic too! So, let’s get into it. Here’s my five questions I now ask myself when reading an article:
- What is the research QUESTION?
- What KIND of study is it?
- WHO is being studied?
- HOW are they being studied?
- How VALID are the study results – both internally and externally?
- Bonus – WHY might the authors be reporting or carrying out the study (i.e. do they have any disclosure of interest mentioned, or sponsorship by a drug company for the study?)
What is the research QUESTION?
First of all, ask yourself – what question is this article trying to answer? It might be an investigation into a new treatment for a medical condition, or it might be a review of the existing treatments out there, maybe to see what has had the most success. It might be a description of the causes of a condition, or a summary overview of the condition with updates from a previous study (e.g. the Lancet Journal group publish reviews on conditions such as tuberculosis, and HIV infection, among many others, with updated statistics on numbers affected globally, deaths due to the condition, risk factors, distribution across developed and developing countries, etc). Figuring out what the question is is a great first step that will keep you focused throughout your interpretation, as ultimately you want to know how and if the study did indeed address and answer the question.
What KIND of Study is it?
Broadly speaking, studies will either serve to describe or analyse something. An analytical study will examine a group of people with, for example, a condition or characteristic of interest, and compare them to a ‘healthy’ (i.e. don’t have the characteristic or condition of interest) but otherwise similar group of people. Studies will call these groups ‘cases’ and ‘controls’ usually, the controls being the healthy group that are ideally the same as the cases EXCEPT for the given disease or characteristic. Within analytical studies, the time direction of the study determines what kind of study it it. Cohort studies follow the 2 groups FORWARD in time (couple of exceptions here beyond the scope of this article, but generally!), examining the 2 groups (one with risk factors under study, one without) to see which individuals develop disease and which don’t. Case-Control studies look BACKWARDS in time, comparing the 2 groups (cases who have a disease, controls who don’t) to see which individuals were exposed to certain potential risk factors. Cross-sectional studies are ‘snapshot’ studies, looking at risk factors and disease at the same time. The goal of analytical studies is to examine for ASSOCIATIONS between, for example, risk factors/exposures and diseases of interest. You may have heard of ‘randomised’ and ‘non-randomised’ controlled trials – these are also analytical studies, but ones where the exposures are assigned to the study participants. For example, a group of researchers might want to investigate a new treatment for a disease compared to the current gold standard of care. They could give one group the new treatment and the second group the current one (either in a random or non-random way), and look at the effect of the treatment on the disease.
Descriptive studies don’t feature a ‘control’ group. Examples include case studies, case series, cross-sectional studies and ecological studies. Case studies and case series generally examine patient cases and aspects of a condition of interest, or it’s treatment, usually to report on some unusual feature. Ecological studies examine the health of populations. I won’t go into much detail here except to say that descriptive studies aim to do just that – describe, such as the natural history of a disease, it’s risk factor, it’s distribution across or within countries, and propose possible aetiologies for the condition. These studies CAN’T test for associations or contribute proof of causation, because they don’t feature case comparisons with controls.
WHO is being studied?
You need to look at the group under investigation in the article. Are they male, female, adults, children? What ethnicity are they? Where are they from? What characteristics do they have – for example, a study might collect information on certain lifestyle risk factors people have, such as being a smoker, or having high blood pressure, or being overweight or obese. Taking a critical view of who the cases (and if present, controls) are is important to allow us to examine how relevant the study findings and conclusions are to our own lives. For example, studies that report on heart disease risk factors and rates in Asian male populations aren’t fully applicable to, say Caucasian female populations, because ethnicity, gender and therefore genetics, as well as environmental factors, affecting the two groups are totally different.
HOW are they being studied?
This point brings us back to how we answered the question ‘What KIND of study is it?’ , and applies more so to analytical studies (remember, those are the studies with a control/comparison group). We need to consider when reading published studies, how the group went about trying to answer their research question. For example, let’s say a group of researchers want to examine for an association between smoking cigarettes and lung cancer. Of course, this is a well-established phenomenon, but it’s also a good example to illustrate my point. So, the group would do an analytical study – but which one? They would need to decide whether they want to follow a group of smokers forward in time to see if they have a greater rate of lung cancer than non-smokers (i.e. do a cohort study), or look backwards into the history of a group of patients with lung cancer and a group without, to see if the group with disease has a greater proportion of smokers than the group without (i.e. a case-control study). Similarly, with randomised and non-randomised controlled trials where groups are assigned exposures and followed up to ascertain outcomes, it’s important to look at HOW these exposures are assigned by the researchers, and how they determine outcomes.
How VALID are the study results – both internally and externally?
Validity refers to accuracy, and there’s two main types in research. Internal validity refers to whether the study measured what it set out to, and external validity refers to how generalisable the study findings and conclusions are to it’s target population. Basically, you should ask yourself after reading an article – did this study answer it’s own research question? What were the limitations the group identified to their work? These are usually described in the ‘Discussion’ section of a paper. If a study is flawed in it’s accuracy, it very much limits how much we can or should take from it in terms of extrapolating it’s findings to our own health. And FYI – EVERY study has limitations.
If anyone has read ‘Bad Pharma’ by Dr. Ben Goldacre, you’ll know all about industry-funding and hiding of trials in medicine. I won’t get into that now, but Bad Pharma is an awesome read, and just one of Dr. Goldacre’s books worth checking out. At the end of the paper you’re reading, have a look and see if the authors have mentioned any sponsorship or disclosure of interest. There has been a huge move towards greater transparnecy in reporting of various types of research over the last few years, but it’s still far from perfect.
My final point to finish touches on a quick reference guide for how the strength of the evidence from different study design are ranked, because not all research designs are equal in this regard. Each study design has its own strengths and limitations of course, but there is a hierarchy which is very important for us to be aware of. Here’s a little diagram below to show you:
A Final Note On Causation – Which Is NOT Correlation
Correlation and causation are very frequently confused by people particularly when newspapers or magazines get a hold of a study that they can create a catchy headline from – and ESPECIALLY when we look at nutritional research.
This diagram is always shown as the classic example – it illustrates a correlation between ice cream sales, and shark attacks during the summer months. Correlation simply indicates that there is a relationship between two factors – in this case, ice-cream sales and shark attacks. But that doesn’t mean that one causes the other. Both are more common during the summer months (as people eat more ice cream and swim more in hot weather), which is why rates of both go up together in this graph. Correlation do not mean causation.
Okay gang! That’s the end of a whistle-stop tour for now. There is of course, a LOT more that I could discuss but I’ve found these five questions I’ve laid out above a great place to start. I would love to hear your thoughts – leave a comment, send me an email, or pop a DM on Instagram/Twitter/Facebook – you know where to find me, @theirishbalance on Instagram/Twitter/Facebook!
Ciara 🙂 x