Diving Deep: The Importance of In-Depth Statistical Analysis in Medical Research
Article Information
Marcos Sforza^{1*}, Renee Okhiria^{2}, Terrell Okhiria^{3}
^{1}Dolan Park Hospital, Stoney Lane, B60 1LY, England, UK
^{2}University of Manchester, Manchester, UK
^{3}Imperial College London, London, UK
^{*}Corresponding Author: Marcos Sforza, Dolan Park Hospital, Stoney Lane, B60 1LY, England, UK
Received: 22 August 2020; Accepted: 02 September 2020; Published: 09 September 2020
Citation: Marcos Sforza, Renee Okhiria, Terrell Okhiria. Diving Deep: The Importance of In-Depth Statistical Analysis in Medical Research. Journal of Surgery and Research 3 (2020): 301-308.
View / Download Pdf Share at FacebookAbstract
Background: Motiva silicone breast implants were recently discovered to perform much more effectively than competing silicone implants, with a reoperation rate less than 1%. However, a vast majority of the underlying methods behind this analysis was removed from the original published paper.
Objective: This paper aims to reintroduce the detailed methods used by the authors to statistically justify this low reoperation rate, whilst further supporting the importance of including this information in the available medical literature.
Methods: This paper demonstrated the detailed methods omitted from the published paper, estimating true risk ratio using confidence intervals to make accurate inferences regarding the reoperation rates of the Motiva silicone breast implants, based on 5813 consecutive cases of breast augmentation.
Results: The true risk rate for Motiva silicone breast implants is less than 1.02%, with an even lower reoperation rate possible if sources of endogeneity were to be omitted from the study.
Conclusion: Overall, the inclusion of these detailed statistical methods is needed for readers to gain a full understanding of the low reoperation rates related to Motiva Implants.
Keywords
Breast implants; Silicone implants
Article Details
Introduction
It has been three years since the publication of our popular paper entitled “Preliminary 3-Year Evaluation of Experience with SilkSurface and VelvetSurface Motiva Silicone Breast Implants” [1]. With over 27,000 downloads, the paper was the first to show, through statistical analysis, a highly impressive, yet unprecedented revision rate of less than 1% through the use of silicone breast implants in breast implant surgery.
Statistical analyses are an imperative component of research and its consumption [2]. They provide a gateway for the reader to evaluate how an author came to their conclusion, allowing the author to justify or explain their hypothesis.
In-depth statistical analysis is of even greater significance in a study like this, using more rigorous methods of statistics to test the hypothesis. In the context of our published paper, this type of analysis increases the reliability of conclusions drawn through the demonstration of the results being easily reproduced, giving clinicians and other readers exact information on how the two types of implants impacted the patients involved [3]. This limits the level of bias and error that may otherwise have been present, giving the conclusion a stronger foundation. Clinicians can draw more informed and accurate inferences from the analysis, facilitating greater patient outcomes.
However, for editorial reasons, the detailed explanations of the underlying methods behind these in-depth statistical analyses used in our paper had to be omitted.
Nevertheless, the authors would like to present the methods used on their paper, further clarifying the reasonings behind the proposed statistical methods that had only been accessible to the peer reviewers of the original publication.
The Risk Ratio and Confidence Intervals
In our published paper, the Kaplan-Meier survival analysis was the sole statistical analysis used to interpret data. The survival analysis focuses on the expected time until a complication occurs - providing an estimate regarding the likelihood of suffering from a complication at a given period of time [4]. However, there are several limitations to purely using this analytical method. Clinicians may want to know more, including the size of any potential differences between the risks of two groups. The survival analysis also disregards confounding variables, such as a patient suffering from any other complications prior to the studied complication [4].
Acknowledging the constraints of using only the Kaplan-Meier analysis, our original submission also included a supplementary method to deepen the paper’s statistical analysis - the risk ratio.
The risk ratio, unlike the Kaplan-Meier analysis, looks into the complication rate over the entire period of the study - granting comparison between different groups (i.e., VelvetSurface and SilkSurface patients), or formulation of an alternative hypothesis regarding the accuracy of the risk rate.
The risk ratio changes from sample to sample, as it is influenced by other random variables between samples. Hence, we have to estimate the true risk ratio in order to make accurate inferences regarding complication rate for the given population. An acceptable range of risk ratios is formed by using the highest risk ratio consistent with a given sample, and by estimating the confidence interval for this risk ratio.
The confidence interval gives the possible risk ratios which are consistent with data. To put into practise, if we construct a 95% confidence interval in an experiment repeated 100 times, the true risk ratio would be contained within this interval in 95 cases - however in 5 cases it would not. This theory is further explained in Table 1.
Confidence Interval |
Cases where True Risk Ratio is contained in Confidence Interval |
Cases where True Risk Ratio is not contained in Confidence Interval |
0% |
0 |
100 |
5% |
5 |
95 |
25% |
25 |
75 |
50% |
50 |
50 |
75% |
75 |
25 |
95% |
95 |
5 |
100% |
100 |
0 |
Table 1: Table showing Confidence Intervals and their effect on the True Risk Ratio
Our original study showed a reoperation rate following Motiva Implants® (Establishment Labs, Alajuela, Costa Rica) of 0.76%, hence a risk rate of 0.76% was estimated for the intervention. However, since this is such a low probability, it was hypothesised that complications following the intervention could be due to chance. For example, it is possible that the sample group had healthier lifestyles than the actual population, resulting in a lower chance of developing postoperative complications.
As previously mentioned, the true risk rate must stay within the acceptable confidence interval if the study was to be repeated. Thus, our original submission explored the validity of the 0.76% risk rate - hypothesising that, as this rate could have been due the chance, the true risk rate was actually higher.
In our original study, we had a sample size of 5816 patients, and the risk rate was calculated as 0.76% due to only 44 patients undergoing reoperation. To investigate our hypothesis, we hypothetically repeated the study 100 times at two different risk rates, both higher than 0.76%. These risk rates were 1.5% and 1.075%.
According to our calculations, if the true risk rate was 1.5%, there would be a 0.0002% chance of observing exactly 44 or less complications in the same sample size. Therefore, there is a very low possibility that the true risk rate is 1.5%.
If the true risk rate was 1.075%, there would be a 1% chance of observing exactly 44 or less complications in the same sample size. Although this is greater than the possibility given at the risk rate of 1.5%, it still produces a low possibility that this is the true risk rate.
In our repeated study, the highest probability of the risk rate being accurate was 1% with a 1.075% risk rate. With this knowledge, we witnessed that the lower the true risk rate, the greater the likelihood of it being observed in our repeated studies. Therefore, we set our acceptance probability to 2.5% - we decided that we would not reject the true risk rate if the probability of the risk being observed in the repeated study was 2.5%
With 2.5% chance of observing exactly 44 or less complications in the same sample size, the true risk rate was calculated as 1.02%. This can be seen in Table 2, alongside the other theoretical risk rates.
Theoretical Risk Rate |
Number of patients experiencing complications with theoretical risk rate (n = 5813) |
Probability of our risk rate (0.76%) applying to the theoretical rate |
Cases with complications in 0.76% or less (n = 100) |
Cases with complications over 0.76% (n = 100) |
1.5% |
87 |
0.0002% |
0.0002 |
99.9998 |
1.075% |
63 |
1% |
1 |
99 |
1.02% |
59 |
2.5% |
3 |
97 |
Table 2: Theoretical Risk Rates and their contingency with our Observed Risk Rates
Previous studies have concluded a risk rate of 10%, however our findings of only 0.76% or less of the sample size experiencing a complication would be virtually impossible in these cases if repeated. Thus, we confidently believe that our study has a much lower true risk rate than 10%.
Although it is unlikely that the true risk rate is 1.02%, it is plausible if we accept a 2.5% probability of our study’s observed risk rate applying in each case as a threshold. However, as seen in the table, as we increase our risk rate from 1.02%, the probability decreases in likelihood. Therefore, we also confidently believe that the risk rate for Motiva implants® have a lower risk than 1% - independent of the sample and analysis.
Nonetheless, the complications reported in our original study were predicted on random chance. The Motiva Implants® should, in actuality, result in an even lower number of complications if sources of endogeneity were to be omitted - thus confirming a very low risk rate for this intervention.
The Role of Randomness in Statistical Analysis
To better understand the role of randomness in statistical analysis, the analogy of a fair coin can be used.
When tossing a fair coin, there is a 50 % chance of getting ahead, and 50% chance of getting a tail. If given to an analyser, blind to the coin is fairness, to determine the probability of getting a head, the person might have received 3 heads from 10 coin tosses.
In this situation, the analyser could conclude that the coin is unfair - with a 30% chance of tossing a head. However,it would be more accurate to say that, consistent with this situation, we cannot reject the hypothesis that the probability of tossing a head is 30%. To conclude, the probability of tossing a head might not be 30%, but this is a valid hypothesis given our findings.
Getting 3 heads from 10-coin tosses is also consistent with other probabilities of tossing a head, as seen in Table 3 below.
Table 3: Assumed probability of tossing a head, and probability of tossing a head 3 times out of 10-coin tosses.
As shown in the table, the highest probability of tossing 3 heads in a 10-coin toss is when assuming the probability of tossing a head is 30%. However, if the assumed probability of tossing a head was 20%, 40% or 50% it would also be probable to toss 3 heads in a 10-coin toss.
To visualise what this would look like if the 10-coin toss was repeated 20 times, Table 4 has been produced.
From Table 4, we can say that tossing 3 heads out of 10-coin tosses is consistent with the assumed probability of tossing a head of 10% to 60%. However, we might suggest rejection of the hypothesis of the assumed probability of head being 70%, as a very small number of cases would have resulted in 3 heads out of 10-coin tosses in this case - and therefore it is a rare event.
Table 4: Assumed probability of tossing a head, probability of tossing a head 3 times out of 10-coin tosses and number of cases where a head is tossed 3 times out of 10-coin tosses in 20 repeats
Instead of tossing the coin 10 times, we can increase the toss to 100 times. If we toss a fair coin 100 times, it is much more likely that we would have around 50 heads. For the next situation, we will assume that 52 heads are thrown from 100-coin tosses. Yet again, this is outcome is consistent with various assumed probabilities of throwing a head, as seen in Table 5.
Table 5: Assumed probability of tossing a head, and probability of tossing a head 52 times out of 100-coin tosses
The probability of throwing 52 heads from 100-coin tosses is now consistent with a narrower range of probabilities: 46% to 54%. Any assumed probability of tossing a head outside of this range is likely to produce this outcome less than 4% of the time - and therefore can be seen as a rare event.
If we further increased the number of coin tosses, the range of consistent assumed probabilities will become narrower and closer to 50% - the true probability of throwing a head on a fair dice.
In our original study, we concluded that the reoperation rate was 0.76%. We can be sure that if another 5813 patients were examined, the reoperation rate would be something else due to a great degree of randomness. However, the question stands - what other risk rates are consistent with our sample? For example, if we assumed the risk rate was truly 1.5% and repeated our study more than 9 million times, we would only have one case where there are exactly 44 reoperations out of 5813 patients. From this, we can conclude that it is unlikely for the reoperation rate to be 1.5%, and it is likely that the 0.76% risk rate is due to chance. On the other hand, if the true risk rate were 1.02%, we would observe 44 or less reoperations in 5 cases out of 100 repeats. Thus, while it is still unlikely, we can accept 5% probability as a threshold and the risk rate would still be plausible. However, any probability greater than 5% would be unlikely - thus we are confident that the reoperation rate with Motiva Implants® has risk rate of lower than 1%.
Discussion
The in-depth statistical analysis removed from our original study, providing attempts to validate whether the observed risk rate was close to the true risk rate, is of great relevance. This trial and error approach to determine the true risk rate helps confirm that, if reproduced, our study would still see a low proportion of complications in patients treated with the Motiva Implants®, compared to other types of implants. This is especially important when comparing the two types of Motiva Implants® studied; the SilkSurface® and VelvetSurface®, allowing us to narrow down which type of implant is safer to patients.
This part of our published paper was ultimately removed for a number of reasons, one being that it is not easy information to digest for the average reader. In order to understand our in-depth analysis, the reader must have prior knowledge and expertise relating to certain statistical theories, such as that of confidence intervals. Without this, the reader may be confused and put off by our analysis.
The analysis involves the use of many hypothetical situations, using theoretical risk rates to predict the true risk rate; this adds to the length and “uncertainty” of the analysis. Our original paper was published in a journal, which contains a multitude of other unrelated papers. This means that our paper would need to be kept concise, to a certain extent, and including this in-depth analysis would greatly increase the length of the paper, preventing this conciseness. Information may have also been repeated between the in-depth analysis and the rest of our published paper, which was ultimately edited out.
In statistics, causation is often unintentionally eluded to by a high correlation between two variables - in this case, the use of Motiva Implants® correlates with a lower reoperation rate [6,7]. However, it is important to remember that causation is not always the case, due to the presence of many confounding factors [5], such as age and lifestyle, which could have impacted our original paper’s findings.
Conclusion
There is a case to be made for this level of in-depth analysis to be accessible to all in the medical research field through open access journals. In doing so, more doctors and surgeons will be exposed to the actual thinking processes behind the conclusions drawn from the study. This will allow for more mindful digestion of the research, rather than readers passively reading and simply accepting the conclusions for what they are.
This act of mindful absorption, in turn, will only enable surgeons with the knowledge needed to perform more effective implant procedures, with lower rates of complications, hopefully through the use of new bioengineered breast implants.
References
- Sforza M, Zaccheddu R, Alleruzzo A, et al.Preliminary 3-year evaluation of experience with SilkSurface and VelvetSurface Motiva silicone breast implants: a single-center experience with 5813 consecutive breast augmentation cases.Aesthet Surg J 38 (2018): S62-S73.
- Campbell M. The importance of Statistics. African Journal of Midwifery and Women's Health 8 (2014): 113-114.
- Siddharth K. Statistical Reliability. Retrieved Feb 07, 2020 from Explorable.com: https://explorable.com/statistical-reliability
- Flynn R. Survival analysis. Journal of Clinical Nursing 21 (2012): 2789-2797.
- Aldrich J. Correlations Genuine and Spurious in Pearson and Yule. Statistical Science 10 (1995): 364–376.
- Sforza M, Hammond DC, Botti G, et al.Expert consensus on the use of a new bioengineered, cell-friendly, smooth surface breast implant.Aesthet Surg J 39 (2019): S95-S102.
- Sforza M, Spear S, Hammond D.The 21st century silicone breast implant.J Surg Open Access 2 (2016).