Rank the evidence generated from the following designs from lowest to highest

8.1.1.3 Wilcoxon rank-sum test and Wilcoxon signed-rank test

Wilcoxon rank-sum test and Wilcoxon signed-rank test were proposed by Frank Wilcoxon in a single paper.599 Wilcoxon rank-sum test is used to compare two independent samples, while Wilcoxon signed-rank test is used to compare two related samples, matched samples, or to conduct a paired difference test of repeated measurements on a single sample to assess whether their population mean ranks differ. They are nonparametric alternatives to the unpaired and paired Student's t-tests (also known as “t-test for matched pairs” or “t-test for dependent samples”), respectively. The two nonparametric tests do not assume that the samples are normally distributed. The Wilcoxon unpaired two-sample test statistic is a technique equivalent to the statistic proposed by the German Gustav Deuchler in 1914. However, Deuchler incorrectly calculated the variance.670 Wilcoxon formulated a test of significance with a point null hypothesis against its complementary alternative in his 1945 paper. However, in this paper the null hypothesis was only given for the equal sample size case and only a few points were tabulated (though Wilcoxon gave larger tables in a later paper). A thorough analysis of the statistic was provided by Henry Mann and Donald Ransom Whitney in their 1947 paper.598 This is the reason that Wilcoxon rank-sum test is also called Wilcoxon-Mann-Whitney test and Mann-Whitney U test is equivalent to Wilcoxon rank-sum test. Wilcoxon rank-sum test and Wilcoxon signed-rank test were used to compare the median differences in alpha-diversity measures, proportion of core genera, and abundance of specific genera for categorical variables and variables in the case of matched samples, respectively, in the microbiome study by Falony et al.671 Other examples of using Mann-Whitney U test or Wilcoxon rank-sum test in microbiome studies are provided in the reports.272,533,606,672–675 For within-group comparison of alpha diversity generally Wilcoxon signed-rank test can be applied to analyze each pairwise within-group comparison of gut microbiota diversity (gene richness)676 and relative abundance of microbial phyla.672 Other examples of Wilcoxon signed-rank test use in microbiome studies can be found in these papers.415,606,677,678

Mann-Whitney U test and Wilcoxon rank-sum test are also often used to identify association between taxa or OTUs and covariates. However, these approaches conduct the association analysis based on the ranks of observed relative abundances, resulting in information loss and high false-negative rates.

3.6.5 Multiple Comparisons with Repeated Measures

The case described earlier for the paired t-test or the Wilcoxon signed-rank test is the simplest form of a repeated measures design. The use of multiple comparisons with repeated measures is very common, especially in studies evaluating the time course of an effect. For example, if the earlier described repeated measures study in which a baseline is measured, and then a measurement is taken after compound treatment in the same animal were extended to include measurements taken every 15 min after dosing for 2 h, each animal will be sampled once for baseline, and 8 times for treatment resulting in 8 comparisons. However, these 9 groups are not independent as they are all obtained from the same set of animals after the same treatment and would be expected to share some variance. This type of study should be analyzed using a one way repeated measures ANOVA (Fig. 3.7).

The repeated measures ANOVA is an extension of the ANOVA that accounts for the shared variance in the groups. The assumptions of the one way repeated measures ANOVA are the same as the ANOVA with the addition of an assumption of sphericity. This is an extension of the equal variance assumption that states that the variance of the differences between all combinations of related groups is equal. Most software packages that provide repeated measured ANOVA will perform tests of sphericity, and these tests should be used as the repeated measures ANOVA is very sensitive to violations of this assumption. Choice of post hoc testing is the same as that discussed for multiple comparisons between independent groups by ANOVA (Fig. 3.7).

For situations where the normality assumption is violated in a repeated measures design involving three or more groups, the Friedman test (Friedman, 1937), a rank nonparametric version of the analysis of variance can be used (Fig. 3.8). The Friedman test is an extension of the Wilcoxon signed-rank test and carries all of the assumptions of that test described earlier with the additional assumption of sphericity. The null hypothesis for the Friedman test states that all groups have the same median value and the p-value is interpreted as the probability that differences in the median can be attributed to chance alone. As with the Wilcoxon signed-rank test, Dunn’s test can be used as a post hoc analysis to determine which groups are significantly different (Fig. 3.8).

Results

The data collected was subjected to a battery of statistical tests including repeated-measures ANOVA, Friedman's test, Bonferroni Post-hoc test, Wilcoxon Signed Ranks test and ‘T’ test. The main conclusion was that there was no significant difference between the different displays and thus the replacement of the CRT with flat screens would not result in any significant degradation in signaller performance. The key results are presented in the bar charts (Figs. 13 and 14).

The targets missed results were tested statistically. Although the bar chart shows a difference between the error scores, this was not statistically significant. The number of targets missed, in general, was not affected by display type.

The performance with the mono TFT is marginally worse than the existing condition, mono CRT, and the full colour condition is marginally better. Statistical tests showed the differences to be significant. However, in real terms a reaction time difference of a few tens of milliseconds would not be a significant differentiator: the nature of the operational task means that operators are required to take the time necessary to satisfy themselves that the crossing is clear – thus the operator is not under time pressure.

In general, participants felt that the colour clips presented on the TFT monitor provided better definition for the images.

2.5 Statistical analysis

We performed statistical analyses using SPSS 11.5J for Windows (SPSS Japan Inc., Tokyo). Since we could not be sure as to the exact nature of the population distribution under study, we adopted non-parametric statistical tests. As a result, we used Wilcoxon's signed ranks test to compare the uMAP area, CMAP area, and the MUNEs on the affected and unaffected sides of each of the patients with cerebral infarction. In order to compare the MUNE ratios of patients with cerebral infarction and hand weakness to the ratios of the patients having normal hand strength we used the Mann–Whitney U test. Spearman's correlation test was used to find a statistically significant correlation between the MUNE ratio and the JSS-UM.

Results Using the Personal Gold Standard

As was discussed previously, the personal gold standard was set by taking a radiologist's recorded vessel size on the uncompressed image to be the correct measurement for judging her or his performance on the compressed images. Using a personal gold standard in general accounts for a measurement bias attributed to an individual radiologist, thereby providing a more consistent result among the measurements of each judge at the different compression levels. The personal gold standard thus eliminates the interobserver variability present with the independent gold standard. However, it does not allow us to compare performance at compressed bit rates to performance at the original bit rates, since the standard is determined from the original bit rates, thereby giving the original images zero error. As before, we first consider visual trends and then quantify differences between levels by statistical tests.

Figure 7 shows average pme vs mean bit rate for the five compressed levels for each judge separately and for the judges pooled, whereas Fig. 8 is a display of the actual pme vs. actual achieved bit rate for all the data points. The data for the judges pooled are the measurements from all judges, images, levels, and vessels, with each judge's measurements compared to her or his personal gold standard. In each case, quadratic splines with a single knot at 1.0 bpp were fit to the data. Figs 9 and 10 are the corresponding figures for the apme. As expected, with the personal gold standard the pme and the apme are less than those obtained with the independent gold standard. The graphs indicate that whereas both judges 2 and 3 overmeasured at all bit rates with respect to the independent gold standard, only judge 3 overmeasured at the compressed bit rates with respect to the personal gold standard.

The t-test results indicate that levels 1 (0.36 bpp) and 4 (1.14 bpp) have significantly different pme associated with them than does the personal gold standard. The results of the Wilcoxon signed rank test on percent measurement error using the personal gold standard are similar to those obtained with the independent gold standard. In particular, only level 1 at 0.36 bpp differed significantly from the originals. Furthermore, levels 1, 3, and 4 were significantly different from level 5.

Since the t-test indicates that some results are marginally significant when the Wilcoxon signed rank test indicates the results are not significant, a Bonferroni simultaneous test (union bound) was constructed. This technique uses the significance level of two different tests to obtain a significance level that is simultaneously applicable for both. For example, in order to obtain a simultaneous significance level of α% with two tests, we could have the significance of each test be at (α/2)%. With the simultaneous test, the pme at level 4 (1.14 bpp) is not significantly different from the uncompressed level. As such, the simultaneous test indicates that only level 1 (0.36 bpp) has significantly different pme from the uncompressed level. This agrees with the corresponding result using the independent gold standard. Thus, pme at compression levels down to 0.55 bpp does not seem to differ significantly from the pme at the 9.0 bpp original.

In summary, with both the independent and personal gold standards, the t-test and the Wilcoxon signed rank test indicate that pme at compression levels down to 0.55 bpp did not differ significantly from the pme at the 9.0 bpp original. This was shown to be true for the independent gold standard by a direct application of the tests. For the personal gold standard, this was resolved by using the Bonferroni test for simultaneous validity of multiple analyses. The status of measurement accuracy at 0.36 bpp remains unclear, with the t-test concluding no difference and the Wilcoxon indicating significant difference in pme from the original with the independent gold standard, and both tests indicating significant difference in pme from the original with the personal gold standard. Since the model for the t-test is fitted only fairly to moderately well by the data, we lean towards the more conservative conclusion that lossy compression by our vector quantization compression method is not a cause of significant measurement error at bit rates ranging from 9.0 bpp down to 0.55 bpp, but it does introduce error at 0.36 bpp.

A radiologist's subjective perception of quality changes more rapidly and drastically with decreasing bit rate than does the actual measurement error. Radiologists evidently believe that the usefulness of images for measurement tasks degrades rapidly with decreasing bit rate. However, their actual measurement performance on the images was shown by both the t-test and Wilcoxon signed rank test (or the Bonferroni simultaneous test to resolve differences between the two) to remain consistently high down to 0.55 bpp. Thus, the radiologist's opinion of an image's diagnostic utility seems not to coincide with its utility for the clinical purpose for which the image was taken. The radiologist's subjective opinion of an image's usefulness for diagnosis should not be used as the sole predictor of actual usefulness.

Example Posed: Bias in Early Sampling of Prostate Biopsy Patients

We ask if the PSA levels of the first 20 patients differ significantly from 8.96 (the average of the remaining 281). When the sample is large, it exceeds the tabulated probabilities, but may be approximated by the normal distribution, with μ and σ calculated by simple formulas.

Method: The Normal Approximation to the Signed-Rank Test

The normal approximation to the Wilcoxon signed-rank test tests the hypothesis that the distribution of differences has a median of 0. (The median and mean are the same in the normal distribution.) It may test (1) a set of observations deviating from a hypothesized common value or (2) pairs of observations on the same individuals, such as before-and-after data. The p-value will not be identical with the exact method, but only rarely will this difference change the outcome decision. The steps in performing the test by hand are as follows:

Calculate the differences between pairs or from a hypothesized central value

Rank the magnitudes (i.e. the differences without signs)

Reattach the signs to the ranks

Add up the positive and negative ranks

Denote by T the unsigned value of the smaller; n is sample size (number of ranks)

Calculate μ= n(n + 1)/4, σ2= (2n + 1)μ/6, and then z= (T − μ)/σ

Obtain the p-value (as if it were α) from Table I.

Matched Pairs and Measures of Agreement

Usually measurements for a study are independent of one another. For example, we may wish to compare measurements made in two groups of subjects where we know that the groups are composed of separate individuals that bear no relationship to one another. In order to reduce variation or to control for a given factor, we may create pairs of individuals matched for a common characteristic. Alternatively, the two groups may not be independent but have an individual level relationship, such as a group of subjects and a group of their siblings. The subjects and their sibling represent matched pairs. When this is the case, we must use statistical testing that takes into account the fact that the two groups are not independent. If the independent variable is categorical, we would use a McNemar chi-square test. If the independent variable is ordinal, we would use an appropriate nonparametric type of test, such as Wilcoxon signed rank test. If the independent variable is continuous, we would use a paired t test. Each of these tests would relate to a different and specific type of probability distribution.

Sometimes, we will make repeat measurements in the same subject but using different methodology. By their nature, it is not surprising that the measurements will correlate, since they are measuring the same thing. What we are actually interested in is the degree to which the measurements agree, or agree with a criterion standard. Agreement between two binary variables can be expressed in one of two ways, either through the raw agreement or through the chance corrected agreement. The raw agreement is merely the number of times two measures agree divided by the total number of measures. By chance, two binary variables will agree approximately half of the time. Based on this, raw agreement is of limited interest. Cohen’s kappa or chance-corrected agreement is the degree of agreement between variables beyond that expected by chance alone. When continuous variables are of interest, agreement is assessed and depicted using Bland-Altman plots. A Bland-Altman plot plots the difference between two measurements in a pair on the y-axis versus the mean of those two measurements on the x-axis. If the agreement were perfect, all of the points would be at a difference of zero regardless of the value of the measurement. The plot can show the degree and limits of agreement, but also any patterns. Systematic bias can be noted, as well as changes in the magnitude of agreement as the average values get larger or smaller. A paired t test can be used to determine if any systematic differences exist between pairs of measures.

Nonparametric Tests

Nonparametric tests are used in detecting population differences when certain assumptions are not satisfied.

10.3 Statistical Procedures

The experimental study designs involve an intervention or manipulation by the investigator which is expected to influence the outcome and its impact is measured in terms of before–after effects. The experimental study designs are categorized into two types: randomized and nonrandomized trials (Bradshaw, 2016). Randomization is the random method of allocation of intervention group and control group (after they are selected and they provide informed written consent to participate in the study) by an independent biostatistician not related to the designated research project. This is done to minimize the selection bias as well as allocation bias. The randomization process is conducted by using a lottery method or random number table or electronically generated random numbers (Sagarin et al., 2014). The randomized controlled trial (RCT) always contain a control group which is adequately matched with the intervention group through at least one common parameter which can be either a sociodemographic correlate or an exposure or a disease or any other outcome. The level of evidence generated by a RCT is of the highest quality. Hence, evidence generated from best quality RCTs are included in systematic reviews and meta-analysis for evidence-based practice. On the other hand, a nonrandomized trial (NRT) does not contain any control group. Although the nonrandomized controlled trial (NRCT) always contains a control group, it is not appropriately matched with the intervention group. The levels of evidence generated by NRTs or NRCTs are of poor quality. Hence, evidence generated from them are never included in systematic reviews and meta-analysis for evidence-based practice (Borgnakke et al., 2014). The major experimental study designs are presented in Table 10.1.

Table 10.1. Experimental Study Designs

The experimental studies are conducted in four phases based on the size of the study population or sample size and quality of evidence that it is expected to be generated (Busk and Marascuilo, 2015). The Phase I trial is the beginning of the pharmaceutical product development process. It includes laboratory experiments and animal experiments on a very small amount of sample to study the pharmacokinetics, pharmacodynamics and toxicity levels.

The Phase II trial is also known as a clinical trial where a relatively smaller number of individuals (biostatistically approved) having a common exposure or disease are recruited to estimate the dose–response, efficacy, side-effects, and adverse reactions. This could be either pa reventive trial (vaccine for post-exposure prophylaxis or nutritional supplement or drug for chemoprophylaxis) or a therapeutic trial (drug for treatment). The vaccines for post-exposure prophylaxis, nutritional supplements, drugs for chemoprophylaxis, and drugs for treatment of any disease are never released in the market before they pass this phase (Wright, 2017).

The Phase III trial is also known as a field trial where a significant number of individuals having a common exposure are recruited from the field practice area to study the efficacy, side-effects, and adverse reactions. These are usually preventive trials involving a new vaccine for pre-exposure prophylaxis or nutritional supplement or drug for chemoprophylaxis. The new vaccines for pre-exposure prophylaxis, nutritional supplements, and drugs for chemoprophylaxis are never released in the market before they pass this phase. Some drugs for treatment of any disease might go through a Phase III trial before being released to the market. However, this is not mandatory (DeAngelis, 2017; Wright, 2017).

The Phase IV trial is also known as a community trial where a large community having a common exposure is involved to study the efficacy, side-effects, and adverse reactions. These trials are conducted on various brands of the same vaccine or nutritional supplement or drug which are marketed for a significant period and the respective manufacturing companies want to know which product is the best in terms of highest efficacy, least side-effects, and least adverse reactions for addressing the common, specific health issue (Wright, 2017). This is the reason why this phase is also known as post-marketing trial. As the Phase IV trials involve a large number of individuals who need to be followed up for a considerable period and monitored by many investigators, paramedical staff, and clinicians, this is the costliest study design among all experimental studies. Hence, the conduction of Phase IV trial is not mandatory under any international guidelines or regulations. It is left to the decree of renowned companies to conduct this trial depending on their financial capacity (Mahan, 2014).

Biostatistics serves a dominant role in quantification or inferential assessments in experimental study designs. Biostatistical procedures are usually more incorporated in areas where international regulators have issued guidelines like preclinical testing of cardiac liability, stability testing, and carcinogenicity. The Good Manufacturing Practice (GMP) has become an important regulatory authority in recent years. Hence, statistical inputs are regularly sought for high-risk screening, chemical and formulation development, as well as drug delivery and assay (Arias et al., 2017).

When repeated samples are drawn from the general population, the database follows a symmetric bell-shaped curve which is considered as the reference curve in biostatistics and is used for all comparison purposes. This is also known as the normal distribution or Gaussian curve (Barua et al., 2015a) Fig. 10.1.

Here, the data is symmetrically distributed on both sides with total area as 1; all central tendencies like mean, median, and mode coincide at the center; mean is zero and standard deviation (SD) is 1. Here, approximately 68% of the database is covered under ±1SD, 95% of the database is covered under ±2SD, and 99.7% of the database is covered under ±3SD. This is the ideal distribution of data from the general population and serves as a reference base. Since, ±1SD covers only 68% of the database where normal findings will appear as abnormal and ±3SD covers nearly whole of the database (99.7%) where abnormal findings will appear as normal, only ±2SD covering 95% of the database is considered as the reference point for normal distribution while following the middle path. Hence, the abnormal distribution of 5% of the remaining database (considered as chance) is used to determine the level of significance or probability or p-value which is set at <0.05. If the probability of difference between the two groups merely happening due to chance is less than 0.05 or 5%, this suggests that chance has very less weightage and there is a real difference. This is the interpretation of the p-value in biostatistics. The confidence interval (CI) is a measure of reliability of the study findings. It is hypothesized that if the same study is repeated 100 times by using the same methodology and conducted under similar environmental conditions, then 95 times the findings will fall within the estimated range. The CI is calculated for all statistical parameters like incidence rates, prevalence rates, and strengths of associations such as odds ratio (OR), relative risk (RR), and hazard ratio (HR) (Higgins, 2017).

In biostatistical procedures, parametric tests are applied when the database closely follows a normal distribution curve, while nonparametric tests are applied when the database becomes skewed and gets significantly deviated from the normal distribution curve. The biostatistical tools used to study the nature of distribution of a database are histogram, Q–Q plot, and Shapiro Wilk test. The basic algorithms of statistical tests of significance which are applicable for analytical and experimental studies are given in Table 10.2 for parametric tests and in Table 10.3 for nonparametric tests, respectively (Barua et al., 2015b).

Table 10.2. The Basic Algorithms of Parametric Tests of Significance Which are Applicable for Analytical and Experimental Studies

Table 10.3. The Basic Algorithms of Nonparametric Tests of Significance Which are Applicable for Analytical and Experimental Studies

The biostatistical model of univariate analysis assesses one independent variable at a time against the outcome or dependent variable. It provides a list of probable risk factors associated with the outcome (morbidity or mortality). Here, the strength of association is expressed in terms of unadjusted odds ratio or unadjusted relative risk. The multivariate analysis uses a complex biostatistical model which allows multiple independent variables to interact with each other at a time to produce the outcome (Hieke et al., 2016). It is used to study the independent effect of each variable over the outcome. It can eliminate the confounders from univariate analysis and identify the predictors of the outcome or dependable variable. The multiple logistic regression and Cox’s proportional hazards model are commonly used biostatistical instruments for multivariate analysis in analytical and experimental studies. Here, the strength of association is expressed in terms of adjusted odds ratio or adjusted relative risk or hazard ratio (Binder and Blettner, 2015).

Survival analysis is conducted in prospective studies which have a significant follow-up period for a health-related event to occur. The primary endpoint in many prospective cohort or experimental studies is time until an event occurs (e.g., death, remission). Survival function describes the proportion of individuals surviving until a given point of time or living beyond. Data are subject to censoring when a study ends before the event occurs (Lacny et al., 2017). The Kaplan–Meier estimate of survival function is conducted by using the log-rank test to observe whether the difference in survival rates between the intervention and control groups is statistically significant (Collett, 2015) (Fig. 10.2).

The hazard function of survival time provides the conditional failure rate. It is the risk of failure per unit time during the aging process. Hence, the hazard function is also known as the instantaneous failure rate, force of mortality, and age-specific failure rate. A proportional hazards model takes into consideration the fact that different individuals have hazard functions which are proportional to one another (Ediebah et al., 2014). The Cox’s proportional hazards model (CPHM) is a multivariate analysis technique for investigating the relationship between survival time and independent variables (Fig. 10.3). It compares two or more intervention groups by adjusting their risk factors on survival times like the multiple regression. It follows a log-linear model to assesses the hazard ratio by modeling the relative risk of an event as the function of time and covariates (Ediebah et al., 2014).

Noninferiority trials are sometimes conducted to determine whether the alternative treatments are good enough to be used side by side with the main regimen. The comparison of superiority, equivalence, and noninferiority hypotheses are based on 2% margin of difference in event rates. Some of the new pharmaceutical products which are developed might prove to be potentially as effective as existing standard treatments. They eventually become more preferred as compared to the standard ones if they cost less or have fewer side-effects or have fewer adverse reactions or are found to be easier for administration (Scott, 2009).

Every type of study design adopted in health research or pharmaceutical product development generates some evidence which is considered as optimum according to feasibility for that particular time frame for the designated study population located in a specified place. The complexity of study designs and biostatistical applications improves from observational studies to analytical ones which are followed by experimental studies and finally result in systematic review and meta-analysis. The hierarchy of evidence in health research is depicted in Fig. 10.4.

The evidence in healthcare and pharmaceutical product development initiates from observational studies. Every health research on a new disease starts from a single case report. When several similar case reports are available, then a case series analysis is conducted (Guo et al., 2016). The cross-sectional studies or prevalence studies and ecological studies are conducted next to identify the sociodemographic and environmental correlates associated with a specific health outcome. All the correlates which are found to be associated with morbidity or mortality in observational studies are only the probable ones and not definitive. These factors are further evaluated in analytical studies like case-control, cohort, and nested case-control. The highest level of evidence among these are from prospective cohort and nested case-control studies. The experimental studies are the strongest study designs, but they are not feasible in every circumstance. Among these, the most powerful study design for producing the best quality of evidence is RCT. However, for evidence-based practice, the highest level of evidence is obtained from the meta-analysis followed by the systematic reviews on RCTs (Walsh et al., 2014).

Data Warehousing is the random storage of data from various study designs conducted across the world in a separate database. The systematic arrangement of this historical data from warehouse followed by extraction, synthesis, analysis, and interpretation to generate new information and evidence is called Data Mining. This is the most cost-effective way of generating good quality of evidence in pharmaceutical product development and health research (Truong et al., 2017). However, the process of data mining involves very complex and laborious quality assessment and biostatistical methods. Here, the pooled data from various analytical and experimental studies, conducted in different parts of the world, are synthesized and analyzed to evaluate the evidence of whether a specific pharmaceutical product is beneficial or harmful for the consumers. The systematic review and meta-analysis are a part of this multifaceted data mining process, which are frequently used in evidence-based practice (Athanassoulis et al., 2015). The types of reviews on pooled data in evidence-based practice are presented in Fig. 10.5.

The meta-analysis of individual patient data through pooled analysis is the best quality of evidence in evidence-based practice of health research. The next highest level is meta-analysis of experimental studies followed by meta-analysis of analytical studies (Schmidt and Hunter, 2014). The systematic reviews provide the following level of evidence. Every RCT included in the meta-analysis or systematic review must satisfy the CONSORT guidelines with minimum amount of attrition in a preferred reporting items for systematic reviews and meta-analysis (PRISMA) chart. Only the meta-analysis and systematic review provide optimum quality evidence for pharmaceutical product development and clinical practice. The ordinary reviews are considered as overviews and they contain a considerable amount of biased information. Hence, they are never used in decision-making in health research (Moher et al., 2015).

Systematic review is a process of polling of data from various multicentric trials by using predesignated criteria followed by synthesis, analysis, and interpretation of results for rational decision-making. Due to the best quality of study design, the RCTs are preferred for inclusion in a systematic review to explore the effectiveness of an intervention (Jones et al., 2016). The PRISMA chart depicts the number of drop-outs at every stage of the study which are enrollment, allocation, follow-up, and analysis (Fig. 10.6).

The consolidated standards of reporting trials (CONSORT) guidelines are used to evaluate the amount of bias or confounders in each RCT (Schulz et al., 2010). The biases in experimental studies are inappropriate techniques adopted during the pharmaceutical product development which lead to erroneous results, thereby reducing the quality of evidence. The major biases which are identified during this criterion-based procedure are problems with random sequence generation (selection bias), allocation concealment (selection bias), blinding (performance bias and detection bias), incomplete outcome data (attrition bias), and selective reporting (reporting bias) (Schmidt and Hunter, 2014). The major biases in experimental studies and the strategies to minimize them are depicted in Fig. 10.7.

After applying the CONSORT criteria, all the available RCTs related to a specific intervention are arranged in descending order of their quality of evidence and the study with minimum bias tops the list. Only the studies with moderate to high-quality evidence are included in the systematic review (Calvert et al., 2013).

A meta-analysis is often conducted on the selected RCTs which are short-listed during the systematic review to generate evidence for decision-making in clinical practice. Here, complex biostatistical procedures are applied on pooled data available from the selected RCTs (Ford et al., 2014). The strengths of association in the form of pooled odds ratio, pooled relative risk, and pooled hazard ratio are calculated and depicted on forest plots to evaluate the indicators for decision-making in evidence-based clinical practice. The position of the diamond which represents the pooled strength of association in a forest plot provides overall evidence of whether the pharmaceutical product is beneficial or harmful (Haber et al., 2015). This ultimately indicates whether a specific intervention is really working for a designated outcome or not, as shown in Fig. 10.8.

The Cochrane Collaboration, named after the British researcher Archie Cochrane, is famous for its huge database of systematic reviews and meta-analysis. The five main criteria which the Cochrane Collaboration uses for grading the quality of evidence for each experimental study are limitations of study designs, inconsistency between results, indirectness of evidence, imprecision of effect measurement, and funnel plot summary for publication bias (Anderson et al., 2016). The funnel plot is the graphical representation of the reporting of positive and negative findings of a specific type of RCT which is conducted in different settings. This is used to assess publication bias which is suspected when the funnel plot shows mostly positive findings like proving an intervention to be beneficial in most of the occasions (Finnerup et al., 2015) (Fig. 10.9).

The Cochrane Collaboration also conducts sensitivity analysis, random effects model, heterogeneity tests, and subgroup analysis to assess whether the results are consistent and reliable. Thus, every Cochrane systematic review and meta-analysis maintain the highest level of research and provide the best quality of evidence for decision-making in public health policy management for the World Health Organization (Moher et al., 2015).

The following section presents an outline of the inferential statistics used in Experimental Studies:

For assessing before–after effect for categorical variables with one post-test:

Parametric (normal distribution)—Mc Nemar’s test and unadjusted relative risk (RR).

Nonparametric (skewed distribution)—nonparametric Mc Nemar’s test and unadjusted relative risk (RR).

For assessing before–after effect for categorical variables with more than one post-test:

Parametric (normal distribution)—chi-square test, Mc Nemar’s test, and unadjusted relative risk (RR).

Nonparametric (skewed distribution)—Kendall’s W test, nonparametric Mc Nemar’s test, and unadjusted relative risk (RR).

For assessing before–after effect for continuous variables with one post-test:

Parametric (normal distribution)—paired t-test and unadjusted Pearson’s correlation.

Nonparametric (skewed distribution)—Wilcoxon Signed rank test and unadjusted Spearman’s correlation.

For assessing before–after effect for continuous variables with more than one post-test:

Parametric (normal distribution)—Pearson’s ANOVA, paired t-test, and unadjusted Pearson’s correlation.

Nonparametric (skewed distribution)—Kruskal Wallis ANOVA, Wilcoxon signed rank test, and unadjusted Spearman’s correlation.

For baseline risk and treatment effect or absolute risk difference—absolute risk reduction (ARR).

For reduction in the rate of the outcome in the treatment group relative to that in the control group—relative risk reduction (RRR).

For numbers of patients to be treated with experimental therapy to prevent one bad outcome—number needed to treat analysis (NNT)

For treatment success—Kaplan–Meier survival analysis.

For treatment failure—Cox proportional hazards model (regression analysis) with estimation of hazard ratio (hr) and adjusted relative risk.

For missing data and lost to follow-up cases—intention to treat analysis (ITT).

For trial of equal potential/efficacy for intervention as well as gold standard—nonsuperiority and equivalence analysis.

For evaluation of clinical significance—reliability change index (RCI) by Jacobson–Traux.

Statistical Analyses

In this section, statistical methods for data analysis are briefly described. Statistical methods described here have been widely used for the analysis of observational and experimental studies.