CRAN Task View: Meta-Analysis

This task view covers packages which include facilities for meta-analysis of summary statistics from primary studies. The task view does not consider the meta-analysis of individual participant data (IPD) which can be handled by any of the standard linear modelling functions but it does include some packages which offer special facilities for IPD.

The standard meta-analysis model is a form of weighted least squares and so any of the wide range of R packages providing weighted least squares would in principle be able to fit the model. The advantage of using a specialised package is that (a) it takes care of the small tweaks necessary (b) it provides a range of ancillary functions for displaying and investigating the model. Where the model is referred to below, it is this model which is meant.

Where summary statistics are not available, a meta-analysis of significance levels is possible. This is not completely unconnected with the problem of adjustment for multiple comparisons but the packages below which offer this, chiefly in the context of genetic data, also offer additional functionality.

Univariate meta-analysis

Preparing for meta-analysis

• The primary studies often use a range of statistics to present their results. Convenience functions to convert these onto a common metric are presented by: compute.es which converts from various statistics to d, g, r, z, and the log odds ratio, MAd which converts to mean differences, and metafor which converts to an extensive set of effect size measures for comparative studies (such as binary data, person years, mean differences and ratios, and so on), for studies of association (a wide range of correlation types), and for non-comparative studies (proportions, incidence rates, and mean change). It also provides for a measure used in psychometrics (Cronbach’s alpha). esc provides a range of effect size calculations with partial overlap with metafor but with some extras, noticeably for converting test statistics, and also includes a convenience function for collating its output for input to another package like metafor or producing a CSV file. estimraw estimates the cell frequencies from odds ratios, risk ratios, or risk differences. effsize contains functions to compute mean difference effect sizes (Cohen’s d and Hedges’ g) and measures of dominance (Cliff’s delta) and stochastic superiority (Vargha-Delaney A). effectsize provides a large number of different effect sizes and converts between them. psychmeta provides extensive facilities for converting effect sizes and correcting for various restrictions and measurement artifacts. metansue provides some methods for converting to effect sizes, while es.dif computes Cohen’s d, Hedges’ g, biased/unbiased c (an effect size between a mean and a constant), and e (an effect size between means without assuming variance equality) from raw data. MOTE provides a variety of conversions based on Cohen’s d. estmeansd converts between quantiles and means and standard deviations. metaBLUE estimates means and standard deviations from various order statistics. SingleCaseES provides basic effect sizes for single-case designs including parametric and non-overlap measures. smd computes standardised mean differences. CohensdpLibrary (archived) provides Cohen’s d from a variety of designs. metaHelper calculates values commonly used in meta-analysis with a focus on standardized mean differences and related statistics. metaConvert estimates 11 effect size measures.
• meta provides functions to read and work with files output by RevMan 4 and 5.
• metagear provides many tools for the systematic review process including screening articles, downloading the articles, generating a PRISMA diagram, and some tools for effect sizes. revtools provides tools for downloading from bibliographic databases and uses machine learning methods to process them. citationchaser assists in the process of chasing citations. MetaNLP pre–processes titles and abstracts for use in screening before a meta-analysis.
• esci also provides a large range of effect sizes.
• metavcov computes the variance-covariance matrix for multivariate meta-analysis when correlations between outcomes can be provided but not between treatment effects.
• clubSandwich and metafor provide functions to impute variance-covariance matrix for multivariate meta-analysis.
• metafuse uses a fused lasso to merge covariate estimates across a number of independent datasets.
• metapower provides power analysis for meta-analysis and meta-regression. POMADE does the same for the overall average effect size in a meta-analysis of dependent effect sizes.
• PRISMA2020 produces an interactive flow diagram that conforms to PRISMA 2020 version, PRISMAstatement (archived) also generates flowcharts conforming to the PRISMA statement.
• reappraised provides tools for checking the integrity of groups of trials.
• RTSA provides Trial Sequential Analysis.
• Several packages provide assistance in digitising data from published figures: metaDigitise and juicr provide graphical interfaces and accept various input formats. digitize has a more limited range of facilities.

Fitting the model

• Four packages provide the inverse variance weighted, Mantel-Haenszel, and Peto methods: meta, metafor, rmeta, and epiR.
• For binary and time-to-event data, metafor also provides binomial-normal and the Poisson-normal models.
• Packages which work with specific effect sizes may be more congenial to workers in some areas of science. MAd provides functions for the meta-analysis of standardized mean differences and provides a range of graphics.
• psychmeta implements the Hunter-Schmidt method (also known as psychometric meta-analysis) including corrections for reliability and other artifacts.
• Bayesian approaches are contained in various packages. bspmma provides two different models: a non-parametric and a semi-parametric. Graphical display of the results is provided. bayesmeta includes shrinkage estimates, meta-regression, posterior predictive p-values, and forest plots via either metafor or forestplot. Diagnostic graphical output is available. metaBMA provides a Bayesian approach using model averaging; a variety of priors are provided and it is possible for the user to define new ones. MetaStan includes binomial-normal hierarchical models and can use weakly informative priors for the heterogeneity and treatment effect parameters. baggr provides facilities using Stan for hierarchical Bayesian models; graphical facilities are provided. brms can also fit Bayesian meta-analytic models using Stan as the backend. BayesCombo provides facilities using a Bayesian approach and has graphical facilities. RBesT uses Bayesian synthesis to generate priors from various sources. metamisc provides a method with priors suggested by Higgins. RoBMA provides a framework for estimating ensembles of meta-analytic models and using Bayesian model averaging to combine them it also provides meta-regression. ra4bayesmeta provides principled reference analysis within the Bayesian normal-normal model. metabup provides a Bayesian approach using basic uncertainty pooling.
• Some packages concentrate on providing a specialised version of the core meta-analysis function without providing a full range of ancillary functions. These are: metaLik which uses a more sophisticated approach to the likelihood, metatest which provides another improved method of obtaining confidence intervals, and CoTiMA which performs meta-analyses of correlation matrices of repeatedly measured variables for studies with different time lags using a SEM framework with OpenMx as the engine.
• metaplus fits random effects models relaxing the usual assumption that the random effects have a normal distribution by providing t or a mixture of normals.
• ratesci fits random effects models to binary data using a variety of methods for confidence intervals.
• RandMeta estimates exact confidence intervals in random effects models using an efficient algorithm.
• rma.exact estimates exact confidence intervals in random effects normal-normal models and also provides plots of them.
• pimeta provides a range of methods for prediction interval estimation from random effects models and has graphical facilities.
• metamedian implements several methods to meta-analyze one-group or two-group studies that report the median as the outcome. These methods estimate the pooled median in the one-group context and the pooled raw difference of medians across groups in the two-group context. meta and metafor also provide methods for medians.
• MetaUtility proposes a metric for estimating the proportion of effects above a cut-off of scientific importance.
• metasens provides imputation methods for missing binary data.
• metagam provides a framework for meta-analysis of generalised additive models including the case where individual participant data cannot be shared across locations and EvidenceSynthesis also combines across sites where individual participant data cannot be shared.
• metawho implements a method for combining within study interactions.
• metarep provides replicability analyses after a conventional analysis.
• rema uses a permutation approach to handle meta-analyses of rare event data.
• meta.shrinkage uses shrinkage methods to provide better estimates of individual means in meta-analysis.
• metaumbrella provides facilities for umbrella reviews.
• vcmeta provides functions for varying-coefficient meta-analysis as an alternative to the usual fixed- or random-effects methods.
• robustmeta provides methods for meta-analysis for cases where primary studies may have influential outlying values.
• coefa provides a method for conducting a meta-analysis of factor analyses based on co-occurrence matrices.
• CausalMetaR provides robust and efficient methods for estimating causal effects in a target population using a multi-source dataset.
• metainc assesses inconsistency in meta-analyses by calculating the Decision Inconsistency index (DI) and the Across-Studies Inconsistency (ASI) index. Allows input from a variety of packages.
• aides provides analysis of study sizes and includes sequential analysis.

Graphical methods

An extensive range of graphical procedures is available.

• Forest plots are provided in forplo, forestly (interactive plots), forestmodel (using ggplot2), forestplot, forestploter, meta, metafor, metansue, psychmeta, and rmeta. Although the most basic plot can be produced by any of them, they each provide their own choice of enhancements. metaviz provides a range of enhancements.
• Funnel plots are provided in meta, metafor, metansue, rmeta and weightr. In addition to standard funnel plots, a funnel plot for limit meta-analysis is provided in metasens, and metaviz provides an extensive range of enhanced funnel plots and also facilities for their use in the context of visual inference.
• Radial (Galbraith) plots are provided in meta and metafor.
• L’Abbe plots are provided in meta and metafor.
• Baujat plots are provided in meta and metafor.
• meta provides drapery plots.
• MetaAnalyser provides an interactive visualisation of the results of a meta-analysis.
• metaviz provides rainforest plots, an enhanced version of forest plots. It accepts input from metafor.
• DTAplots produces various plots for diagnostic studies including forest and SROC plots.
• GOSH plots are provided in metafor.
• robvis can be used to visualize the results of risk-of-bias (rob) assessments.
• xmeta provides galaxy plots, an analogue of funnel plots for multivariate meta-analysis.

Investigating heterogeneity

• Confidence intervals for the heterogeneity parameter are provided in metafor.
• altmeta presents a variety of alternative methods for measuring and testing heterogeneity with a focus on robustness to outlying studies.
• mc.heterogeneity implements a Monte Carlo based test for heterogeneity.
• boot.heterogeneity provides a bootstrap test for heterogeneity for mean differences, correlations, and odds ratios.
• heterometa converts between various summary measures of heterogeneity.

Model criticism

• An extensive series of plots of diagnostic statistics for detecting outliers and influential studies is provided in metafor.
• metaplus provides outlier diagnostics.
• EValue provides a sensitivity analysis of the effect of unmeasured confounders.
• boutliers provides bootstrap distributions for outlier detection and influence diagnostics.
• metaconfoundr provides a number of ways to visualise confounding relationships in meta-analysis.
• RoBMA allows comparison of different meta-analytic models.

Small study bias / Unobserved studies / Publication bias

The issue of whether small studies give different results from large studies can be addressed by visual examination of the funnel plots mentioned above. In addition:

• meta and metafor provide both the non-parametric rank correlation test suggested by Begg and Mazumdar and a range of regression tests modelled after the approach of Egger.
• metamisc provides funnel plots and tests for asymmetry.
• xmeta provides methods for small study effects in multivariate meta-analysis.

A related issue in meta-analysis is the problem of unobserved studies and publication bias.

• Rosenthal’s fail safe n is provided by MAd. metafor provides it as well as two more recent methods by Orwin and Rosenberg and a generalization to random-effects models.
• fsn computes the fail-safe number with confidence interval.
• Duval’s trim and fill method is provided by meta and metafor.
• metasens provides the Copas selection model and also the method of limit meta-analysis (a regression based approach for dealing with small study effects) due to Rücker and colleagues.
• RobustBayesianCopas fits a robust version of the Copas selection model.
• selectMeta provides various selection models: the parametric model of Iyengar and Greenhouse, the non-parametric model of Dear and Begg, and proposes a new non-parametric method imposing a monotonicity constraint.
• weightr provides facilities for using the weight function model of Vevea and Hedges. metafor also provides an implementation thereof and a variety of other selection models.
• RoBMA includes Bayesian versions of selection models.
• phacking models only non-affirmative studies to allow for selection of studies and selection within studies.
• puniform provides methods using only the statistically significant studies, methods for the special case of replication studies, and sample size determinations.
• PublicationBias performs sensitivity analysis of the number of unpublished studies needed to have a specified influence.
• The metansue package allows the inclusion by multiple imputation of studies known only to have a non-significant result.
• publipha estimates models accounting for publication bias or p-hacking using a Bayesian framework.
• metafor provides the test of excess significance.
• multibiasmeta conducts sensitivity analyses for the joint effects of internal and publication biases.
• metabias provides common components (classes, methods, documentation) for several other packages to investigate within- and across-study biases in meta-analysis.

Other study designs

• SCMA provides single case meta-analysis. It is part of a suite of packages dedicated to single-case designs.
• joint.Cox provides facilities for the meta-analysis of studies of joint time-to-event and disease progression.
• dfmeta provides meta-analysis of Phase I dose-finding clinical trials.
• metamisc provides for meta-analysis of prognostic studies.
• metamicrobiomeR provides meta-analysis of zero-inflated beta microbiome data fitted via GAMLSS models.
• metaSurvival estimates the survival curves from data extracted from primary study survival curves.

Meta-analysis of significance values

• Fisher’s method and Lancaster’s are available in aggregation, metap, metapro, and poolr.
• Stouffer’s, Tippett’s, and Wilkinson’s method are available in metap and poolr.
• Edgington’s method, inverse-t, logit, mean of p, and mean of z are all available in metap.

In all cases poolr considers correlated p-values in addition to independent. The others above do not.

• TFisher provides Fisher’s method using both hard and soft thresholding for the p-values. There is a wrapper in metap for the hard threshold case.
• harmonicmeanp uses a method based on the harmonic mean of p-values which is robust to correlation between the p-values.
• amanida provides meta-analysis of metabolite data using p-values and fold change.
• metap provides simple graphics including albatross plots.

Some methods are also provided in some of the genetics packages mentioned below.

Multivariate meta-analysis

Standard methods outlined above assume that the effect sizes are independent. This assumption may be violated in a number of ways: within each primary study multiple treatments may be compared to the same control, each primary study may report multiple endpoints or multiple assessments, or primary studies may be clustered for instance because they come from the same country or the same research team.

• mvmeta assumes the within study covariances are known and provides a variety of options for fitting random effects. metafor provides fixed effects and likelihood based random effects model fitting procedures. Both these packages include meta-regression, metafor also provides for clustered and hierarchical models.
• mixmeta provides an integrated interface to standard meta-analysis and extensions like multivariate and dose-response models.
• mvtmeta provides multivariate meta-analysis using the method of moments for random effects although not meta-regression.
• clubSandwich, robumeta, and metafor provide cluster-robust variance estimates for clustered and hierarchical estimates.
• wildmeta conducts single coefficient tests and multiple-contrast hypothesis tests of meta-regression models using cluster wild bootstrapping.
• metaSEM provides multivariate (and univariate) meta-analysis and meta-regression by embedding it in the structural equation framework and using OpenMx for the structural equation modelling. It can provide a three-level meta-analysis taking account of clustering and allowing for level 2 and level 3 heterogeneity. It also provides via a two-stage approach meta-analysis of correlation or covariance matrices.
• dosresmeta concentrates on the situation where individual studies provide information about a dose-response relationship. MBNMAdose provides a Bayesian analysis using network meta-analysis of dose-response studies.
• CIAAWconsensus has a function for multivariate meta-analysis in the context of atomic weights and estimating isotope ratios.
• BayesMultMeta provides Bayesian inference for the parameters of a multivariate random-effects model with application to multivariate meta-analysis.
• remaCor provides multivariate meta-analysis when the correlation between effect sizes is known.
• xmeta provides a variety of methods for multivariate meta-analysis.
• crwbmetareg uses the WLS estimator of Stanley and Doucouliagos both with and without covariates followed by computing the relevant p-values using the cluster robust wild bootstrap methodology.

Meta-analysis of studies of diagnostic tests

A special case of multivariate meta-analysis is the case of summarising studies of diagnostic tests. This gives rise to a bivariate, binary meta-analysis with the within-study correlation assumed zero although the between-study correlation is estimated. This is an active area of research and a variety of methods are available including what is referred to here as Reitsma’s method and the hierarchical summary receiver operating characteristic (HSROC) method. In many situations these are equivalent.

• mada provides various descriptive statistics and univariate methods (diagnostic odds ratio and Lehman model) as well as the bivariate method due to Reitsma. Meta-regression is provided.
• bamdit provides Bayesian meta-analysis with a bivariate random effects model (using JAGS to implement the MCMC method).
• meta4diag provides Bayesian inference analysis for bivariate meta-analysis of diagnostic test studies and an extensive range of graphical methods.
• diagmeta considers the case where the primary studies use an analysis using multiple cut-offs.
• NMADiagT provides network meta-analysis of diagnostic tests in a Bayesian framework using Stan as the engine.
• DTAplots produces various plots for diagnostic studies including forest and SROC plots. The packages above also provide various graphical methods.
• MVPBT provides tests for small study effects in diagnostic test meta-analysis.
• dmetatools provides confidence intervals for the AUC of the summary ROC curve and related methods.
• CopulaREMADA provides the bivariate copula mixed model for meta-analysis of diagnostic test accuracy studies.

Meta-regression

Where suitable moderator variables are available they may be included using meta-regression. All these packages are mentioned above, this just draws that information together.

• metafor provides meta-regression (multiple moderators are catered for). Various packages rely on metafor to provide meta-regression (meta, and MAd) and all of these provide bubble plots. psychmeta also uses metafor.
• metaLik, metansue, metaSEM, and metatest also provide meta-regression.
• mvmeta provides meta-regression for multivariate meta-analysis as do metafor and metaSEM.
• mada provides for the meta-regression of diagnostic test studies.
• GENMETA uses generalised meta-analysis to handle the situation where the studies do not all use the same regressors.
• jarbes uses a Bayesian approach of hierarchical meta-regression.
• metacart uses classification and regression trees to identify interactions between moderators.
• metaforest investigates heterogeneity using random forests. Note that it has nothing to do with forest plots.
• pema provides a penalised approach to meta-regression useful for situations with a large number of moderators relative to observations.
• bayesmeta includes meta-regression in a Bayesian framework.
• CAMAN (archived) offers the possibility of using finite semiparametric mixtures as an alternative to the random effects model where there is heterogeneity. Covariates can be included to provide meta-regression.
• crwbmetareg fits meta-regression models using weighted least squares and then uses cluster robust wild bootstrap methodology for inferences.

Individual participant data (IPD)

Where all studies provide individual participant data, software for the analysis of multi-centre trials or multi-centre cohort studies should prove adequate but is outside the scope of this task view (see the MixedModels task view for relevant packages). Other packages which provide facilities related to IPD are:

• ecoreg which is designed for ecological studies enables estimation of an individual level logistic regression from aggregate data or individual data.
• multinma provides network meta-analysis and network meta-regression models for aggregate data, individual patient data, and mixtures thereof.
• MetaIntegration combines IPD data with external models.
• bipd uses a Bayesian approach for IPD. It includes facilities for multiple imputation using mice.

Network meta-analysis

Also known as multiple treatment comparison, this is a very active area of research and development. Note that some of the packages mentioned above under multivariate meta-analysis can also be used for network meta-analysis with appropriate setup.

• netmeta works in a frequentist framework. It provides an extensive range of graphical and other displays including network graphs and a heatmap for displaying inconsistency and heterogeneity. A frequentist analogue of SUCRA is also available.
• A Bayesian framework is provided by pcnetmeta, which uses JAGS. It provides a number of data-sets. nmaINLA uses integrated nested Laplace approximations as an alternative to MCMC. It provides a number of data-sets. NMADiagT provides network meta-analysis of diagnostic tests in a Bayesian framework using Stan as the engine; graphical output is provided. gemtc acts as a front-end to BUGS or JAGS, bnma provides arm-based methods using JAGS as the engine, and metapack provides methods using built-in MCMC code.
• multinma provides network meta-analysis and network meta-regression models for aggregate data, individual patient data, and mixtures thereof.
• nmathresh provides decision-invariant bias adjustment thresholds and intervals the smallest changes to the data that would result in a change of decision. NMAoutlier detects outliers in NMA using a forward search.
• pcnetmeta provides network graphs. nmaplateplot displays the results from an NMA using a heatplot style and also displays SUCRA.
• nmarank evaluates hierarchies of evidence in network meta-analysis.
• rnmamod uses a Bayesian approach to perform NMA while addressing (aggregate) missing participant outcome data and provides graphics.
• viscomp provides several visualization tools for exploring the behavior of the components in a network meta-analysis of multi-component interventions.
• OssaNMA calculates minimum total sample size needed for prespecified power or optimal allocation for each treatment group with a fixed total sample size.
• MBNMAtime provides for analysis of multiple time points from studies using a Bayesian framework.
• crossnma can be used to conduct a network meta-analysis for individual participant data, aggregate data, and mixtures thereof in a Bayesian framework.
• PINMA provides improved methods to construct prediction intervals for network meta-analysis.
• rankinma provides treatment ranking in NMA.
• ssifs evaluates consistency in NMA.
• NMA uses a contrast-based approach and includes standard diagnostic and graphical methods. It uses an improved REML estimation procedure.
• CBnetworkMA performs contrast-based NMA in a Bayesian framework.
• closeloop calculate distance between single-arm observational studies using covariate information to remove heterogeneity in NMA.

Genetics

There are a number of packages specialising in genetic data: catmap combines case-control and family study data, graphical facilities are provided, CPBayes uses a Bayesian approach to study cross-phenotype genetic associations, corrmeta performs correlated meta-analysis across multiple scans, gap combines p-values, getmstatistic quantifies systematic heterogeneity, getspres uses standardised predictive random effects to explore heterogeneity in genetic association meta-analyses, GMCM uses a Gaussian mixture copula model for high-throughput experiments, GSEMA performs the different steps of gene set enrichment meta-analysis, MendelianRandomization provides several methods for performing Mendelian randomisation analyses with summarised data, metaGE provides functions for a meta-analysis of genome-wide association studies for studying genotype x environment interactions. MetaHD performs multivariate meta-analysis for high-dimensional metabolomics data, MetaIntegrator (archived) provides meta-analysis of gene expression data, metaMA provides meta-analysis of p-values or moderated effect sizes to find differentially expressed genes, metaRNASeq does meta-analysis from multiple RNA sequencing experiments, MetaSKAT provides for meta-analysis of the SKAT, MetaSubtract uses leave-one-out methods to validate meta-GWAS results, ofGEM provides a method for identifying gene-environment interactions using meta-filtering, RobustRankAggreg provides methods for aggregating lists of genes, and SPAtest combines association results.

Data-sets

• metadat provides a large number of data-sets used in meta-analysis.
• psymetadata provides more data-sets from meta-analyses in psychology research.
• nmadb (archived) provides access to a database of network meta-analyses.
• KenSyn provides data-sets to accompany a French language book on meta-analysis in the agricultural sciences.
• metabolic provides data and code for a published meta-analysis.

Interfaces

• Plug-ins for Rcmdr are provided by: RcmdrPlugin.EZR which uses meta and metatest, RcmdrPlugin.MA which uses MAd and metafor, and RcmdrPlugin.RMTCJags for network meta-analysis using BUGS code.
• Both JASP and jamovi provide meta-analysis with R as the backend (the latter via the MAJOR module).
• miniMeta provides a shiny interface to meta.

CRAN packages

Related links

• R mailing list for meta-analysis

Other resources

• CRAN Task View: MixedModels