Meta-analysis is a research method for systematically combining and synthesizing findings from multiple quantitative studies in a research domain. Despite its importance, most literature evaluating meta-analyses are based on data analysis and statistical discussions. This paper takes a holistic view, comparing meta-analyses to traditional systematic literature reviews. We described steps of the meta-analytic process including question definition, data collection, data analysis, and reporting results. For each step, we explain the primary purpose, the tasks required of the meta-analyst, and recommendations for best practice. Finally, we discuss recent developments in meta-analytic techniques, which increase its effectiveness in business research.
It is expected that a new study will build on previous findings to contribute to knowledge formation and development in a research domain. To accomplish this, authors must define their research objectives based on gaps in the relevant literature, and design a study to address this gap. This requires deep knowledge and understanding of a research domain, which can be facilitated by systematic literature reviews (SLRs).
This method is considered to be a scientific and highly informative method for systematically collecting, reviewing, and synthesizing research findings on a particular topic to determine what is known –and what is not known at domain. SLRs allow readers to glean a deep understanding of literature and also help them to identify research gaps in the area. In this way, an SLR may be viewed as a platform for knowledge advancement.
Meta-analysis follows very technical and sophisticated procedures to collect, combine, and analyze empirical research. This rigorous approach guarantees the validity and reliability of the method, while at the same time obfuscating the technique for the researchers and practitioners who could benefit from conducting meta-analyses. the primary objective of this white Paper is to detail current practices and advancements in meta-analysis research and to contrast the technique with traditional SLRs. The blog eschews technical jargon to enhance the accessibility of this research among readers who do not possess advanced statistical knowledge. Moreover, instead of focusing exclusively on data collection and analysis, this blog covers the entire meta-analytic process including question definition, data collection, data analysis, and presentation of results. Finally, it will introduce recent advancements in meta-analytic technique to demonstrate the ongoing developments.
There are several fundamental differences between traditional SLRs and meta‐analysis, distinguishing these popular methods for accumulating knowledge in a research domain. There are several fundamental differences between traditional SLRs and meta‐analysis, distinguishing these popular methods for accumulating knowledge in a research domain.
Traditional SLRs and Meta-Analysis
There are several fundamental differences between traditional SLRs and meta-analysis, distinguishing these popular methods for accumulating knowledge in a research domain provides an overview of these differences based on five key questions (i.e., What, Why, When, Where, and How). We believe these key questions provide a better picture of both traditional SLRs and meta-analysis and their difference and help researchers choose an appropriate literature review method.
A traditional SLR is a “process for assembling, arranging, and assessing existing literature in a research domain”. In this process, assembling involves identification (i.e., defining the literature review domain, main question, and source type/quality) and acquisition (i.e., obtaining papers to be included). The scientific steps for an SLR include organization (i.e., specifying the codes and framework) and purification (i.e., specifying the inclusion and exclusion criteria). The final step is setting the future research agenda, based on a gap analysis. Following this process helps researchers to meet two main goals:
- Providing a comprehensive picture of what is known in a research domain (i.e., defining its scope overview, identifying inconsistencies, and their probable explanation, and developing a framework to summarize previous research);
- Providing directions for future research based on what is not known in that research domain.
Traditional SLRs include: domain-based reviews; theory-based reviews, and method-based reviews. Domain-based reviews synthesize studies in the same research domain to extend the body of literature in this domain. Synthesizing diverse perspectives allows authors to describe state-of-the-art knowledge in the research domain and identify useful paths for research.
There are several types of domain-based reviews:
- Structured Reviews
- Framework Based Reviews
- Bibliometric Reviews
- Hybrid Reviews
Meta-analysis is a collection of statistical methods that integrates the results of a large number of studies to provide an aggregate summary of knowledge in a research domain. The advantage of meta-analysis over an individual study is in its higher power. A meta-analysis combines the findings of single studies for specific relationships, it allows authors to achieve statistically precise and accurate conclusions about the strength and direction of a relationship between variables and to resolve contradictory results in prior studies by examining the impact of moderator variables. Meta-analysts calculate an “effect size,” which indicates the direction and strength of association between two variables and is a standardized metric that is comparable across studies. The researcher extracts information from each study comprising the meta-analytic database to calculate an effect size from every single study. After calculating an effect size, for each study, the meta-analyst combines all of the effect sizes to determine the strength and direction of associations between pairwise relationships at the aggregate level. Popular effect sizes in business and management are the correlation coefficient (r) and, standardized mean difference coefficient.
Usually, there are conflicting findings in a literature stream. Therefore, the meta-analyst identifies appropriate moderators in an attempt to explain variations across studies. Some control variables may also be defined to account for other sources of variation in effect sizes. In their meta-analysis, studied the impact of firm innovativeness on firm performance. In addition to measuring how firm innovativeness influences firm performance, these researchers assess the impact of control variables such as product diversification, firm age, intangible factors, and competitive intensity on their relationships. They also explore the moderating impact of firm size, advertising intensity, industry, and country.
In overall conclusion meta-analysis is an effective way to advance current knowledge in business and management, and is more scientific than a pure bibliometric type of SLRs. Therefore, there is increasing interest among researchers to publish meta-analysis papers because of their impact on knowledge development. However, the technical nature of meta-analyses may prove daunting for academics and practitioners to understand and conduct. Thus, in the current research, we demonstrate this study method to facilitate researchers’ understanding of how to conduct meta-analysis.
A meta-analysis begins with a fruitful and novel research question, such as reconciling the conflicting findings in a research domain. This question definition helps researchers develop a Meta-analytic framework to guide the whole meta-analysis process. The authors then engaged in data collection, employing different strategies to include different types of publication in the process and applying logical inclusion/exclusion criteria to finalize the meta-analytic database. Then, the meta-analyst uses a coding manual to code primary variables, moderators, and control variables in each individual study, extracting or calculating the effect sizes, assessing outliers and publication bias, and combining effect sizes. Once the meta-analyst has selected a fixed-effects model or random-effects model, heterogeneity Moderators are then analyzed. is assessed and the overall testing pairwise relationships in the framework are conducted of various software packages are available for conducting meta-analyses including commercial programs (e.g., CMA, Review Manager, and Stata) and open-source (R packages) software. Finally, the meta-analyst reports the result. This manuscript proposes an overarching structure to cover all important aspects of a meta-analysis in business and management.
It is worth noting that meta-analysis is an evolving method that has seen several advancements in recent years that expand the effectiveness and accuracy of results. Meta-analytic Bayesian analysis and network analysis are examples of promising advancements in meta-analyses. Finally, employing machine learning in meta-analysis has been shown to facilitate the meta-analytic process and increase its quality. This promising approach is in its initial stages and needs more development before use in practice.
