Commentary: four ways in which data-free papers on animal personality fail to be impactful

A commentary on
Four ways in which data-free papers on animal personality fail to be impactful

by DiRienzo, N., and Montiglio, P.-O. (2015). Front. Ecol. Evol. 3: 23. doi: 10. 3389/fevo. 2015. 00023

In a recent Frontiers in Ecology and Evolution paper, DiRienzo and Montiglio (2015) suggest that “ the literature on animal personality is dominated by papers lacking any data” and that additional data-free papers do little to move the field forward. Here, we present our quantitative analysis of data-free papers in the animal personality and behavioral ecology literature, as well as discuss the value of such articles. We found that data-free papers composed 20% of all publications on animal personalities in the last decade. We further determined the prevalence of data-free papers within the top-ten most cited papers in other behavioral ecology fields, showing that animal personality did not have an unusual excess of highly cited data-free papers when compared to other subfields of animal behavior. Regarding the impact of data-free papers, we argue that impactful data-free papers provide coherent syntheses and reviews of current knowledge, integrate different fields of thought in novel ways, or identify important future directions within a framework beyond the scope typical of empirical papers. Ultimately, we suggest that a combination of robust empirical studies and effectual data-free papers is vital to advance a field.

To quantify the frequency of data-free papers in the animal personality literature, we conducted a web search using Web of Science (BIOSIS Previews). Specifically, we used the search terms “ behavio* syndrome*” OR “ animal personalit*” for the period 2005–2014, which follows Sih et al.’s influential papers in 2004 ( Sih et al., 2004a , b ). Our search differed from DiRienzo and Montiglio’s (“ individual behavioral variation”) because we felt that ours more accurately assessed the overall animal personality literature; however, searches using any combination of these terms yielded the same conclusions. By reading the title and abstract for each paper, we determined whether it contained data. We classified data-free papers into sub-categories: literature reviews, hypothetical ideas/concepts, and general methods. Maintaining consistency with DiRienzo and Montiglio’s definition of data-free, all statistical methods, meta-analyses, simulations, or mathematical modeling papers were not considered data-free as they generate quantitative analyses. We excluded papers not explicitly related to behavioral ecology and animal personality (e. g., human studies).

We obtained 473 papers, of which 20% were data-free. Of these data-free papers, 70% were literature reviews, and 19% were hypothetical ideas/conceptual papers. Many of these reviews and idea papers sought to: combine animal personality with other fields, update the state of the field, or survey the animal personality literature within a specific taxon ( Jandt et al., 2014 ). There were also a small number of data-free papers describing general methodologies, approximately 11% of data-free papers. We therefore determined that data-free papers represent the minority of publications in the animal personality field to date, and most of these data-free papers synthesize research to develop conceptual frameworks or reviews.

In addition to overall number of papers, DiRienzo and Montiglio argue that data-free papers receive disproportionately more citations than data papers. However, data-free papers that synthesize information and provide new avenues of research are likely the most highly cited publications in many fields. To test this view, we assessed the number of data-free papers among the top 10 most cited papers in: animal social networks, kin selection, parental care, optimal diet, sexual selection, alternative mating tactics, and anti-predator behavior. We searched citations from 1993–2014, consistent with the years in DiRienzo and Montiglio’s analysis. On average, 49% of the top 10 most cited papers in these fields were data-free, with variation from 10% in alternative mating tactics to 80% in kin selection. While this inter-field comparison illuminates the variability of data-free papers among top-cited articles across topics, the fields we evaluated are at different stages and of different scopes. Thus, analyzing the impact and contributions of data-free papers may require a more qualitative approach.

We suggest that data-free papers can be impactful in several ways. Conceptual papers can provide frameworks that bridge multiple fields of thought. Literature reviews can synthesize the state of a field, providing organization and focus to generate predictions. More specific reviews can outline the progression of subfields. Data-free papers can also highlight gaps in a field and suggest future research directions. Notably, as animal personality integrates multiple disciplines, combining concepts from life history ( Réale et al., 2010 ), physiology ( Biro and Stamps, 2010 ), genetics ( Van Oers and Sinn, 2013 ), development ( Stamps and Groothuis, 2010a , b ), ecology ( Sih et al., 2012 ), and evolution ( Wolf and Weissing, 2012 ), data-free papers can be particularly useful, connecting these diverse topics into cohesive frameworks.

DiRienzo and Montiglio (2015) argue, in particular, that although some earlier data-free publications have substantially inspired the field ( Dall et al., 2004 ; Sih et al., 2004a , b ; Wolf et al., 2007 ; Réale et al., 2010 ; Stamps and Groothuis, 2010a , b ), the continued publication of data-free papers fails to push animal personality forward. We believe that while some excess data-free papers may provide little additional insight, many data-free articles continue to be impactful for the same reasons discussed above. As DiRienzo and Montiglio (2015) mention, time will tell if the data-free papers being published currently ultimately prove valuable. While impact is often evaluated primarily based upon number of citations, we think that the most impactful papers are ones that stimulate future research and provide new avenues to confront complex problems.

Data-free papers that accomplish this goal often necessarily cover an extensive topical scope. DiRienzo and Montiglio suggest that such data-free papers could usually be replaced by empirical studies testing the hypotheses of interest. While papers that simultaneously propose and test a new idea are admirable, we contend that the purview of high impact data-free papers almost always exceeds any one empirical study. Articles with data usually address only a small part of the overall frameworks championed by good data-free papers, partly because journals do not provide the page space to describe both empirical results and a novel, larger framework in one paper.

We close by emphasizing that the advancement of a field requires both empirical and data-free articles. For example, fields inundated with mostly data-free papers (though we doubt that this ever happens) would gain little from further data-free articles, while fields dominated by empirical papers would benefit from data-free papers that organize results within a cohesive, conceptual framework, and provide predictions. This scenario illustrates the notion that measuring impact requires a more nuanced analysis than citation counts, and that arguing that one type of paper could completely replace the other ignores their interrelatedness. Data-free and empirical papers complement each another, and impact has no single, simple definition.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Biro, P. A., and Stamps, J. A. (2010). Do consistent individual differences in metabolic rate promote consistent individual differences in behavior? Trends Ecol. Evol . 25, 653–659. doi: 10. 1016/j. tree. 2010. 08. 003

CrossRef Full Text | Google Scholar

Dall, S., Houston, A., and McNamara, J. (2004). The behavioural ecology of personality: consistent individual differences from an adaptive perspective. Ecol. Lett . 7, 734–739. doi: 10. 1111/j. 1461-0248. 2004. 00618. x

CrossRef Full Text | Google Scholar

DiRienzo, N., and Montiglio, P.-O. (2015). Four ways in which data-free papers on animal personality fail to be impactful. Front. Ecol. Evol. 3: 23. doi: 10. 3389/fevo. 2015. 00023

CrossRef Full Text | Google Scholar

Jandt, J., Bengston, S., Pinter-Wollman, N., Pruitt, J., Raine, N., Dorhaus, A., et al. (2014). Behavioural syndromes and social insects: personality at multiple levels. Biol. Rev . 89, 48–67. doi: 10. 1111/brv. 12042

PubMed Abstract | CrossRef Full Text | Google Scholar

Réale, D., Garant, D., Humphries, M. M., Bergeron, P., Careau, V., and Montiglio, P.-O. (2010). Personality and the emergence of the pace-of-life syndrome concept at the population level. Philos. Trans. R. Soc. Lond. B. Biol. Sci . 365, 4051–4063. doi: 10. 1098/rstb. 2010. 0208

PubMed Abstract | CrossRef Full Text | Google Scholar

Sih, A., Bell, A. M., and Johnson, J. C. (2004a). Behavioral syndromes: an ecological and evolutionary overview. Trends Ecol. Evol . 19, 372–378. doi: 10. 1016/j. tree. 2004. 04. 009

PubMed Abstract | CrossRef Full Text | Google Scholar

Sih, A., Bell, A. M., Johnson, J. C., and Ziemba, R. E. (2004b). Behavioral syndromes: an integrative overview. Q. Rev. Biol . 79, 241–277. doi: 10. 1086/422893

PubMed Abstract | CrossRef Full Text | Google Scholar

Sih, A., Cote, J., Evans, M., Fogarty, S., and Pruitt, J. (2012). Ecological implications of behavioural syndromes. Ecol. Lett . 15, 278–289. doi: 10. 1111/j. 1461-0248. 2011. 01731. x

PubMed Abstract | CrossRef Full Text | Google Scholar

Stamps, J. A., and Groothuis, T. G. G. (2010a). Developmental perspectives on personality: implications for ecological and evolutionary studies of individual differences. Philos. Trans. R. Soc. Lond. B Biol. Sci . 365, 4029–4041. doi: 10. 1098/rstb. 2010. 0218

PubMed Abstract | CrossRef Full Text | Google Scholar

Stamps, J. A., and Groothuis, T. G. G. (2010b). The development of animal personality: relevance, concepts and perspectives. Biol. Rev . 85, 301–325. doi: 10. 1111/j. 1469-185X. 2009. 00103. x

PubMed Abstract | CrossRef Full Text | Google Scholar

Van Oers, J., and Sinn, D. L. (2013). “ Quantitative and molecular genetics of animal personality,” in Animal Personalities: Behavior, Physiology and Evolution , eds C. Carere and D. Maestripieri (Chicago, IL: University of Chicago Press), 149–200.

Wolf, M., van Doorn, G. S., and Weissing, F. J. (2007). Evolutionary emergence of responsive and unresponsive personalities. Proc. Natl. Acad. Sci. U. S. A . 105, 15825–15830. doi: 10. 1073/pnas. 0805473105

PubMed Abstract | CrossRef Full Text | Google Scholar

Wolf, M., and Weissing, F. J. (2012). Animal personalities: consequences for ecology and evolution. Trends Ecol. Evol. 27, 452–461. doi: 10. 1016/j. tree. 2012. 05. 001

PubMed Abstract | CrossRef Full Text | Google Scholar