During the last half of the 20th century, social and technological changes in academic research groups have challenged traditional research record-keeping practices, making them either insufficient or obsolete. New practices have developed but standards (best practices) are still evolving. Based on the authors’ review and analysis of a number of sources, they present a set of systematically compiled best practices for research record-keeping for academic research groups. These best practices were developed as an adjunct to a research project on research ethics aimed at examining the actual research record-keeping practices of active academic scientists and their impact on research misconduct inquiries.
The best practices differentiate and provide separate standards for three different levels within the university: the individual researcher, the research group leader, and the department/institution. They were developed using a combination of literature reviews, surveys of university integrity officials, focus groups of active researchers, and inspection of university policies on research record-keeping. The authors believe these best practices constitute a “snapshot” of the current normative standards for research records within the academic research community. They are offered as ethical and practical guidelines subject to continuing evolution and not as absolute rules. They may be especially useful in training the next generation of researchers.
Good record-keeping is central to the scientific process. 1–4 Good research records encompass much more that just research data. They include but are not limited to planning and protocol descriptions, data manipulations and analysis procedures, personal and group interpretations of the results, and important communications and group decisions among collaborators. Data management is a subset of the broader concept of research record-keeping. Research records are important for managing and planning research, for replicating results, for documenting collaborations, for publishing and peer review, and for complying with governmental and institutional rules and regulations. In recent decades, legal and regulatory uses of research records have become prominent. Research records have long been used to resolve intellectual property disputes. 2 However, research misconduct scandals in the 1980s and 1990s involving falsification and fabrication of research records provoked the federal government to require universities to implement research misconduct policies. 2 Examination of research records is a central feature of inquiries and investigations under such university policies.
We recently conducted a survey of 96 university officials who are responsible for such inquiries and investigations as part of a research project on the impact of research records in research misconduct cases. 5 We found that over half of the officials who reported experience with misconduct cases at their institutions also reported that they had been hampered in their inquires/investigations by inadequate research records. Also, another recent survey of investigators who had been funded by the National Institutes of Health (NIH) on the prevalence of questionable research practices noted that 27% of the 3,247 respondents admitted to “inadequate record-keeping related to research projects.” 6
Given the importance of good research records, it is somewhat surprising that formal standards for such records are the exception rather than the rule in academic research laboratories. Although governments have mandated standards for good research records for certain segments of the research community—most notably in the area of human health and safety research through the stringent regulations of the U.S. Food and Drug Administration (FDA) and the Environmental Protection Agency (EPA) 7,8 —the majority of academic researchers are not constrained by any external set of record-keeping guidelines. In fact, most academic scientists find the mandated FDA record-keeping practices both onerous and unnecessary. Academic researchers prefer informal guidelines rather than formal standards for record-keeping.
During the last half of the 20th century, technological changes in how records are produced, collected, analyzed and stored, coupled with social changes in the nature of research groups, have created new challenges for research record-keeping. 9 Traditional practices for such record-keeping are either no longer sufficient or, at worse, obsolete for the modern researcher. New record-keeping practices have arisen to meet these challenges; however, very little research has been done on defining in a systematic way the best practices * for research record-keeping in the 21st century academy. The literature is replete with descriptions of record-keeping practices, but they are usually personal assessments. In this article, we will discuss the modern challenges to traditional research record-keeping practices. We will then articulate best practices (principles and specific practices) for keeping research records that have emanated from our research and which, we believe, can help researchers respond to these challenges. Because these principles and practices were developed in a systematic way, we believe that they represent current norms within the scientific community (excluding FDA- or EPA-regulated research). We offer these best practices as ethical guidelines for researchers, research group leaders, and research institutions, not as absolute rules or legal requirements.
During the 19th and early 20th centuries, a tradition for research record-keeping developed in the academic physical and biological sciences that focused on the research notebook as the primary tool for organizing research projects, planning experiments, recording data, analyzing the results, and storing these records for future reference. Research groups were small, allowing group leaders to personally train young researchers and to regularly examine primary research records; this process ensured the transmission of the record-keeping tradition to the next generation. During the last half of the 20th century, a number of trends in research have complicated the traditional record-keeping process and interfered with the training of new researchers in record-keeping practices. 9 Chief among these trends were (1) the rise of large research groups; (2) the introduction of research instruments and methods that self-recorded data either on paper printouts or photographically (e.g., X-ray films, slides, and photographs); and (3) the rise of computers in the laboratory.
The current environment for research training, where a graduate student or a postdoctoral fellow is usually a part of the large research group, encourages the teaching of record-keeping practices to new researchers by peers rather than by the senior group leader. Oversight of research is often focused only on the results of a young researcher’s project; it appears to be rare for a researcher’s primary research records to be examined by a senior research group leader. If there are no standards imposed by the research group leader, the actual record-keeping practices within the laboratory group may deviate from accepted norms. The practices actually in use will depend on an individual researcher’s own attitude toward record-keeping and on the attitudes and practices of peers. An extreme example of this occurs when foreign students and postdocs keep their research records in their native languages if there is no requirement to keep records in a common language. There are many anecdotal cases of this practice because international students and postdocs are prevalent in academic research groups. Kanare 10 describes a case where a research group leader discovered, after his Japanese postdoc had returned to Japan, that the postdoc had written his notebooks in Japanese.
The introduction of self-recording research instruments and new photography-based techniques were a major technical advance for science, but they have had the unfortunate byproduct of reducing the utility of the traditional bound research notebook, and more important, of beginning the fragmentation of the research record that is so common today. The first self-recording instruments in the 1940s and 1950s produced paper-based output (they now produce digital files). Photographic recording techniques also appeared at about this time, such as imaging radiolabels on paper chromatographs. These types of records were difficult to incorporate into traditional bound laboratory notebooks due to the cumbersome need to glue or tape such output into notebooks and the sheer volume of such records. Many research groups abandoned the traditional bound notebook in favor of loose-leaf binders or manila folders for both the collection of data output (instrument and photographic data) and for the planning of experiments. Soon, the processing and analysis of data was also placed into loose-leaf binders or manila folders, thus contributing to the physical fragmentation of research records.
The rise of computers in the laboratory was another major technical advance. They made possible the collection and processing of monumental amounts of data that were critical for the advancement of many fields (e.g., X-ray crystallography, genomics, and proteomics). However, the computer also promoted the fragmentation of the research records in academic laboratories and made oversight more difficult. Researchers routinely use many different computer applications in the course of their work and produce a variety of research records (e.g., word processing files, spreadsheets, instrument data output files, etc.). For a single research project, relevant computer files can be scattered among several computers within a research group, depending on the number of individuals working on the project. Within each computer, the pertinent files may be scattered among apparently unrelated folders on the hard drive. In addition, the rapid obsolescence of computer software and hardware (often only three to five years per generation) has made the long-term electronic retrieval of research records problematic, if not impossible.
Computer technology has also been called upon to provide solutions to these modern record-keeping challenges. Laboratory information management systems (LIMS) and electronic laboratory notebooks (ELNs) are the two major types of tools that have been developed. 11–13 Industry has embraced these systems due, in part, to intellectual property concerns and stringent FDA regulations. LIMS systems in particular have been extremely helpful in managing laboratories that generate a great deal of data and use standardized procedures, which is often the case in industrial and clinical labs and pharmaceutical quality control facilities. However, in academic (discovery-type) research, protocols and methods can change frequently, and the amount of data collected tends to be relatively limited. Because LIMS systems need to be customized for each process they handle, such systems are therefore viewed as being too inflexible for the typical academic laboratory. 12 The exceptions here are areas like genomics and proteomics, whose procedures have become highly standardized with a need to process high amounts of data. ELNs supposedly mimic the bound paper notebook, but again due to flexibility issues, anecdotal evidence suggests that such tools are not commonly used in academic laboratories. 12
These technological solutions themselves are limited by the basic assumptions used in their design about what constitutes good records and good record-keeping. The best practice principles that we developed and have described below can be used to guide the development of future generations of LIMS and ELNs. We believe that these principles transcend the mechanism for their implementation, whether they be paper-or computer-based.
We developed the best practices described below as an adjunct to our research-on-research-ethics project entitled “Scientific Record Keeping and Responsible Research Conduct” funded by a grant from the National Science Foundation (NSF). 5 We carried out the research to determine the actual record-keeping practices of active NSF- and NIH-funded scientists and how these influence the adjudication of research misconduct investigations. To help prepare a national survey of active NSF and NIH scientists on their record-keeping practices and to aid in the analysis of the responses, it was necessary to have a set of generally recognized best practices, that is, normative standards for research record-keeping. We derived these best practices from four sources: (1) the research training and ethics literature; (2) institutional policies and guidelines; (3) interviews with university officials responsible for investigating allegations of scientific misconduct; and (4) focus groups composed of faculty members from both medical and nonmedical faculties of Duke University, East Carolina University, North Carolina State University, and the University of North Carolina at Chapel Hill.
The research training and ethics literature was our primary source of record-keeping practices. The regular research literature (e.g., journal articles, monographs, conference proceedings, etc.) was not excluded from our review; however, this literature usually does not discuss research record-keeping practices. The ethics and research training literature is large and often included detailed and discipline specific recommendations for record-keeping. Many of the references for record-keeping practices were in textbooks or primers for students learning laboratory techniques for the first time. They usually focused on the proper construction of a research notebook and more, recently, its electronic counterparts. The monograph Writing the Laboratory Notebook by Kanare 10 is the acknowledged classic in this field. The practices described in this book and the other references are the practices that an individual researcher should follow when he or she is planning, collecting, recording, and analyzing research. We named these specific practices individual best practices. Only recently have references appeared that speak to higher-level best practices, such as those that research group leaders, departments and institutions could emulate to promote good record-keeping. Selected primary documents used in our compilation of our best practices are listed in the references at the end of this article. 2–4,9,10,14–25
Institutional policies and guidelines that deal with research records from many research universities were reviewed. Although our review was far from inclusive, we found that such policies and guidelines usually spoke to issues of retention, ownership, access and transfer of research records. Only rarely did a policy or guideline (e.g., those of the University of California, San Francisco) 18 set specific standards for record-keeping beyond these administrative issues. Sources that contained representative policies and guidelines that were useful are noted in the reference list. 18–25
Our research ethics project included interviews with university officials from 96 research universities. 5 These officials provided us with a number of positive practices for good record-keeping based on their experiences. In almost every case, however, these practices reiterate those that appeared in the literature. Thus, the university officials provided anecdotal evidence for the value of these practices.
As the number of specific best practices grew, they became too unwieldy to be a useful guide if compiled into a single list. An organizational structure was needed. We divided the best practices into three hierarchical groups: best practices for individual researchers, best practices for research group leaders, and best practices for departments and institutions. We also developed summaries—essentially statements of principles for the best practices for each group (see Lists 1 , ,2, 2 , and and3)—and 3 )—and compiled the detailed best practices separately. The resulting document, containing both principles and specific practices, was provided to the focus groups for their review and suggested revisions.
Best Practice Principles for Individual Researchers *