AIS Transactions on Replication Research

Policies

The Editorial Philosophy of TRR

TRR publishes replications of information systems articles published in other journals and conferences. TRR will also consider publishing replications of research reported by industry-focused analyst firms, if the topic is within the bounds of information system research. It does not publish original research. Articles are therefore short and the review process primarily focuses on the contribution of the replication, rather than on the theoretical contribution of the replication or the original article.

Not all prior research needs additional replication. Some research questions have received sufficient empirical research so that a scientific consensus has been reached (e.g., many aspects related to the Technology Acceptance Model). TRR is particularly interested in replications of articles that have had significant impact (as measured, for example, by citations or awards) but little empirical replication. We should also note that one or even two replications are probably not sufficient to produce a scientific consensus.

TRR welcomes papers from conferences. During the submission, authors must indicate that an earlier version was presented at a conference and highlight how the paper has developed in light of feedback from the conference presentation.

The Peer Review Process

The review process is designed to support the editorial philosophy. Because TRR focuses on replication, the review is ordinarily conducted by one Senior Editor (SE) and one or two experienced reviewers. One reviewer may be an author of the original study. The reviewers do not know the identity of the authors; the authors do not know the identities of the reviewers. The authors know the identity of the SE and vice-versa.

Every effort will be made to ensure that the review cycle is as short and fast as possible without compromising quality. Reviews should be completed within one month, with editorial decisions being completed in less than two additional weeks. Articles should have no more than one round of revision prior to the publication decision. Authors are expected to complete revisions within three months.

The Role of Advisors

Members of the advisory board are appointed by the EICs. The appointments are normally for three years. The primary role of an advisory board member is to provide advice on an as needed basis. Advisors help with setting policies for the journal, proposing editorial board members, and actively marketing the journal.

The Role of Senior Editors

(Adapted from The SE Role at MISQ by Detmar Straub, http://misq.org. Used with permission.)

SEs are appointed by the EICs. SEs play an important role at TRR in handling submissions in a timely and quality manner, as well as helping to set policies and market the journal and replication research in general.

SEs are final decision makers who conduct the regular review process on submitted manuscripts that are assigned to them. SEs have two goals. (1) To be the advocate and guardian of promising papers that may otherwise be declined by reviewers; there is no perfect research. (2) To make sure that non-promising papers do not go too deep in the review process before being declined. Non-promising manuscripts should be declined right away without wasting reviewers’ and authors’ time.

If a manuscript is sent out for review, this implies that the SE sees substantial promise in the manuscript. The SE should clearly signal this to the reviewers. Language such as “I like the paper for these reasons…” can be used. Yet reviewers are requested to make independent judgments, share with the SE any fatal flaws they see in the paper, and make constructive suggestions on how to fix flaws in the paper. The SE should maintain continued communication (formally or informally) with the reviewers to ensure their reviews are on time and adhere to the spirit of being constructive.

Once the reviewers’ reports and recommendations are in, the SE makes an independent decision after his/her own careful evaluation and taking into account the reviewers' comments and recommendations. In other words, the SE should not be completely influenced by the reviewers, or simply take a “majority vote” on the reviewers' recommendations. This is especially important where the reviewers are split or negative about a good paper. In such cases, the SE should be more inclined to overrule the reviewers if s/he believes there is true potential in the paper. The handling SE will be recognized when an article is published.

SEs participate in setting policies for the journal, proposing editorial board members, nominating best papers, attending the journal's board meetings, and actively marketing the journal whenever or wherever.

The Role of Reviewers

(Adapted from The Reviewer Role at MISQ by Detmar Straub, http://misq.org. Used with permission.)

The reviewer works in cooperation with the SE, who incorporates reviewers' assessments to make the final decision. A reviewer's role is to make an independent judgment by carefully reading the paper to determine its strengths and weaknesses.

The primary review criteria for assessing a replication article are:

The extent to which the submitted manuscript faithfully assesses the research questions and/or hypotheses proposed by the original article. Any of the three types of replications – exact, methodological, or conceptual – are valuable (see Information for Authors for more details). We especially encourage conceptual replications whose measures, treatments, analyses etc. differ from the original article. Nonsignificant results are quite appropriate when the analyses include a power analysis.
The extent to which additional replication of the original article is needed. Articles that have received many replications may not need additional replications. In most cases, more than a few replications will be needed.

Reviewers should have a positive and constructive spirit in reviewing a paper. Reviewers would not have been asked to review a paper unless the SE saw promise in the paper. Hence, reviewers should provide detailed comments and reasonable suggestions for improvement.

Reviewers are recognized annually for their contributions in the final issue of that annual volume. They will also be eligible for the nomination of the annual best reviewer award at TRR. Outstanding reviewers will be invited to handle special submissions or to the SE board.

Ethical Responsibilities for SEs and Reviewers

(Adapted from The SE/AE/Reviewer Role at MISQ by Detmar Straub, http://misq.org. Used with permission.)

Submissions and all other review documents to TRR are solely for the purpose of evaluation. An SE, editorial board member or reviewer may not share them with any other parties for other purposes. The manuscript under review is not citable, and its contents remain the intellectual property of the authors until such time as it is accepted for publication and the authors sign a copyright transfer to AIS.

Accepting an SE/reviewer position also indicates an acceptance of conditions in relation to suspected or alleged instances of unethical behavior related to TRR that are either identified by or brought to the attention of the SE/editorial board member/reviewer. Specifically, the conditions are that the SE/editorial board member/reviewer (i) will keep the incident confidential unless otherwise advised by the EICs, (ii) will report the incident to the EICs as soon as possible, and (iii) will take no other actions himself or herself unless otherwise advised by the EICs.

Archiving and Subscription

All TRR published articles are available on the AIS electronic library.

Openness

TRR is a signatory of the Transparency and Openness Promotion (TOP) guidelines of the Center for Open Science (COS).