The evaluation of information systems development methodologies is becoming increasingly important. The lack of a commonly acceptable set of criteria for evaluation, however, hinders aggregating knowledge from field studies. Our study attempts to fill the gap. To reduce the effects of subjectivity, we surveyed the opinions of a group (28) of experienced IS researchers. Fifty-one criteria were generated from the research participants. A systematic content analysis technique will be applied for grouping the unique criteria into general categories. Upon completion, our study can make both academic and practical contributions in evaluating ISD methodologies.