Recent years have seen increased interest in text summarization with emphasis on evaluation of prototype ststems. Many factors can affect the design of such evaluations, requiring choices among competing alternatives. This paper examines several major themes running through three major evaluations: SUMMAC, NTCIR and respectively DUC. The themes are extrinsic/intrinsic evaluation, evaluation procedures and methods, generic/focused summaries, single- and multi-document summaries, length and compression, extracts versus abstracts, and issues with genre.
Special issue of Information Processing and Management on Text Summarization