Embodied conversational agents (ECAs) are synthetic characters which can converse with the user (or with other ECAs) by some natural modalities of human-human communication. ECAs have become increasingly popular, as the promising new interface for several traditional computer applications or as the basis for applications in entirely new domains. Much research has been going on to endow ECAs with own behavioral, personality and emotional models, to enable ECAs to act more or less autonomously. The embodiment of the ECAs poses also challenging research tasks, to define which modalities to use to convey some information, how to generate good-quality output in the single modalities (speech, facial-, hand- and body gestures) and how to co-ordinate them. Often ECAs are meant to be involved in conversation with the user, when the maintenance of the discourse is an additional task. The proliferation of research issues, the diversity of prototypes for different application domains and the multitude of used paradigms and tools makes it difficult to evaluate and compare embodied agents.

In this paper we set out to provide a full checklist to compare ECAs, from four points of view: design, usability, practical usage and user perception. By listing all the factors which contribute to the ‘mind’ and ‘body’ aspects of an ECA, we hope to create a common ground to compare ECAs from a technical, design point of view. By identifying usability aspects and methods to measure those, ECAs could be compared from the point of view of usefulness. Thirdly, there are the aspects of how the agent is ‘subjectively experienced’ by the user. Finally there are aspects of practical applicability like cost etc. Ideally the outcome of dedicated evaluation experiments could serve as ‘design guidelines’ to define the ‘best’ ECA for a given purpose.