Ejento AI
Guides
QuickstartRecipesREST APIsRelease NotesFAQs
Guides
QuickstartRecipesREST APIsRelease NotesFAQs
Ejento AI
  1. Guides
  • Basic Operations
    • Features
      • Organization → Projects → Assistants → Teams Hierarchy
    • Guides
      • Login/Signup
  • Assistants
    • Overview
    • Features
      • Assistant Access Control
      • Caching Responses for Assistants
      • Assistant Evaluation
      • Evaluation Metrics
      • URL-based Chat Thread Creation and Prepopulation
      • Reasoning Patterns
    • Guides
      • Add Assistant
      • Evaluate Assistant
      • Edit Assistant
      • Assistant Edit Access
      • Embed Assistant
      • Delete Assistant
      • Add Favourite Assistants
      • View Assistant Id
      • View Dataset Id
      • Voice Calling with Assistants
  • Corpus
    • Overview
    • Features
      • Corpus Permissions
      • PII Redaction
      • ETag Setup for Corpus Incremental Refresh
    • Guides
      • Assistant Corpus Setup
      • Assistant Corpus Settings
      • Corpus Access Control
      • Corpus Connections
      • View Corpus Id
      • View Document Id
      • Tagging
        • Corpus tagging
        • Document tagging
  • Teams
    • Overview
    • Guides
      • Add a Team
      • Edit a Team
      • Delete a Team
      • View Team Id
  • Projects
    • Overview
    • Guides
      • Add a Project
      • Edit a Project
      • Managing Assistants in a Project
      • Delete a Project
      • View Project Id
  • User Settings
    • Overview
    • Features
      • Ejento AI User Access Levels
    • Guides
      • Add new user
      • View my User Id
  • API Keys
    • Overview
    • Guides
      • How to generate API Key and Auth Token
  • Workflows
    • Overview
    • Guides
      • Add Workflow
      • Workflow Chat
  • Tools
    • Overview
    • Guides
      • Tools Overview
      • Create External Tool
      • Connect Tool to Assistant
  • Analytics
    • Overview
    • Guides
      • Analyzing Data in the Analytics Dashboard
  • Chatlogs
    • Overview
    • Guides
      • Managing Chatlogs
      • View Chatlog & Chat thread Id
  • Integrations
    • Overview
    • Guides
      • Email Indexing
      • Microsoft Teams
      • Sharepoint Indexing
      • MS Teams Integration Setup
      • Creating a Connection in Credential Manager
      • Slack App
      • Discord Bot
  • Ejento AI Shield
    • Overview
    • Features
      • Understanding Guardrails
    • Guides
      • How to enable Guardrails
  • Assistant Security
    • Overview
    • Features
      • Assistant Red Teaming
    • Guides
      • Red Team an Assistant
  1. Guides

Evaluate Assistant

1. For the Assistant you want to evaluate, click on the three dots next to it.#

Step 1 screenshot

2. Click on Evaluate to open the evaluation tab#

Step 2 screenshot

3. Here you will be able to see the list of datasets#

Step 3 screenshot

4. Lets create a dataset by clicking on Create Dataset#

Step 4 screenshot

5. Type a Name for the dataset#

Step 5 screenshot

6. Add a description for the dataset#

Step 6 screenshot

7. From the list of queries, select relevant queries for the dataset#

Step 7 screenshot

8. Select by clicking the checkboxes next to queries#

Step 8 screenshot

9. Click on Save Changes#

Step 9 screenshot

10. The datasets table shows the newly added dataset now#

Step 10 screenshot

11. Lets create another dataset following the same steps#

Step 11 screenshot

12. Select queries for dataset#

Step 12 screenshot
Step 13 screenshot
Step 14 screenshot

15. Type a Name for the dataset#

Step 15 screenshot

16. Add the description for the dataset#

Step 16 screenshot

17. Click on Save Changes#

Step 17 screenshot

18. Here we have two datasets for running our evaluation test now#

Step 18 screenshot

19. Select a dataset from the table for the evaluation test#

Step 19 screenshot

20. You can also select multiple datasets at the same time for a bulk evaluation#

Step 20 screenshot

21. Lets start the evaluation for a single dataset by clicking on Start Evaluation#

Step 21 screenshot

22. Click on Confirm#

Step 22 screenshot

23. Go to the Test Runs Tab#

Step 23 screenshot

24. The evaluation has started with the Status "Pending"#

Step 24 screenshot

25. The status will soon update to "Running"#

Step 25 screenshot

26. Click on the Refresh Table button after a while to check for results#

Step 26 screenshot

27. Finally, the evaluation scores will be available#

Step 27 screenshot
Previous
Add Assistant
Next
Edit Assistant