Tools

Audit

Audits allow you to manually measure, on a sample of conversations, the relevance of the answers given by your bot by evaluating the quality of the matching. As a result of your audit you obtain an index which is in fact a point of comparison.

Performing this regularly allows you to measure the bot's progress through your knowledge base improvement process.

For example, the audit will allow the bot's manager to confirm or not the answers given by the bot, to validate that the bot has chosen the right knowledge and thus to detect false positives.

We recommend carrying out an audit on average once a quarter. To carry out an audit:

1. Go to the Contents > Tools > Audit

2. Click on the "Add" button to create a new audit

3. Select:

   • the sample size of conversations to be measured

   • the period of your audit

   • if necessary, a specific viewing area

A summary line of your audit is then available:

• Period: corresponds to the start date of the audit period;

• Creator: corresponds to the name of the audit creator;

• Consultation space: corresponds to the viewing area you have selected;

• Size: the size of the sample you have selected;

• Direct answers: the chatbot has directly answered the user's question;

• Reword: the chatbot was not sure it understood the user's question, so it proposed 1 to 3 questions that could correspond to the user's question;

• Misunderstood sentences: the chatbot did not understand the user's question;

• Completed: this is the progress of your audit (when you start an audit, it is already partly completed because there are duplicate questions and these are already grouped together;

• Note: depending on the results of the audit, a score between 0 and 100% is given;

• Incorrect answers: this part will be filled in when you have finished the audit. This is the number of incorrect answers defined through the audit.

Hover over the audit line. You can modify or delete it. To start your audit click on the Edit button.

Three filters for your audit are then displayed:

• Action state

• To correct

• Summary

Action state​

In this section, you will find all the interactions in your sample. As you process them, the interactions will be organised according to the classification you give them.

Not yet processed

These are interactions that you have not yet checked. The check to be made will depend on the matching of the interaction:

• Direct answer: check that the answers provided by the chatbot are correct. You have the question asked by the user and the knowledge used by the chatbot to provide the answer. You then have to decide if

  • the knowledge used by the chatbot was the right knowledge, choose: OK

  • it should have matched with another knowledge, choose: Another knowledge

  • the user's question does not fit in the chatbot's scope, choose: Out of scope

• Reword: check whether the rewords proposed by the chatbot are relevant or not. You have to decide if:

  • the reword proposal was necessary, choose: OK

  • one of the proposals was the right one, choose: MatchOneOfRewordedKnowledges

  • none of the proposals correspond to the user's question, choose: Another knowledge

  • the user's question contained superfluous information, choose: LongSentenceWithNoise

  • the user's question contained several questions, choose: LongSentenceWithMultipleKnowledge

  • the user's question was outside the scope of the sculpin, choose: OutOfScope

• Misunderstood sentence: check whether a sentence that was misunderstood by the chatbot could have been understood. You have to choose if:

  • you validate that it is an incomprehensible sentence by the chatbot and to which there is no knowledge, choose: Ok

  • the user's question contained superfluous information, choose: LongSentenceWithNoise

  • the user's question contained several questions, choose: LongSentenceWithMultipleKnowledge

  • If the user's question did not fit into the scope of the scab, choose: OutOfScope

Note: For each interaction to be processed, you can click on Show dialog to place the user's question in the context of the conversation.

Once the interactions are processed, they are organised according to the classification choice that was made. You can cancel the classification.

To correct​

In this section, you will find all the interactions of your sample to be corrected. These are organised according to the classification choice made in the "Action status" step.

For each interaction, you will see :

• the user's question

• the knowledge that the chatbot provided as an answer

• the consultation space that was queried

When you click in the field Choose correct knowledge, the list of knowledge close to the user's question is displayed. Then click on the correct knowledge to which to associate this question and click on Apply correction. Clicking this button will add the user's question as the knowledge formulation.

Alternatively, you can choose to ignore the correction or cancel the classification initially made.

Summary​

Once the audit is complete, click on the Summary tab to access the audit summary tables.

The first table shows you the status of your audit.

The second table gives you a numerical summary of your Audit and a final score.

When you have performed several audits, a graph will appear on the Contents > Tools > Audits page.

This graph shows the evolution of the relevance of the answers given according to the different audits carried out.

Here is the legend:

• Green: proportion of direct answers

• Black: proportion of rewords

• Orange: proportion of misunderstood sentences

• Blue: note

• Red: proportion of incorrect answers

Test dialogs​

When you have quite complex dialog paths, you can save a dialog that went well in the test dialogs as a template (from the Dialogs page) Go to Learning > Dialogs

1. Click on a dialog

2. Click the "copy dialog" icon when you are on a dialog you want to save.

3. Click Copy.

1. Then return to the Test dialogs menu where you can replay the dialogs to make sure the bot is still responding as expected. Go to Content > Tools > Test dialogs

Find more information about dialogs on this page.

Close knowledge​

This menu lets you locate the knowledge items with similar question formulations. For example, it makes it possible to see if two knowledge items could be gathered in one.

This menu also allows you to view the latest changes that were made and by whom they were made, over the selected period, under a different view than the History menu.

This allows you to be able to directly make changes if you notice an error while editing This can be particularly useful for a proofreader when he has to validate the last changes made on the database.

Go to Content > Tools > Close knowledge

History​

The history allows you to view all the changes that have been made to the knowledge base. You can filter these changes by action type and by user.