Tools
Audit
Audits allow you to manually measure, on a sample of conversations, the relevance of the answers given by your bot by evaluating the quality of the matching. As a result of your audit you obtain an index which is in fact a point of comparison.
Performing this regularly allows you to measure the bot's progress through your knowledge base improvement process.
For example, the audit will allow the bot's manager to confirm or not the answers given by the bot, to validate that the bot has chosen the right knowledge and thus to detect false positives.
We recommend carrying out an audit on average once a quarter. To carry out an audit:
Go to the Contents > Tools > Audit
Click on the "Add" button to create a new audit
Select:
the sample size of conversations to be measured
the period of your audit
if necessary, a specific viewing area
A summary line of your audit is then available:
Period: corresponds to the start date of the audit period;
Creator: corresponds to the name of the audit creator;
Consultation space: corresponds to the viewing area you have selected;
Size: the size of the sample you have selected;
Direct answers: the chatbot has directly answered the user's question;
Reword: the chatbot was not sure it understood the user's question, so it proposed 1 to 3 questions that could correspond to the user's question;
Misunderstood sentences: the chatbot did not understand the user's question;
Completed: this is the progress of your audit (when you start an audit, it is already partly completed because there are duplicate questions and these are already grouped together;
Note: depending on the results of the audit, a score between 0 and 100% is given;
Incorrect answers: this part will be filled in when you have finished the audit. This is the number of incorrect answers defined through the audit.
Hover over the audit line. You can modify or delete it. To start your audit click on the Edit button.
Three filters for your audit are then displayed:
Action state
To correct
Summary
Action state
In this section, you will find all the interactions in your sample. As you process them, the interactions will be organised according to the classification you give them.
Not yet processed
These are interactions that you have not yet checked. The check to be made will depend on the matching of the interaction:
Direct answer: check that the answers provided by the chatbot are correct. You have the question asked by the user and the knowledge used by the chatbot to provide the answer. You then have to decide if
the knowledge used by the chatbot was the right knowledge, choose: OK
it should have matched with another knowledge, choose: Another knowledge
the user's question does not fit in the chatbot's scope, choose: Out of scope
Reword: check whether the rewords proposed by the chatbot are relevant or not. You have to decide if:
the reword proposal was necessary, choose: OK
one of the proposals was the right one, choose: MatchOneOfRewordedKnowledges
none of the proposals correspond to the user's question, choose: Another knowledge
the user's question contained superfluous information, choose: LongSentenceWithNoise
the user's question contained several questions, choose: LongSentenceWithMultipleKnowledge
the user's question was outside the scope of the sculpin, choose: OutOfScope
Misunderstood sentence: check whether a sentence that was misunderstood by the chatbot could have been understood. You have to choose if:
you validate that it is an incomprehensible sentence by the chatbot and to which there is no knowledge, choose: Ok
the user's question contained superfluous information, choose: LongSentenceWithNoise
the user's question contained several questions, choose: LongSentenceWithMultipleKnowledge
If the user's question did not fit into the scope of the scab, choose: OutOfScope
Note: For each interaction to be processed, you can click on Show dialog to place the user's question in the context of the conversation.
Once the interactions are processed, they are organised according to the classification choice that was made. You can cancel the classification.
To correct
In this section, you will find all the interactions of your sample to be corrected. These are organised according to the classification choice made in the "Action status" step.
For each interaction, you will see :
the user's question
the knowledge that the chatbot provided as an answer
the consultation space that was queried
When you click in the field Choose correct knowledge, the list of knowledge close to the user's question is displayed. Then click on the correct knowledge to which to associate this question and click on Apply correction. Clicking this button will add the user's question as the knowledge formulation.
Alternatively, you can choose to ignore the correction or cancel the classification initially made.
Summary
Once the audit is complete, click on the Summary tab to access the audit summary tables.
The first table shows you the status of your audit.
The second table gives you a numerical summary of your Audit and a final score.
When you have performed several audits, a graph will appear on the Contents > Tools > Audits page.
This graph shows the evolution of the relevance of the answers given according to the different audits carried out.
Here is the legend:
Green: proportion of direct answers
Black: proportion of rewords
Orange: proportion of misunderstood sentences
Blue: note
Red: proportion of incorrect answers
Test dialogs
When you have quite complex dialog paths, you can save a dialog that went well in the test dialogs as a template (from the Dialogs page) Go to Learning > Dialogs
Click on a dialog
Click the "copy dialog" icon when you are on a dialog you want to save.
Click Copy.
Then return to the Test dialogs menu where you can replay the dialogs to make sure the bot is still responding as expected. Go to Content > Tools > Test dialogs
Find more information about dialogs on this page.
Close knowledge
This menu lets you locate the knowledge items with similar question formulations. For example, it makes it possible to see if two knowledge items could be gathered in one.
This menu also allows you to view the latest changes that were made and by whom they were made, over the selected period, under a different view than the History menu.
This allows you to be able to directly make changes if you notice an error while editing This can be particularly useful for a proofreader when he has to validate the last changes made on the database.
Go to Content > Tools > Close knowledge
History
The history allows you to view all the changes that have been made to the knowledge base. You can filter these changes by action type and by user.
Last updated
