Qualification

This section allows you to analyze the rate of failed conversations by categorizing conversations, as well as the rate of misunderstandings by categorizing interactions. You will find several graphs here to help you observe the categorization of your conversations and interactions.

Conversation qualification

Conversation qualification

Conversations and their constituent interactions are classified into 3 categories based on their qualification. The 3 qualification categories are as follows:

• Only direct answers: the dialog consists only of direct questions / answers. This means that the bot has been able to understand all the questions and provide the answers;

• Ending with direct answer: during the dialog, the question or an incomprehension can be reworded, but the dialog ends with an answer;

• Failed: the dialog ends with an incomplete question or a suggestion for formulations and the user has not clicked on any of them.

The first two categories are considered successful dialogs. The evolution of the dialogs distribution over time reflects the training of the bot.

The qualification statistics page presents the distribution of conversations and interactions by qualification type. Clicking on any of the statistics displays details for that specific qualification type.

Qualification of conversations with provided contacts

During the creation of a knowledge entry, if the response includes an external contact provided by the bot, it's possible to specify the type of contact provided. This helps to understand the alternative solution offered to users based on their qualification. Therefore, it's possible to know, among the percentage of failures, what was proposed to users who did not initially receive their answer through the bot and were therefore redirected to another type of contact that could assist them.

Qualification of conversations based on the number of interactions

Qualification of conversations based on the number of interactions (in %)

Qualification of conversations by browser

Qualification of interactions

For the qualification of interactions, it is possible to obtain several details in the BMS: the type of interactions with the probability of the next interaction, the qualification of interactions based on sentence length, as well as the qualification according to the number of words per question.

Qualification by interactions is based on the analysis of all recorded interactions, except those from test or qualification conversations, which are systematically ignored.

Some interactions are also excluded from the calculation, especially those that correspond to knowledge ignored in statistics. All other interactions are then distributed into 14 categories, each corresponding to a specific type of event or processing:

• CHART_BUSINESS_MATCH (0)

• CHART_PROTOCOL_MATCH (1)

• CHART_DISABLED_KNOWLEDGE (2)

• CHART_REWORD (3)

• CHART_OUT_OF_SCOPE (4)

• CHART_GARBAGE (5)

• CHART_CLICKED_REWORD (6)

• CHART_NO_INTERACTION (7)

• CHART_OPERATOR (8)

• CHART_OTHERS (9)

• CHART_CLICKED_REDIRECTION_LINK (10)

• CHART_PUSH (11)

• CHART_CLICKED_REWORD_AUTO (12)

• CHART_CLICKED_JAVASCRIPT_LINK (13)

Only production interactions (excluding tests, qualifications, and ignored knowledge) are therefore taken into account and allocated to these categories, which can be viewed in the BMS statistics.

For the calculation of the failure rate in V1 statistics, only the category CHART_GARBAGE (5) is considered a failure. The other categories are not counted as failures.

Qualification of responses

Qualification of interactions based on sentence length

Qualification of interactions based on the number of words per question (%)