minuva/MiniLMv2-agentflow-v2

LLM agent flow classification

This model identifies common events and patterns within the conversation flow. Such events include an apology, where the LLM acknowledges a mistake. The flow labels can serve as foundational elements for sophisticated LLM analytics.

It is a fined-tuned version of MiniLMv2-L6-H384. The quantized version in ONNX format can be found here

This model is only for the LLM agent texts in the dialog. For the user texts use this model.

Load the Model

from transformers import pipeline

pipe = pipeline(model='minuva/MiniLMv2-agentflow-v2', task='text-classification')
pipe("thats my mistake")
# [{'label': 'agent_apology_error_mistake', 'score': 0.9965628981590271}]

Categories Explanation

Click to expand!

- OTHER: Responses or actions by the agent that do not fit into the predefined categories or are outside the scope of the specific interactions listed.

- agent_apology_error_mistake: When the agent acknowledges an error or mistake in the information provided or in the handling of the request.

- agent_apology_unsatisfactory: The agent expresses an apology for providing an unsatisfactory response or for any dissatisfaction experienced by the user.

- agent_didnt_understand: Indicates that the agent did not understand the user's request or question.

- agent_limited_capabilities: The agent communicates its limitations in addressing certain requests or providing certain types of information.

- agent_refuses_answer: When the agent explicitly refuses to answer a question or fulfill a request, due to policy restrictions or ethical considerations.

- image_limitations": The agent points out limitations related to handling or interpreting images.

- no_information_doesnt_know": The agent indicates that it has no information available or does not know the answer to the user's question.

- success_and_followup_assistance": The agent successfully provides the requested information or service and offers further assistance or follow-up actions if needed.

Metrics in our private test dataset

Model (params)	Loss	Accuracy	F1
minuva/MiniLMv2-agentflow-v2 (33M)	0.1540	0.9616	0.9618

Deployment

Check our llm-flow-classification repository for a FastAPI and ONNX based server to deploy this model on CPU devices.