File size: 3,202 Bytes
ba5180e 2e5e32e ba5180e ceb8492 ba5180e ceb8492 1ec392d ceb8492 1ec392d ceb8492 1ec392d ceb8492 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 |
---
base_model: nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large
tags:
- generated_from_trainer
- text-classification
- multi-class-classification
metrics:
- accuracy
- f1
model-index:
- name: MiniLMv2-L6-H384-distilled-from-RoBERTa-Large-agentflow-distil
results: []
license: apache-2.0
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# LLM agent flow classification
This model identifies common events and patterns within the conversation flow.
Such events include an apology, where the LLM acknowledges a mistake.
The flow labels can serve as foundational elements for sophisticated LLM analytics.
It is a fined-tuned version of [MiniLMv2-L6-H384](https://huggingface.co/nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large).
The quantized version in ONNX format can be found [here](https://huggingface.co/minuva/MiniLMv2-agentflow-v2-onnx)
This model is *only* for the LLM agent texts in the dialog. For the user texts [use this model](https://huggingface.co/minuva/MiniLMv2-userflow-v2).
# Load the Model
```py
from transformers import pipeline
pipe = pipeline(model='minuva/MiniLMv2-agentflow-v2', task='text-classification')
pipe("thats my mistake")
# [{'label': 'agent_apology_error_mistake', 'score': 0.9965628981590271}]
```
# Categories Explanation
<details>
<summary>Click to expand!</summary>
- OTHER: Responses or actions by the agent that do not fit into the predefined categories or are outside the scope of the specific interactions listed.
- agent_apology_error_mistake: When the agent acknowledges an error or mistake in the information provided or in the handling of the request.
- agent_apology_unsatisfactory: The agent expresses an apology for providing an unsatisfactory response or for any dissatisfaction experienced by the user.
- agent_didnt_understand: Indicates that the agent did not understand the user's request or question.
- agent_limited_capabilities: The agent communicates its limitations in addressing certain requests or providing certain types of information.
- agent_refuses_answer: When the agent explicitly refuses to answer a question or fulfill a request, due to policy restrictions or ethical considerations.
- image_limitations": The agent points out limitations related to handling or interpreting images.
- no_information_doesnt_know": The agent indicates that it has no information available or does not know the answer to the user's question.
- success_and_followup_assistance": The agent successfully provides the requested information or service and offers further assistance or follow-up actions if needed.
</details>
<br>
# Metrics in our private test dataset
| Model (params) | Loss | Accuracy | F1 |
|--------------------|-------------|----------|--------|
| minuva/MiniLMv2-agentflow-v2 (33M) | 0.1540 | 0.9616 | 0.9618 |
# Deployment
Check our [llm-flow-classification repository](https://github.com/minuva/llm-flow-classification) for a FastAPI and ONNX based server to deploy this model on CPU devices.
|