Hugging Face's logo

sarahahtee
/
t5-dialogue-classification-4

Text2Text Generation
Transformers
TensorBoard
Safetensors
t5
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
Edit model card

t5-dialogue-classification-4

This model is a fine-tuned version of google/flan-t5-small on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5317

  • Classification Report: precision recall f1-score support

    ALLERGY     0.6667    0.5000    0.5714         4

    ASSESSMENT 0.0000 0.0000 0.0000 4 Afib 0.0000 0.0000 0.0000 0 CC 0.0000 0.0000 0.0000 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8462 1.0000 0.9167 22 GENHX 0.7917 0.9500 0.8636 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1

IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.8333 0.7143 0.7692 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.4444 1.0000 0.6154 4 PASTSURGICAL 0.7273 1.0000 0.8421 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.7500 0.2727 0.4000 11

 accuracy                         0.6400       100
macro avg     0.2647    0.3065    0.2688       100

weighted avg 0.5929 0.6400 0.5937 100

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 7
  • eval_batch_size: 7
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss Classification Report
No log 1.0 172 0.7096 precision recall f1-score support 
  ALLERGY     0.3333    0.2500    0.2857         4

ASSESSMENT 0.0000 0.0000 0.0000 4 Afib 0.0000 0.0000 0.0000 0 CC 0.0000 0.0000 0.0000 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.0000 0.0000 0.0000 1 FAM/SOCHX 0.7222 0.5909 0.6500 22 GENHX 0.3846 1.0000 0.5556 20 GYNHX 0.0000 0.0000 0.0000 1 I T 0.0000 0.0000 0.0000 0 I V 0.0000 0.0000 0.0000 0 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.4444 0.5714 0.5000 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.2000 0.2500 0.2222 4 PASTSURGICAL 0.8000 0.5000 0.6154 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 Psychiatric 0.0000 0.0000 0.0000 0 ROS 0.0000 0.0000 0.0000 11 SPECIALHX 0.0000 0.0000 0.0000 0

 accuracy                         0.4300       100
macro avg     0.1154    0.1265    0.1132       100

weighted avg 0.3523 0.4300 0.3587 100 | | No log | 2.0 | 344 | 0.5308 | precision recall f1-score support

         ALLERGY     0.5000    0.2500    0.3333         4
      ASSESSMENT     0.0000    0.0000    0.0000         4
            Afib     0.0000    0.0000    0.0000         0
              CC     0.0000    0.0000    0.0000         4
       DIAGNOSIS     0.0000    0.0000    0.0000         1
     DISPOSITION     0.0909    0.5000    0.1538         2
        EDCOURSE     0.0000    0.0000    0.0000         3
            EXAM     0.0000    0.0000    0.0000         1
       FAM/SOCHX     0.8750    0.9545    0.9130        22
           GENHX     0.6129    0.9500    0.7451        20
           GYNHX     0.0000    0.0000    0.0000         1

How are you feeling? 0.0000 0.0000 0.0000 0 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.6250 0.7143 0.6667 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.2727 0.7500 0.4000 4 PASTSURGICAL 0.7273 1.0000 0.8421 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.0000 0.0000 0.0000 11

        accuracy                         0.5800       100
       macro avg     0.1684    0.2327    0.1843       100
    weighted avg     0.4497    0.5800    0.4963       100
                                   |

| 0.6083 | 3.0 | 516 | 0.5336 | precision recall f1-score support

  ALLERGY     0.5000    0.2500    0.3333         4

ASSESSMENT 0.0000 0.0000 0.0000 4 CC 0.0000 0.0000 0.0000 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.0000 0.0000 0.0000 1 FAM/SOCHX 0.8462 1.0000 0.9167 22 GENHX 0.7600 0.9500 0.8444 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.5714 0.5714 0.5714 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.3636 1.0000 0.5333 4 PASTSURGICAL 0.8000 1.0000 0.8889 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 1.0000 0.0909 0.1667 11

 accuracy                         0.5900       100
macro avg     0.2421    0.2431    0.2127       100

weighted avg 0.5867 0.5900 0.5347 100 | | 0.6083 | 4.0 | 688 | 0.5260 | precision recall f1-score support

             ALLERGY     1.0000    0.2500    0.4000         4
          ASSESSMENT     0.0000    0.0000    0.0000         4
                  CC     0.0000    0.0000    0.0000         4
           DIAGNOSIS     0.0000    0.0000    0.0000         1
         DISPOSITION     0.0000    0.0000    0.0000         2
            EDCOURSE     0.0000    0.0000    0.0000         3
                EXAM     0.0000    0.0000    0.0000         1
           FAM/SOCHX     0.8148    1.0000    0.8980        22
               GENHX     0.6552    0.9500    0.7755        20
               GYNHX     0.0000    0.0000    0.0000         1
             IMAGING     0.0000    0.0000    0.0000         1
       IMMUNIZATIONS     0.0000    0.0000    0.0000         1
                LABS     0.0000    0.0000    0.0000         1
         MEDICATIONS     0.5714    0.5714    0.5714         7
       OTHER_HISTORY     0.0000    0.0000    0.0000         1
       PASTMEDICALHX     0.3636    1.0000    0.5333         4
        PASTSURGICAL     0.7273    1.0000    0.8421         8
                PLAN     0.0000    0.0000    0.0000         3
          PROCEDURES     0.0000    0.0000    0.0000         1
                 ROS     1.0000    0.1818    0.3077        11

What is the DISPOSITION? 0.0000 0.0000 0.0000 0

            accuracy                         0.6000       100
           macro avg     0.2444    0.2359    0.2061       100
        weighted avg     0.5730    0.6000    0.5312       100

| | 0.6083 | 5.0 | 860 | 0.4168 | precision recall f1-score support

         ALLERGY     0.6667    0.5000    0.5714         4
      ASSESSMENT     0.0000    0.0000    0.0000         4
              CC     0.0000    0.0000    0.0000         4
       DIAGNOSIS     0.0000    0.0000    0.0000         1
     DISPOSITION     0.0000    0.0000    0.0000         2
        EDCOURSE     0.0000    0.0000    0.0000         3
            EXAM     0.5000    1.0000    0.6667         1
       FAM/SOCHX     0.8800    1.0000    0.9362        22
           GENHX     0.7600    0.9500    0.8444        20
           GYNHX     0.0000    0.0000    0.0000         1

How are you feeling? 0.0000 0.0000 0.0000 0 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.7143 0.7143 0.7143 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.3000 0.7500 0.4286 4 PASTSURGICAL 1.0000 1.0000 1.0000 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.7500 0.2727 0.4000 11

        accuracy                         0.6300       100
       macro avg     0.2653    0.2946    0.2648       100
    weighted avg     0.6018    0.6300    0.5955       100
                                                                                                 |

| 0.163 | 6.0 | 1032 | 0.4511 | precision recall f1-score support

         ALLERGY     1.0000    0.5000    0.6667         4
      ASSESSMENT     0.0000    0.0000    0.0000         4
            Afib     0.0000    0.0000    0.0000         0
              CC     0.0000    0.0000    0.0000         4
       DIAGNOSIS     0.0000    0.0000    0.0000         1
     DISPOSITION     0.0000    0.0000    0.0000         2
        EDCOURSE     0.0000    0.0000    0.0000         3
            EXAM     0.5000    1.0000    0.6667         1
       FAM/SOCHX     0.8800    1.0000    0.9362        22
           GENHX     0.7500    0.9000    0.8182        20
           GYNHX     0.0000    0.0000    0.0000         1

How are you feeling? 0.0000 0.0000 0.0000 0 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.8000 0.5714 0.6667 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.3333 1.0000 0.5000 4 PASTSURGICAL 0.6667 1.0000 0.8000 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 1.0000 0.2727 0.4286 11

        accuracy                         0.6200       100
       macro avg     0.2695    0.2838    0.2492       100
    weighted avg     0.6213    0.6200    0.5807       100
                                   |

| 0.163 | 7.0 | 1204 | 0.3896 | precision recall f1-score support

  ALLERGY     0.6667    0.5000    0.5714         4

ASSESSMENT 0.0000 0.0000 0.0000 4 CC 0.3333 0.2500 0.2857 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8800 1.0000 0.9362 22 GENHX 0.7200 0.9000 0.8000 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.6667 0.5714 0.6154 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.2857 1.0000 0.4444 4 PASTSURGICAL 1.0000 1.0000 1.0000 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.7500 0.2727 0.4000 11

 accuracy                         0.6300       100
macro avg     0.2901    0.3247    0.2860       100

weighted avg 0.6032 0.6300 0.5918 100 | | 0.163 | 8.0 | 1376 | 0.4200 | precision recall f1-score support

  ALLERGY     1.0000    0.2500    0.4000         4

ASSESSMENT 0.0000 0.0000 0.0000 4 CC 0.4286 0.7500 0.5455 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.9167 1.0000 0.9565 22 GENHX 0.6667 0.9000 0.7660 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.5556 0.7143 0.6250 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.4286 0.7500 0.5455 4 PASTSURGICAL 1.0000 1.0000 1.0000 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.7500 0.2727 0.4000 11

 accuracy                         0.6400       100
macro avg     0.3123    0.3319    0.2953       100

weighted avg 0.6157 0.6400 0.5977 100 | | 0.1057 | 9.0 | 1548 | 0.4642 | precision recall f1-score support

  ALLERGY     0.6667    0.5000    0.5714         4

ASSESSMENT 0.0000 0.0000 0.0000 4 Afib 0.0000 0.0000 0.0000 0 CC 0.2500 0.2500 0.2500 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8800 1.0000 0.9362 22 GENHX 0.7826 0.9000 0.8372 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.8000 0.5714 0.6667 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.3636 1.0000 0.5333 4 PASTSURGICAL 0.8000 1.0000 0.8889 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.7500 0.2727 0.4000 11

 accuracy                         0.6300       100
macro avg     0.2759    0.3092    0.2738       100

weighted avg 0.6088 0.6300 0.5960 100 | | 0.1057 | 10.0 | 1720 | 0.4837 | precision recall f1-score support

  ALLERGY     1.0000    0.2500    0.4000         4

ASSESSMENT 0.0000 0.0000 0.0000 4 Afib 0.0000 0.0000 0.0000 0 CC 0.0000 0.0000 0.0000 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8462 1.0000 0.9167 22 GENHX 0.6429 0.9000 0.7500 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.6667 0.5714 0.6154 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.4000 1.0000 0.5714 4 PASTSURGICAL 0.8000 1.0000 0.8889 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.7500 0.2727 0.4000 11

 accuracy                         0.6100       100
macro avg     0.2669    0.2854    0.2480       100

weighted avg 0.5689 0.6100 0.5554 100 | | 0.1057 | 11.0 | 1892 | 0.4834 | precision recall f1-score support

  ALLERGY     0.5000    0.2500    0.3333         4

ASSESSMENT 0.0000 0.0000 0.0000 4 Afib 0.0000 0.0000 0.0000 0 CC 0.3333 0.5000 0.4000 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.9167 1.0000 0.9565 22 GENHX 0.7826 0.9000 0.8372 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.6667 0.5714 0.6154 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.3000 0.7500 0.4286 4 PASTSURGICAL 0.7273 1.0000 0.8421 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.7500 0.2727 0.4000 11

 accuracy                         0.6200       100
macro avg     0.2608    0.2973    0.2609       100

weighted avg 0.5959 0.6200 0.5855 100 | | 0.0688 | 12.0 | 2064 | 0.5085 | precision recall f1-score support

  ALLERGY     1.0000    0.2500    0.4000         4

ASSESSMENT 0.0000 0.0000 0.0000 4 CC 0.4000 0.5000 0.4444 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 1.0000 1.0000 1.0000 1 FAM/SOCHX 0.8800 1.0000 0.9362 22 GENHX 0.7391 0.8500 0.7907 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.5714 0.5714 0.5714 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.2857 1.0000 0.4444 4 PASTSURGICAL 0.8000 1.0000 0.8889 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.7500 0.2727 0.4000 11

 accuracy                         0.6200       100
macro avg     0.3213    0.3222    0.2938       100

weighted avg 0.6054 0.6200 0.5808 100 | | 0.0688 | 13.0 | 2236 | 0.5189 | precision recall f1-score support

  ALLERGY     1.0000    0.2500    0.4000         4

ASSESSMENT 0.0000 0.0000 0.0000 4 CC 0.4000 0.5000 0.4444 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8800 1.0000 0.9362 22 GENHX 0.7083 0.8500 0.7727 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.3333 1.0000 0.5000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.5714 0.5714 0.5714 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.3636 1.0000 0.5333 4 PASTSURGICAL 0.6154 1.0000 0.7619 8 PLAN 1.0000 0.3333 0.5000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.7500 0.2727 0.4000 11

 accuracy                         0.6400       100
macro avg     0.3561    0.3889    0.3243       100

weighted avg 0.6159 0.6400 0.5872 100 | | 0.0688 | 14.0 | 2408 | 0.5504 | precision recall f1-score support

  ALLERGY     1.0000    0.5000    0.6667         4

ASSESSMENT 0.0000 0.0000 0.0000 4 Afib 0.0000 0.0000 0.0000 0 CC 0.6667 0.5000 0.5714 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8800 1.0000 0.9362 22 GENHX 0.6897 1.0000 0.8163 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.8000 0.5714 0.6667 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.3000 0.7500 0.4286 4 PASTSURGICAL 0.7273 1.0000 0.8421 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.6667 0.1818 0.2857 11

 accuracy                         0.6400       100
macro avg     0.2967    0.3097    0.2800       100

weighted avg 0.6027 0.6400 0.5880 100 | | 0.0493 | 15.0 | 2580 | 0.5524 | precision recall f1-score support

  ALLERGY     0.6667    0.5000    0.5714         4

ASSESSMENT 0.0000 0.0000 0.0000 4 CC 0.0000 0.0000 0.0000 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8800 1.0000 0.9362 22 GENHX 0.7600 0.9500 0.8444 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.7143 0.7143 0.7143 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.3750 0.7500 0.5000 4 PASTSURGICAL 0.7273 1.0000 0.8421 8 PLAN 0.5000 0.3333 0.4000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.6667 0.1818 0.2857 11

 accuracy                         0.6300       100
macro avg     0.2895    0.3215    0.2880       100

weighted avg 0.5888 0.6300 0.5852 100 | | 0.0493 | 16.0 | 2752 | 0.5291 | precision recall f1-score support

  ALLERGY     0.6667    0.5000    0.5714         4

ASSESSMENT 0.0000 0.0000 0.0000 4 CC 0.2500 0.2500 0.2500 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8800 1.0000 0.9362 22 GENHX 0.7500 0.9000 0.8182 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.7143 0.7143 0.7143 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.5000 1.0000 0.6667 4 PASTSURGICAL 0.6154 1.0000 0.7619 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.6667 0.1818 0.2857 11

 accuracy                         0.6300       100
macro avg     0.2772    0.3273    0.2836       100

weighted avg 0.5778 0.6300 0.5782 100 | | 0.0493 | 17.0 | 2924 | 0.5434 | precision recall f1-score support

  ALLERGY     0.5000    0.2500    0.3333         4

ASSESSMENT 0.0000 0.0000 0.0000 4 CC 0.0000 0.0000 0.0000 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8800 1.0000 0.9362 22 GENHX 0.7500 0.9000 0.8182 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.6250 0.7143 0.6667 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.4000 1.0000 0.5714 4 PASTSURGICAL 0.7273 1.0000 0.8421 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.6667 0.1818 0.2857 11

 accuracy                         0.6100       100
macro avg     0.2524    0.3023    0.2560       100

weighted avg 0.5599 0.6100 0.5579 100 | | 0.0353 | 18.0 | 3096 | 0.5396 | precision recall f1-score support

  ALLERGY     0.6667    0.5000    0.5714         4

ASSESSMENT 0.0000 0.0000 0.0000 4 Afib 0.0000 0.0000 0.0000 0 CC 0.0000 0.0000 0.0000 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8462 1.0000 0.9167 22 GENHX 0.7600 0.9500 0.8444 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.8333 0.7143 0.7692 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.5000 1.0000 0.6667 4 PASTSURGICAL 0.6667 1.0000 0.8000 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.6667 0.1818 0.2857 11

 accuracy                         0.6300       100
macro avg     0.2590    0.3022    0.2629       100

weighted avg 0.5748 0.6300 0.5760 100 | | 0.0353 | 19.0 | 3268 | 0.5345 | precision recall f1-score support

  ALLERGY     0.6667    0.5000    0.5714         4

ASSESSMENT 0.0000 0.0000 0.0000 4 Afib 0.0000 0.0000 0.0000 0 CC 0.0000 0.0000 0.0000 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8462 1.0000 0.9167 22 GENHX 0.7917 0.9500 0.8636 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.8333 0.7143 0.7692 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.4444 1.0000 0.6154 4 PASTSURGICAL 0.7273 1.0000 0.8421 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.7500 0.2727 0.4000 11

 accuracy                         0.6400       100
macro avg     0.2647    0.3065    0.2688       100

weighted avg 0.5929 0.6400 0.5937 100 | | 0.0353 | 20.0 | 3440 | 0.5317 | precision recall f1-score support

  ALLERGY     0.6667    0.5000    0.5714         4

ASSESSMENT 0.0000 0.0000 0.0000 4 Afib 0.0000 0.0000 0.0000 0 CC 0.0000 0.0000 0.0000 4 DIAGNOSIS 0.0000 0.0000 0.0000 1 DISPOSITION 0.0000 0.0000 0.0000 2 EDCOURSE 0.0000 0.0000 0.0000 3 EXAM 0.5000 1.0000 0.6667 1 FAM/SOCHX 0.8462 1.0000 0.9167 22 GENHX 0.7917 0.9500 0.8636 20 GYNHX 0.0000 0.0000 0.0000 1 IMAGING 0.0000 0.0000 0.0000 1 IMMUNIZATIONS 0.0000 0.0000 0.0000 1 LABS 0.0000 0.0000 0.0000 1 MEDICATIONS 0.8333 0.7143 0.7692 7 OTHER_HISTORY 0.0000 0.0000 0.0000 1 PASTMEDICALHX 0.4444 1.0000 0.6154 4 PASTSURGICAL 0.7273 1.0000 0.8421 8 PLAN 0.0000 0.0000 0.0000 3 PROCEDURES 0.0000 0.0000 0.0000 1 ROS 0.7500 0.2727 0.4000 11

 accuracy                         0.6400       100
macro avg     0.2647    0.3065    0.2688       100

weighted avg 0.5929 0.6400 0.5937 100 |

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.16.1
  • Tokenizers 0.15.0
Downloads last month
7
Safetensors
Model size
77M params
Tensor type
F32
·
Inference Examples
Text2Text Generation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for sarahahtee/t5-dialogue-classification-4

Base model

google/flan-t5-small
Finetuned
(298)
this model

Evaluation results

Metadata error: specify a dataset to view leaderboard