Hi Team,
This is a report from Giskard Bot Scan 🐢.
We have identified 7 potential vulnerabilities in your model based on an automated scan.
This automated analysis evaluated the model on the dataset financial_phrasebank (subset sentences_allagree, split train).
👉Performance issues (7)
For records in the dataset where text contains "share", the Precision is 100.0% lower than the global Precision.
| Level |
Data slice |
Metric |
Deviation |
| major 🔴 |
text contains "share" |
Precision = 0.000e+00 |
-100.00% than global |
Taxonomy
avid-effect:performance:P0204
🔍✨Examples
|
text |
label |
Predicted label |
| 10 |
Its board of directors will propose a dividend of EUR0 .12 per share for 2010 , up from the EUR0 .08 per share paid in 2009 . |
neutral |
positive (p = 0.95) |
| 11 |
MegaFon 's subscriber base increased 16.1 % in 2009 to 50.5 million users as of December 31 , while its market share by the number of customers amounted to 24 % as of late 2009 , up from 23 % as of late 2008 , according to TeliaSonera estimates . |
neutral |
positive (p = 0.96) |
| 33 |
The world 's second largest stainless steel maker said net profit in the three-month period until Dec. 31 surged to euro603 million US$ 781 million , or euro3 .33 US$ 4.31 per share , from euro172 million , or euro0 .94 per share , the previous year . |
neutral |
positive (p = 0.95) |
For records in the dataset where text contains "finland", the Balanced Accuracy is 66.72% lower than the global Balanced Accuracy.
| Level |
Data slice |
Metric |
Deviation |
| major 🔴 |
text contains "finland" |
Balanced Accuracy = 0.003 |
-66.72% than global |
Taxonomy
avid-effect:performance:P0204
🔍✨Examples
|
text |
label |
Predicted label |
| 37 |
At the request of Finnish media company Alma Media 's newspapers , research manager Jari Kaivo-oja at the Finland Futures Research Centre at the Turku School of Economics has drawn up a future scenario for Finland 's national economy by using a model developed by the University of Denver . |
negative |
neutral (p = 0.84) |
| 52 |
Circulation revenue has increased by 5 % in Finland and 4 % in Sweden in 2008 . |
neutral |
positive (p = 0.95) |
| 84 |
According to HKScan Finland , the plan is to increase J+ñrvi-Suomen Portti 's net sales to EUR 80mn to EUR 100mn . |
neutral |
positive (p = 0.95) |
For records in the dataset where text contains "mn", the Precision is 59.69% lower than the global Precision.
| Level |
Data slice |
Metric |
Deviation |
| major 🔴 |
text contains "mn" |
Precision = 0.004 |
-59.69% than global |
Taxonomy
avid-effect:performance:P0204
🔍✨Examples
|
text |
label |
Predicted label |
| 2 |
In the third quarter of 2010 , net sales increased by 5.2 % to EUR 205.5 mn , and operating profit by 34.9 % to EUR 23.5 mn . |
neutral |
positive (p = 0.96) |
| 3 |
Operating profit rose to EUR 13.1 mn from EUR 8.7 mn in the corresponding period in 2007 representing 7.7 % of net sales . |
neutral |
positive (p = 0.95) |
| 4 |
Operating profit totalled EUR 21.1 mn , up from EUR 18.6 mn in 2007 , representing 9.7 % of net sales . |
neutral |
positive (p = 0.96) |
For records in the dataset where text contains "year", the Precision is 56.53% lower than the global Precision.
| Level |
Data slice |
Metric |
Deviation |
| major 🔴 |
text contains "year" |
Precision = 0.005 |
-56.53% than global |
Taxonomy
avid-effect:performance:P0204
🔍✨Examples
|
text |
label |
Predicted label |
| 1 |
For the last quarter of 2010 , Componenta 's net sales doubled to EUR131m from EUR76m for the same period a year earlier , while it moved to a zero pre-tax profit from a pre-tax loss of EUR7m . |
neutral |
positive (p = 0.95) |
| 7 |
Consolidated net sales increased 16 % to reach EUR74 .8 m , while operating profit amounted to EUR0 .9 m compared to a loss of EUR0 .7 m in the prior year period . |
neutral |
positive (p = 0.96) |
| 15 |
Nordea Group 's operating profit increased in 2010 by 18 percent year-on-year to 3.64 billion euros and total revenue by 3 percent to 9.33 billion euros . |
neutral |
positive (p = 0.96) |
For records in the dataset where text contains "operating", the Precision is 52.6% lower than the global Precision.
| Level |
Data slice |
Metric |
Deviation |
| major 🔴 |
text contains "operating" |
Precision = 0.005 |
-52.60% than global |
Taxonomy
avid-effect:performance:P0204
🔍✨Examples
|
text |
label |
Predicted label |
| 2 |
In the third quarter of 2010 , net sales increased by 5.2 % to EUR 205.5 mn , and operating profit by 34.9 % to EUR 23.5 mn . |
neutral |
positive (p = 0.96) |
| 3 |
Operating profit rose to EUR 13.1 mn from EUR 8.7 mn in the corresponding period in 2007 representing 7.7 % of net sales . |
neutral |
positive (p = 0.95) |
| 4 |
Operating profit totalled EUR 21.1 mn , up from EUR 18.6 mn in 2007 , representing 9.7 % of net sales . |
neutral |
positive (p = 0.96) |
For records in the dataset where text contains "million", the Precision is 47.0% lower than the global Precision.
| Level |
Data slice |
Metric |
Deviation |
| major 🔴 |
text contains "million" |
Precision = 0.006 |
-47.00% than global |
Taxonomy
avid-effect:performance:P0204
🔍✨Examples
|
text |
label |
Predicted label |
| 11 |
MegaFon 's subscriber base increased 16.1 % in 2009 to 50.5 million users as of December 31 , while its market share by the number of customers amounted to 24 % as of late 2009 , up from 23 % as of late 2008 , according to TeliaSonera estimates . |
neutral |
positive (p = 0.96) |
| 18 |
The company 's net profit rose 11.4 % on the year to 82.2 million euros in 2005 on sales of 686.5 million euros , 13.8 % up on the year , the company said earlier . |
neutral |
positive (p = 0.96) |
| 19 |
The Lithuanian beer market made up 14.41 million liters in January , a rise of 0.8 percent from the year-earlier figure , the Lithuanian Brewers ' Association reporting citing the results from its members . |
neutral |
positive (p = 0.95) |
For records in the dataset where text contains "eur", the Precision is 36.26% lower than the global Precision.
| Level |
Data slice |
Metric |
Deviation |
| major 🔴 |
text contains "eur" |
Precision = 0.007 |
-36.26% than global |
Taxonomy
avid-effect:performance:P0204
🔍✨Examples
|
text |
label |
Predicted label |
| 2 |
In the third quarter of 2010 , net sales increased by 5.2 % to EUR 205.5 mn , and operating profit by 34.9 % to EUR 23.5 mn . |
neutral |
positive (p = 0.96) |
| 3 |
Operating profit rose to EUR 13.1 mn from EUR 8.7 mn in the corresponding period in 2007 representing 7.7 % of net sales . |
neutral |
positive (p = 0.95) |
| 4 |
Operating profit totalled EUR 21.1 mn , up from EUR 18.6 mn in 2007 , representing 9.7 % of net sales . |
neutral |
positive (p = 0.96) |
Checkout out the Giskard Space and Giskard Documentation to learn more about how to test your model.
Disclaimer: it's important to note that automated scans may produce false positives or miss certain vulnerabilities. We encourage you to review the findings and assess the impact accordingly.
