Spaces:
Running
Running
| EXAMPLE_MODEL_ID = "cardiffnlp/twitter-roberta-base-sentiment-latest" | |
| TITLE_MD = """ | |
| <div style="display: flex; justify-content: center;"> | |
| <h1 style="text-align: center;"> | |
| 🐢Giskard Evaluator | |
| </h1> | |
| </div> | |
| Welcome to the Giskard Evaluator Space! | |
| Get a model vulnerability report immediately and freely by simply providing the model and dataset id on Hugging Face. | |
| You can also checkout our library documentation <a href="https://docs.giskard.ai/en/latest/getting_started/quickstart/index.html">here</a>. | |
| <!-- Don’t hesitate to give the us a <a href="https://github.com/Giskard-AI/giskard">star on GitHub</a> ⭐️ if you find this work useful! --> | |
| """ | |
| FOOTER_HTML = """ | |
| <div style="display: flex; justify-content: center;"> | |
| <p style="text-align: center;"> | |
| This evaluator is built on top of <a href="https://github.com/Giskard-AI/giskard">Giskard Open Source library</a> | |
| and <a href="https://github.com/Giskard-AI/cicd">Giskard CI/CD</a>. | |
| <br/> | |
| If you like our work, please give the projects ⭐️ star on <a href="https://github.com/Giskard-AI/giskard">GitHub</a>. | |
| </p> | |
| </div> | |
| """ | |
| INTRODUCTION_MD = """ | |
| <div style="display: flex; justify-content: center;"> | |
| <h2 style="text-align: center;"> | |
| Text Classification | |
| </h2> | |
| </div> | |
| """ | |
| CONFIRM_MAPPING_DETAILS_MD = """ | |
| <h1 style="text-align: center;"> | |
| Confirm Pre-processing Details | |
| </h1> | |
| Make sure the output variable's labels and the input variable's name are accurately mapped across both the dataset and the model. You can select the output variable's labels from the dropdowns below. | |
| """ | |
| CONFIRM_MAPPING_DETAILS_FAIL_MD = """ | |
| <h1 style="text-align: center;"> | |
| Confirm Pre-processing Details | |
| </h1> | |
| We're unable to automatically map the input variable's name and output variable's labels of your dataset with the model's. Please manually check the mapping below. | |
| """ | |
| CONFIRM_MAPPING_DETAILS_FAIL_RAW = """ | |
| We're unable to automatically map the input variable's name and output variable's labels of your dataset with the model's. Please manually check the mapping below. | |
| """ | |
| CHECK_CONFIG_OR_SPLIT_RAW = """ | |
| Please check your dataset config or split. | |
| """ | |
| CHECK_LOG_SECTION_RAW = """Your have successfully submitted a Giskard evaluation job. Further details are available in the Logs tab. You can find your report posted in your model's community discussion section.""" | |
| PREDICTION_SAMPLE_MD = """ | |
| <h1 style="text-align: center;"> | |
| Model Prediction Sample | |
| </h1> | |
| Here's a sample of your model's prediction on an example from the dataset. | |
| """ | |
| MAPPING_STYLED_ERROR_WARNING = """ | |
| <h3 style="text-align: center;color: orange; background-color: #fff0f3; border-radius: 8px; padding: 10px; "> | |
| ⚠️ We're unable to automatically map the input variable's name and output variable's labels of your dataset with the model's. Please manually check the mapping below. | |
| </h3> | |
| """ | |
| UNMATCHED_MODEL_DATASET_STYLED_ERROR = """ | |
| <h3 style="text-align: center;color: #fa5f5f; background-color: #fbe2e2; border-radius: 8px; padding: 10px; "> | |
| ❌ Your model and dataset have different numbers of labels. Please double check your model and dataset. | |
| </h3> | |
| """ | |
| NOT_FOUND_MODEL_RAW = """ | |
| We cannot find your model on Hugging Face. Please check that the model id is correct and publicly accessible. | |
| """ | |
| NOT_TEXT_CLASSIFICATION_MODEL_RAW = """ | |
| Your model does not fall under the category of text classification. This page is specifically designated for the evaluation of text classification models. | |
| """ | |
| USE_INFERENCE_API_TIP = """ | |
| We use the <b>free</b> | |
| <a href="https://huggingface.co/docs/api-inference/detailed_parameters#text-classification-task"> | |
| Hugging Face Inference API | |
| </a> | |
| to evaluate the models with a <a href="https://huggingface.co/docs/hub/security-tokens#user-access-tokens">Hugging Face acces token</a>. | |
| <br/> | |
| Please <a href="https://huggingface.co/settings/tokens">get your token</a> and input here. This would help us to avoid the API calling rate limit and make the evaluation faster. | |
| <br/> | |
| """ | |
| USE_INFERENCE_API_NOTICE = """ | |
| <b>Notice:</b> | |
| <ul> | |
| <li><b>Your token will only be used for your own evaluation, and will not be saved.</b></li> | |
| <li><b>You will not be charged for the free Hugging Face Inference API.</b></li> | |
| </ul> | |
| """ | |
| HF_TOKEN_INVALID_STYLED = """ | |
| <p style="text-align: left;color: red; "> | |
| Your Hugging Face token is invalid. Please double check your token. | |
| </p> | |
| """ | |
| VALIDATED_MODEL_DATASET_STYLED = """ | |
| <h3 style="text-align: center;color: #4ca154; background-color: #e2fbe8; border-radius: 8px; padding: 10px; "> | |
| Your model and dataset have been validated! | |
| </h3>""" | |
| NOT_FOUND_DATASET_RAW = """ | |
| We cannot find your dataset on Hugging Face. Please check that the dataset id is correct and publicly accessible. | |
| """ | |
| def get_dataset_fetch_error_raw(error): | |
| return f"""Sorry you cannot use this dataset because of an unknwon error: "{error}".""" | |