nohup.out

2024-03-29 14:20:49.463961: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:20:50.507795: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/queueing.py", line 522, in process_events
    response = await route_utils.call_process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1689, in process_api
    result = await self.call_function(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1255, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/utils.py", line 750, in wrapper
    response = f(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 31, in analyze_and_highlight
    sentences_weights, _ = explainer(text, model_type)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 16, in explainer
    exp = explainer_.explain_instance(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 413, in explain_instance
    data, yss, distances = self.__data_labels_distances(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 482, in __data_labels_distances
    labels = classifier_fn(inverse_data)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 8, in predictor_wrapper
    return predict_for_explainanility(text=text, model_type=model_type)
  File "/home/aliasgarov/copyright_checker/predictors.py", line 195, in predict_for_explainanility
    outputs = model(**tokenized_text)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
    outputs = self.bert(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
    encoder_outputs = self.encoder(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
    layer_outputs = layer_module(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
    self_attention_outputs = self.attention(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
    self_outputs = self.self(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 325, in forward
    attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 5.86 GiB. GPU 0 has a total capacity of 14.58 GiB of which 1.76 GiB is free. Including non-PyTorch memory, this process has 12.81 GiB memory in use. Of the allocated memory 11.71 GiB is allocated by PyTorch, and 1008.80 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-03-29 14:31:17.459384: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:31:18.518981: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/queueing.py", line 522, in process_events
    response = await route_utils.call_process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1689, in process_api
    result = await self.call_function(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1255, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/utils.py", line 750, in wrapper
    response = f(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 31, in analyze_and_highlight
    sentences_weights, _ = explainer(text, model_type)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 16, in explainer
    exp = explainer_.explain_instance(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 413, in explain_instance
    data, yss, distances = self.__data_labels_distances(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 482, in __data_labels_distances
    labels = classifier_fn(inverse_data)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 8, in predictor_wrapper
    return predict_for_explainanility(text=text, model_type=model_type)
  File "/home/aliasgarov/copyright_checker/predictors.py", line 195, in predict_for_explainanility
    outputs = model(**tokenized_text)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
    outputs = self.bert(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
    encoder_outputs = self.encoder(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
    layer_outputs = layer_module(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
    self_attention_outputs = self.attention(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
    self_outputs = self.self(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 325, in forward
    attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 5.86 GiB. GPU 0 has a total capacity of 14.58 GiB of which 2.47 GiB is free. Including non-PyTorch memory, this process has 12.10 GiB memory in use. Of the allocated memory 11.71 GiB is allocated by PyTorch, and 278.80 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-03-29 14:36:15.933048: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:36:16.966744: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/queueing.py", line 522, in process_events
    response = await route_utils.call_process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1689, in process_api
    result = await self.call_function(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1255, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/utils.py", line 750, in wrapper
    response = f(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 31, in analyze_and_highlight
    sentences_weights, _ = explainer(text, model_type)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 16, in explainer
    exp = explainer_.explain_instance(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 413, in explain_instance
    data, yss, distances = self.__data_labels_distances(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 482, in __data_labels_distances
    labels = classifier_fn(inverse_data)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 8, in predictor_wrapper
    return predict_for_explainanility(text=text, model_type=model_type)
  File "/home/aliasgarov/copyright_checker/predictors.py", line 195, in predict_for_explainanility
    outputs = model(**tokenized_text)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
    outputs = self.bert(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
    encoder_outputs = self.encoder(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
    layer_outputs = layer_module(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
    self_attention_outputs = self.attention(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
    self_outputs = self.self(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 325, in forward
    attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 5.86 GiB. GPU 0 has a total capacity of 14.58 GiB of which 5.63 GiB is free. Including non-PyTorch memory, this process has 8.95 GiB memory in use. Of the allocated memory 8.59 GiB is allocated by PyTorch, and 234.51 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-03-29 14:38:49.739939: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:38:50.770137: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/queueing.py", line 522, in process_events
    response = await route_utils.call_process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1689, in process_api
    result = await self.call_function(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1255, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/utils.py", line 750, in wrapper
    response = f(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 31, in analyze_and_highlight
    sentences_weights, _ = explainer(text, model_type)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 16, in explainer
    exp = explainer_.explain_instance(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 413, in explain_instance
    data, yss, distances = self.__data_labels_distances(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 482, in __data_labels_distances
    labels = classifier_fn(inverse_data)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 8, in predictor_wrapper
    return predict_for_explainanility(text=text, model_type=model_type)
  File "/home/aliasgarov/copyright_checker/predictors.py", line 195, in predict_for_explainanility
    outputs = model(**tokenized_text)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
    outputs = self.bert(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
    encoder_outputs = self.encoder(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
    layer_outputs = layer_module(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
    self_attention_outputs = self.attention(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
    self_outputs = self.self(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 365, in forward
    context_layer = torch.matmul(attention_probs, value_layer)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 500.00 MiB. GPU 0 has a total capacity of 14.58 GiB of which 285.56 MiB is free. Including non-PyTorch memory, this process has 14.30 GiB memory in use. Of the allocated memory 13.96 GiB is allocated by PyTorch, and 222.09 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-03-29 14:42:21.299532: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:42:22.362964: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/queueing.py", line 522, in process_events
    response = await route_utils.call_process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1689, in process_api
    result = await self.call_function(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1255, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/utils.py", line 750, in wrapper
    response = f(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 31, in analyze_and_highlight
    sentences_weights, _ = explainer(text, model_type)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 16, in explainer
    exp = explainer_.explain_instance(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 413, in explain_instance
    data, yss, distances = self.__data_labels_distances(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/lime/lime_text.py", line 482, in __data_labels_distances
    labels = classifier_fn(inverse_data)
  File "/home/aliasgarov/copyright_checker/highlighter.py", line 8, in predictor_wrapper
    return predict_for_explainanility(text=text, model_type=model_type)
  File "/home/aliasgarov/copyright_checker/predictors.py", line 195, in predict_for_explainanility
    outputs = model(**tokenized_text)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
    outputs = self.bert(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
    encoder_outputs = self.encoder(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
    layer_outputs = layer_module(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
    self_attention_outputs = self.attention(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
    self_outputs = self.self(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 365, in forward
    context_layer = torch.matmul(attention_probs, value_layer)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 500.00 MiB. GPU 0 has a total capacity of 14.58 GiB of which 285.56 MiB is free. Including non-PyTorch memory, this process has 14.30 GiB memory in use. Of the allocated memory 13.96 GiB is allocated by PyTorch, and 222.09 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-03-29 14:48:12.298004: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 14:48:13.329416: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:198: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
Running on local URL:  http://0.0.0.0:80
Running on public URL: https://008ca76c2bb7f8d8a3.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.007239958671520294, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': -0.005940472653387939, 'She was the first woman to earn a geology degree from Stanford.': 0.0016036026130179831, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.001537302361237576, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': -0.007227867941129461, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': -0.0003420683510965876, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': -0.009836457467468768, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.005398886066759868} bc
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.007024535420434749, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': -0.005433933632620999, 'She was the first woman to earn a geology degree from Stanford.': 0.0033503657592824465, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.0012667157053936522, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': -0.007406581188202247, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': -0.0006685564234160865, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': -0.009190228364350466, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.004699842541408435} bc
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.641953608456155, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': 0.020200923452086798, 'She was the first woman to earn a geology degree from Stanford.': 0.008136189058261252, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.12504063362482074, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': 0.14466029601373961, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': 0.045496763632525375, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': 0.11435786746768793, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.3560611292221768} quillbot
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.049232424744256965, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': -0.0808599351295588, 'She was the first woman to earn a geology degree from Stanford.': -0.028306312264799082, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.018576473883078034, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': -0.0658758038308371, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': 0.00520141594810037, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': -0.06700218547318215, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.11886694361432464} bc
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.07048027659860119, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': -0.07512228868644406, 'She was the first woman to earn a geology degree from Stanford.': -0.04560898943130033, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.01102573043004705, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': -0.06753051178176432, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': -0.0016847880819046478, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': -0.06913938144762188, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.13576338155813136} bc
2024-03-29 15:01:50.768841: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 15:01:51.796519: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/optimum/bettertransformer/models/encoder_models.py:301: UserWarning: The PyTorch API of nested tensors is in prototype stage and will change in the near future. (Triggered internally at ../aten/src/ATen/NestedTensorImpl.cpp:177.)
  hidden_states = torch._nested_tensor_from_mask(hidden_states, ~attention_mask)
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
Running on local URL:  http://0.0.0.0:80
Running on public URL: https://e095d1a53e42b16b1b.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.007083724433403481, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': -0.005900632715473411, 'She was the first woman to earn a geology degree from Stanford.': 0.00288471219406703, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.0012162868179568342, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': -0.006270546763081995, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': -6.844510148763104e-05, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': -0.008883191796269094, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.005504050009961782} bc
Original BC scores: AI: 6.408023001114316e-09, HUMAN: 1.0
Calibration BC scores: AI: 0.0, HUMAN: 1.0
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 6.408023001114316e-09, HUMAN: 1.0
Calibration BC scores: AI: 0.0, HUMAN: 1.0
Starting MC
MC Score: {'OpenAI GPT': 0.0, 'Mistral': 0.0, 'CLAUDE': 0.0, 'Gemini': 0.0, 'LLAMA 2': 0.0}
{'Lou Henry Hoover (March 29, 1874 – January 7, 1944) was the first lady of the United States from 1929 to 1933 as the wife of President Herbert Hoover.': -0.599086635981887, 'She was active in community groups, including the Girl Scouts of the USA, which she led from 1922 to 1925 and from 1935 to 1937.': 0.08136319631271138, 'She was the first woman to earn a geology degree from Stanford.': 0.02834857510284846, 'In the first twenty years of their marriage, the Hoovers lived in several countries; during World War I, they led efforts to assist war refugees.': 0.061459884832511476, 'Beginning in 1917, they lived in Washington, D.C., as Herbert became a high government official.': 0.16672173091342543, 'In the White House, Lou Hoover dedicated her time as first lady to her volunteer work, though she did not publicize it.': 0.0820923392682848, 'Her invitation of Jessie De Priest to the White House for tea was controversial in the South.': 0.13399838230662856, "After Herbert's defeat for re-election in 1932, Lou Hoover continued her work, helping provide refugee support with her husband during World War II, and died suddenly of a heart attack in 1944.": 0.3821691921261263} quillbot
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
2024-03-29 19:06:50.019873: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-29 19:06:51.074912: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
/usr/bin/python3: No module named spacy
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/optimum/bettertransformer/models/encoder_models.py:301: UserWarning: The PyTorch API of nested tensors is in prototype stage and will change in the near future. (Triggered internally at ../aten/src/ATen/NestedTensorImpl.cpp:177.)
  hidden_states = torch._nested_tensor_from_mask(hidden_states, ~attention_mask)
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
Token indices sequence length is longer than the specified maximum sequence length for this model (881 > 512). Running this sequence through the model will result in indexing errors
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
Running on local URL:  http://0.0.0.0:80
Running on public URL: https://94e72aa3904122b29c.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)
Original BC scores: AI: 1.0, HUMAN: 1.7993962986295742e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 1.0, HUMAN: 1.7993962986295742e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Starting MC
MC Score: {'OpenAI GPT': 0.9994855300308254, 'Mistral': 5.1601761922609244e-11, 'CLAUDE': 8.48403323344426e-11, 'Gemini': 2.8437433518348655e-10, 'LLAMA 2': 9.498188443606356e-11}
Original BC scores: AI: 1.0, HUMAN: 1.7997051626750249e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Original BC scores: AI: 0.6732428669929504, HUMAN: 0.3267570436000824
Calibration BC scores: AI: 0.4375, HUMAN: 0.5625
Original BC scores: AI: 0.5024993419647217, HUMAN: 0.49750062823295593
Calibration BC scores: AI: 0.4375, HUMAN: 0.5625
Original BC scores: AI: 0.7561723589897156, HUMAN: 0.24382765591144562
Calibration BC scores: AI: 0.4375, HUMAN: 0.5625
Original BC scores: AI: 1.0, HUMAN: 1.8036925286679661e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Original BC scores: AI: 0.7560267448425293, HUMAN: 0.24397319555282593
Calibration BC scores: AI: 0.4375, HUMAN: 0.5625
Original BC scores: AI: 0.989621639251709, HUMAN: 0.010378347709774971
Calibration BC scores: AI: 0.5178571428571429, HUMAN: 0.4821428571428571
Original BC scores: AI: 1.0, HUMAN: 2.039939994702422e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 1.0, HUMAN: 2.039939994702422e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Starting MC
MC Score: {'OpenAI GPT': 0.9994855298515718, 'Mistral': 4.535480345181983e-11, 'CLAUDE': 2.261075985034601e-10, 'Gemini': 3.1878497183516737e-10, 'LLAMA 2': 1.0480460580159845e-10}
Original BC scores: AI: 1.0, HUMAN: 2.039939994702422e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 1.0, HUMAN: 2.039939994702422e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Starting MC
MC Score: {'OpenAI GPT': 0.9994855298515718, 'Mistral': 4.535480345181983e-11, 'CLAUDE': 2.261075985034601e-10, 'Gemini': 3.1878497183516737e-10, 'LLAMA 2': 1.0480460580159845e-10}
{'Add-on features now encompass AI and Source Identification, leveraging forensic linguistic analysis to ascertain the origin, reliability, and authenticity of content.': -0.15216478135731262, 'These advanced tools can distinguish between human and AI-generated material, pinpointing the specific AI models employed in creation.': -0.05895885252560595, 'This enhancement bolsters the ability to assess content trustworthiness effectively.': 0.03353039204460538} bc
Original BC scores: AI: 0.998177170753479, HUMAN: 0.0018228011904284358
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 0.998177170753479, HUMAN: 0.0018228011904284358
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
Starting MC
MC Score: {'OpenAI GPT': 0.6614420057714218, 'Mistral': 2.7132188074993352e-11, 'CLAUDE': 1.2335682936047867e-10, 'Gemini': 1.7620911369483686e-10, 'LLAMA 2': 1.714725314469418e-10}
{'AI Identification and Source Identification are add-on capabilities that use forensic linguistic analysis to offer insights into the origin, dependability, and trustworthiness of content as well as whether it was created by humans or artificial intelligence (AI).': -0.006323229799663152, 'They can even identify the precise AI models that were used to create the content.': 0.017586576131630234} bc
{'AI Identification and Source Identification are add-on capabilities that use forensic linguistic analysis to offer insights into the origin, dependability, and trustworthiness of content as well as whether it was created by humans or artificial intelligence (AI).': -0.43261755952898956, 'They can even identify the precise AI models that were used to create the content.': 0.10732631520197373} quillbot
{'AI Identification and Source Identification are add-on capabilities that use forensic linguistic analysis to offer insights into the origin, dependability, and trustworthiness of content as well as whether it was created by humans or artificial intelligence (AI).': -0.4322117278076279, 'They can even identify the precise AI models that were used to create the content.': 0.10778412185868685} quillbot
{'AI Identification and Source Identification are add-on capabilities that use forensic linguistic analysis to offer insights into the origin, dependability, and trustworthiness of content as well as whether it was created by humans or artificial intelligence (AI).': -0.43300422387049115, 'They can even identify the precise AI models that were used to create the content.': 0.10687924275434384} quillbot
{'Add-on feat ures now encompass AI and Source Identifi cation, leveraging for ensic linguistic analysis to ascertain the origin, reliability, and authen ticity of content.': -0.16172325612226013, 'These advanc ed tools can distinguish between human and AI-generated material, pin pointing the specific AI models employed in creation.': -0.06511130357854991, 'This enhance ment bolsters the ability to assess content trust worthiness effectively.': 0.05332794099561823} bc
{'Add-on feat ures now encompass AI and Source Identifi cation, leveraging for ensic linguistic analysis to ascertain the origin, reliability, and authen ticity of content.': -0.16378145994849636, 'These advanc ed tools can distinguish between human and AI-generated material, pin pointing the specific AI models employed in creation.': -0.06739973523793355, 'This enhance ment bolsters the ability to assess content trust worthiness effectively.': 0.05366690466131973} bc
Original BC scores: AI: 0.995067834854126, HUMAN: 0.004932152573019266
Calibration BC scores: AI: 0.5957446808510638, HUMAN: 0.4042553191489362
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 0.995067834854126, HUMAN: 0.004932152573019266
Calibration BC scores: AI: 0.5957446808510638, HUMAN: 0.4042553191489362
Starting MC
MC Score: {'OpenAI GPT': 0.5957441340683721, 'Mistral': 2.0416833660118585e-10, 'CLAUDE': 5.001776967436859e-07, 'Gemini': 2.5271727453711155e-08, 'LLAMA 2': 2.1129099166428725e-08}
Original BC scores: AI: 0.00025900782202370465, HUMAN: 0.9997410178184509
Calibration BC scores: AI: 0.04296875, HUMAN: 0.95703125
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'LLAMA 2']
Original BC scores: AI: 0.00025900782202370465, HUMAN: 0.9997410178184509
Calibration BC scores: AI: 0.04296875, HUMAN: 0.95703125
Starting MC
MC Score: {'OpenAI GPT': 0.025428532807609403, 'Mistral': 1.6376084024317497e-09, 'CLAUDE': 1.6831211047289287e-06, 'Gemini': 1.8230926181583228e-06, 'LLAMA 2': 0.017536709341059307}
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:205: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 407, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 680, in __call__
    await self.app(scope, receive, send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 78, in wrapped_app
    await response(scope, receive, sender)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/responses.py", line 151, in __call__
    await send(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 50, in sender
    await send(message)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 161, in _send
    await send(message)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 489, in send
    output = self.conn.send(event=response)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/h11/_connection.py", line 512, in send
    data_list = self.send_with_data_passthrough(event)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/h11/_connection.py", line 537, in send_with_data_passthrough
    self._process_event(self.our_role, event)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/h11/_connection.py", line 272, in _process_event
    self._cstate.process_event(role, type(event), server_switch_event)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/h11/_state.py", line 293, in process_event
    self._fire_event_triggered_transitions(role, _event_type)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/h11/_state.py", line 311, in _fire_event_triggered_transitions
    raise LocalProtocolError(
h11._util.LocalProtocolError: can't handle event type Response when role=SERVER and state=MUST_CLOSE
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
2024-04-12 19:20:06.424411: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-04-12 19:20:11.475524: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
Some weights of the model checkpoint at textattack/roberta-base-CoLA were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']
- This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
Framework not specified. Using pt to export the model.
Using the export variant default. Available variants are:
    - default: The default ONNX variant.
Using framework PyTorch: 2.2.2+cu121
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/deberta_v2/modeling_deberta_v2.py:554: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  torch.tensor(mid - 1).type_as(relative_pos),
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/deberta_v2/modeling_deberta_v2.py:558: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  torch.ceil(torch.log(abs_pos / mid) / torch.log(torch.tensor((max_position - 1) / mid)) * (mid - 1)) + mid
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/deberta_v2/modeling_deberta_v2.py:717: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  scale = torch.sqrt(torch.tensor(query_layer.size(-1), dtype=torch.float) * scale_factor)
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/deberta_v2/modeling_deberta_v2.py:717: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  scale = torch.sqrt(torch.tensor(query_layer.size(-1), dtype=torch.float) * scale_factor)
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/deberta_v2/modeling_deberta_v2.py:792: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  scale = torch.sqrt(torch.tensor(pos_key_layer.size(-1), dtype=torch.float) * scale_factor)
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/deberta_v2/modeling_deberta_v2.py:792: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  scale = torch.sqrt(torch.tensor(pos_key_layer.size(-1), dtype=torch.float) * scale_factor)
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/deberta_v2/modeling_deberta_v2.py:804: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  scale = torch.sqrt(torch.tensor(pos_query_layer.size(-1), dtype=torch.float) * scale_factor)
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/deberta_v2/modeling_deberta_v2.py:804: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  scale = torch.sqrt(torch.tensor(pos_query_layer.size(-1), dtype=torch.float) * scale_factor)
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/deberta_v2/modeling_deberta_v2.py:805: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if key_layer.size(-2) != query_layer.size(-2):
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/deberta_v2/modeling_deberta_v2.py:112: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  output = input.masked_fill(rmask, torch.tensor(torch.finfo(input.dtype).min))
Framework not specified. Using pt to export the model.
Using the export variant default. Available variants are:
    - default: The default ONNX variant.
Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
Non-default generation parameters: {'max_length': 512, 'min_length': 8, 'num_beams': 2, 'no_repeat_ngram_size': 4}
Using framework PyTorch: 2.2.2+cu121
Overriding 1 configuration item(s)
	- use_cache -> False
Using framework PyTorch: 2.2.2+cu121
Overriding 1 configuration item(s)
	- use_cache -> True
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/modeling_utils.py:943: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if causal_mask.shape[1] < attention_mask.shape[1]:
Using framework PyTorch: 2.2.2+cu121
Overriding 1 configuration item(s)
	- use_cache -> True
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py:509: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  elif past_key_value.shape[2] != key_value_states.shape[1]:
In-place op on output of tensor.shape. See https://pytorch.org/docs/master/onnx.html#avoid-inplace-operations-when-using-tensor-shape-in-tracing-mode
In-place op on output of tensor.shape. See https://pytorch.org/docs/master/onnx.html#avoid-inplace-operations-when-using-tensor-shape-in-tracing-mode
Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.
Non-default generation parameters: {'max_length': 512, 'min_length': 8, 'num_beams': 2, 'no_repeat_ngram_size': 4}
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
[nltk_data] Downloading package cmudict to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Unzipping corpora/cmudict.zip.
[nltk_data] Downloading package punkt to /home/aliasgarov/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data]     /home/aliasgarov/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
[nltk_data] Downloading package wordnet to
[nltk_data]     /home/aliasgarov/nltk_data...
/usr/bin/python3: No module named spacy
Running on local URL:  http://0.0.0.0:80
Running on public URL: https://06194131b0e8ad4f5d.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)

correcting text..:   0%|          | 0/5 [00:00<?, ?it/s]
correcting text..:  20%|██        | 1/5 [00:05<00:20,  5.15s/it]
correcting text..:  40%|████      | 2/5 [00:07<00:11,  3.79s/it]
correcting text..:  60%|██████    | 3/5 [00:09<00:05,  2.54s/it]
correcting text..:  80%|████████  | 4/5 [00:10<00:02,  2.02s/it]
correcting text..: 100%|██████████| 5/5 [00:12<00:00,  2.10s/it]
correcting text..: 100%|██████████| 5/5 [00:12<00:00,  2.50s/it]
/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/optimum/bettertransformer/models/encoder_models.py:301: UserWarning: The PyTorch API of nested tensors is in prototype stage and will change in the near future. (Triggered internally at ../aten/src/ATen/NestedTensorImpl.cpp:177.)
  hidden_states = torch._nested_tensor_from_mask(hidden_states, ~attention_mask)
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
Original BC scores: AI: 1.0, HUMAN: 3.9213916558367146e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Input Text: sFallout: A Post Nuclear Role Playing Game is a 1997 role-playing video game developed and published by Interplay Productions. Set in a post-apocalyptic world in the mid22nd century, it revolves around the player character seeking a replacement computer chip for their underground nuclear shelter's water supply system. The gameplay involves interacting with other survivors and engaging in turn-based combat. Fallout started development in 1994 as a game engine designed by Tim Cain (pictured). It was originally based on GURPS, a role-playing game system, though the character-customization scheme was changed after the GURPS/s
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'Grammar Enhancer']
Original BC scores: AI: 1.0, HUMAN: 3.9213916558367146e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Starting MC
MC Score: {'OpenAI GPT': 2.6440588756836946e-07, 'Mistral': 3.356145785245883e-10, 'CLAUDE': 4.970491762758412e-09, 'Gemini': 2.893925095001254e-09, 'Grammar Enhancer': 0.9994852579407048}
{'Fallout: A Post Nuclear Role Playing Game is a 1997 role-playing video game developed and published by Interplay Productions.': -0.1607462459261463, "Set in a post-apocalyptic world in the mid–22nd century, it revolves around the player character seeking a replacement computer chip for their underground nuclear shelter's water supply system.": 0.019970291679965425, 'The gameplay involves interacting with other survivors and engaging in turn-based combat.': 0.19539473225341195, 'Fallout started development in 1994 as a game engine designed by Tim Cain (pictured).': -0.030592020309353717, 'It was originally based on GURPS, a role-playing game system, though the character-customization scheme was changed after the GURPS': -0.1206822715329631} bc
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
{'Fallout: A Post Nuclear Role Playing Game is a 1997 role-playing video game developed and published by Interplay Productions.': -0.8857923310524768, "Set in a post-apocalyptic world in the mid–22nd century, it revolves around the player character seeking a replacement computer chip for their underground nuclear shelter's water supply system.": 0.09396163034470774, 'The gameplay involves interacting with other survivors and engaging in turn-based combat.': 0.03435038487713251, 'Fallout started development in 1994 as a game engine designed by Tim Cain (pictured).': -0.0013657031760451715, 'It was originally based on GURPS, a role-playing game system, though the character-customization scheme was changed after the GURPS': -0.028791310913184043} quillbot
Original BC scores: AI: 1.0, HUMAN: 3.9213916558367146e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Input Text: sFallout: A Post Nuclear Role Playing Game is a 1997 role-playing video game developed and published by Interplay Productions. Set in a post-apocalyptic world in the mid22nd century, it revolves around the player character seeking a replacement computer chip for their underground nuclear shelter's water supply system. The gameplay involves interacting with other survivors and engaging in turn-based combat. Fallout started development in 1994 as a game engine designed by Tim Cain (pictured). It was originally based on GURPS, a role-playing game system, though the character-customization scheme was changed after the GURPS/s
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'Grammar Enhancer']
Original BC scores: AI: 1.0, HUMAN: 3.9213916558367146e-09
Calibration BC scores: AI: 0.9994855305466238, HUMAN: 0.0005144694533761873
Starting MC
MC Score: {'OpenAI GPT': 2.6440588756836946e-07, 'Mistral': 3.356145785245883e-10, 'CLAUDE': 4.970491762758412e-09, 'Gemini': 2.893925095001254e-09, 'Grammar Enhancer': 0.9994852579407048}
{'Fallout: A Post Nuclear Role Playing Game is a 1997 role-playing video game developed and published by Interplay Productions.': -0.14584208704141496, "Set in a post-apocalyptic world in the mid–22nd century, it revolves around the player character seeking a replacement computer chip for their underground nuclear shelter's water supply system.": 0.021056781991986122, 'The gameplay involves interacting with other survivors and engaging in turn-based combat.': 0.1916434469369563, 'Fallout started development in 1994 as a game engine designed by Tim Cain (pictured).': -0.032527445466118764, 'It was originally based on GURPS, a role-playing game system, though the character-customization scheme was changed after the GURPS': -0.11670666669110184} bc
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
{'Fallout: A Post Nuclear Role Playing Game is a 1997 role-playing video game developed and published by Interplay Productions.': -0.9034253500750302, "Set in a post-apocalyptic world in the mid–22nd century, it revolves around the player character seeking a replacement computer chip for their underground nuclear shelter's water supply system.": 0.0884857561938886, 'The gameplay involves interacting with other survivors and engaging in turn-based combat.': 0.027812697159959997, 'Fallout started development in 1994 as a game engine designed by Tim Cain (pictured).': -0.006091521770887824, 'It was originally based on GURPS, a role-playing game system, though the character-customization scheme was changed after the GURPS': -0.019728908853879158} quillbot

correcting text..:   0%|          | 0/3 [00:00<?, ?it/s]
correcting text..:  33%|███▎      | 1/3 [00:03<00:06,  3.08s/it]
correcting text..:  67%|██████▋   | 2/3 [00:06<00:03,  3.22s/it]
correcting text..: 100%|██████████| 3/3 [00:07<00:00,  2.15s/it]
correcting text..: 100%|██████████| 3/3 [00:07<00:00,  2.43s/it]
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
Original BC scores: AI: 0.9981676340103149, HUMAN: 0.001832296489737928
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
Input Text: sThe Nobel Prize in Physics (Swedish: Nobelpriset i fysik) is a yearly award given by the Royal Swedish Academy of Sciences for those who have made the most outstanding contributions for humankind in the field of physics. It is one of the five Nobel Prizes established by the will of Alfred Nobel in 1895 and awarded since 1901, the others being the Nobel Prize in Chemistry, Nobel Prize in Literature, Nobel Peace Prize, and Nobel Prize in Physiology or Medicine. Physics is traditionally the first award presented in the Nobel Prize ceremony. /s
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'Grammar Enhancer']
Original BC scores: AI: 0.9981676340103149, HUMAN: 0.001832296489737928
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
Starting MC
MC Score: {'OpenAI GPT': 5.6480643213916335e-05, 'Mistral': 1.7635763073404052e-09, 'CLAUDE': 9.228064192213527e-05, 'Gemini': 7.672706390066632e-07, 'Grammar Enhancer': 0.6612924759502411}
{'The Nobel Prize in Physics (Swedish: Nobelpriset i fysik) is a yearly award given by the Royal Swedish Academy of Sciences for those who have made the most outstanding contributions for humankind in the field of physics.': 0.012666669340240804, 'It is one of the five Nobel Prizes established by the will of Alfred Nobel in 1895 and awarded since 1901, the others being the Nobel Prize in Chemistry, Nobel Prize in Literature, Nobel Peace Prize, and Nobel Prize in Physiology or Medicine.': -0.06928882415531908, 'Physics is traditionally the first award presented in the Nobel Prize ceremony.': -0.10829123054860297} bc
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
Some characters could not be decoded, and were replaced with REPLACEMENT CHARACTER.
{'The Nobel Prize in Physics (Swedish: Nobelpriset i fysik) is a yearly award given by the Royal Swedish Academy of Sciences for those who have made the most outstanding contributions for humankind in the field of physics.': -0.032959514849797276, 'It is one of the five Nobel Prizes established by the will of Alfred Nobel in 1895 and awarded since 1901, the others being the Nobel Prize in Chemistry, Nobel Prize in Literature, Nobel Peace Prize, and Nobel Prize in Physiology or Medicine.': -0.010435877863704418, 'Physics is traditionally the first award presented in the Nobel Prize ceremony.': -0.024178564866869968} quillbot
{"“We’re not early, mid, or late stage venture capital, we’re 'Exit Stage,'” said Paul Burgon, Managing Partner of new Provo-based investment company, Exit Ventures.": -0.027395081354180565, 'Burgon was previously the CEO of the Utah company Vortechs (a company previously covered by TechBuzz), focused on bringing plastic recycling to Utah Valley and the rest of the world.': 0.005064547078286234, 'He sold the company last year and recently launched Exit Ventures with a business partner.': 0.02052684359081724, 'Burgon has been a CVC (Corporate Venture Capital) and corporate M&A investor for most of his career, funding 500+ startups and investing over $3.1 billion as a corporate/strategic investor.': 0.04338634149886007, 'He has closed dozens of M&A transactions to create/expand multiple multi-million dollar platforms including electronics testing, water quality, dental equipment, motion control, and aerospace & defense.': 0.012800786271533615} bc
{'Tonight was nothing short of extraordinary at the prestigious Pillar of the Valley gala, as we came together to pay homage to the indomitable spirit of Gail Miller and her illustrious family.': -0.0032458497962699288, "It was an enchanting evening filled with warmth, gratitude, and an overwhelming sense of admiration for the remarkable contributions they've made to our beloved community.": 0.02009385924409125, 'Their unwavering dedication and philanthropic endeavors have truly sculpted the landscape of our society, leaving an indelible mark that will resonate for generations to come.': 0.013461695623338694, 'It was an honor to be part of such a momentous occasion, celebrating the the boundless power of generosity.': 0.015216925750789142} bc
{'Tonight was nothing short of extraordinary at the prestigious Pillar of the Valley gala, as we came together to pay homage to the indomitable spirit of Gail Miller and her illustrious family.': -0.17391504105937, "It was an enchanting evening filled with warmth, gratitude, and an overwhelming sense of admiration for the remarkable contributions they've made to our beloved community.": 0.13478819830671743, 'Their unwavering dedication and philanthropic endeavors have truly sculpted the landscape of our society, leaving an indelible mark that will resonate for generations to come.': -0.03948787785996315, 'It was an honor to be part of such a momentous occasion, celebrating the the boundless power of generosity.': 0.21453848755823973} quillbot
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
Traceback (most recent call last):
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/queueing.py", line 522, in process_events
    response = await route_utils.call_process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1689, in process_api
    result = await self.call_function(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1255, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/aliasgarov/copyright_checker/venv/lib/python3.10/site-packages/gradio/utils.py", line 750, in wrapper
    response = f(*args, **kwargs)
  File "/home/aliasgarov/copyright_checker/analysis.py", line 71, in depth_analysis
    entity_ratio = entity_density(input_text, nlp)
  File "/home/aliasgarov/copyright_checker/writing_analysis.py", line 59, in entity_density
    return len(doc.ents) / len(doc)
ZeroDivisionError: division by zero
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
Original BC scores: AI: 0.9999804496765137, HUMAN: 1.9520000932971016e-05
Calibration BC scores: AI: 0.9622641509433962, HUMAN: 0.037735849056603765
Input Text: sThis thesis addresses the challenge of enhancing the performance of vision-language retrieval systems for low-resource languages. While existing models like CLIP demonstrate robust capabilities in high-resource environments, they often falter when applied to languages with sparse data. We introduce a novel framework that adapts multimodal vision-language models to effectively process and retrieve information across diverse linguistic contexts. The framework integrates advanced techniques such as machine translation, and lightweight transformers to generate synthetic datasets in low-resource languages, which are crucial for training. Our methodology involves a comparative analysis of various encoder models, emphasizing cost-effective training strategies without compromising on computational efficiency. Experiments conducted demonstrate that our adapted models achieve significant improvements in retrieval accuracy. This thesis enhances the field of multimodal vision-language retrieval systems for under-resourced languages by adapting the typically resource-heavy CLIP models for use with Azerbaijani, a language with limited computational resources. This adaptation involves customizing transformer architectures and implementing memory-efficient training methods, which dramatically reduce computational and memory demands while maintaining high performance levels. Additionally, this work provides a detailed methodology for adapting these technologies to other low-resource languages. It clearly outlines the steps for modifying base models to meet specific linguistic and domain requirements, ensuring that the system is effectively tailored to different settings. By making our configurations and code publicly available, this thesis enables other researchers to replicate and extend our approach, broadening the application of multimodal vision-language technologies across diverse linguistic landscapes. /s
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'Grammar Enhancer']
Original BC scores: AI: 0.9999804496765137, HUMAN: 1.9520000932971016e-05
Calibration BC scores: AI: 0.9622641509433962, HUMAN: 0.037735849056603765
Starting MC
MC Score: {'OpenAI GPT': 0.9622641504876508, 'Mistral': 4.0081573065151293e-11, 'CLAUDE': 8.938057836793557e-11, 'Gemini': 2.0656532292481258e-10, 'Grammar Enhancer': 1.1971809701430604e-10}
Original BC scores: AI: 0.9996999502182007, HUMAN: 0.00030010007321834564
Calibration BC scores: AI: 0.8490566037735849, HUMAN: 0.15094339622641506
Input Text: sThis thesis addresses the challenge of enhancing the performance of vision-language retrieval systems for low-resource languages. While existing models like CLIP demonstrate robust capabilities in high-resource environments, they often falter when applied to languages with sparse data. We introduce a novel framework that adapts multimodal vision-language models to effectively process and retrieve information across diverse linguistic contexts. The framework integrates advanced techniques such as machine translation, and lightweight transformers to generate synthetic datasets in low-resource languages, which are crucial for training. Our methodology involves a comparative analysis of various encoder models, emphasizing cost-effective training strategies without compromising on computational efficiency. Experiments conducted demonstrate that our adapted models achieve significant improvements in retrieval accuracy. This thesis enhances the field of multimodal vision-language retrieval systems for under-resourced languages by adapting the typically resource-heavy CLIP models for use with Azerbaijani, a language with limited computational resources. This adaptation involves customizing transformer architectures and implementing memory-efficient training methods, which dramatically reduce computational and memory demands while maintaining high performance levels. Additionally, this work provides a detailed methodology for adapting these technologies to other low-resource languages. It clearly outlines the steps for modifying base models to meet specific linguistic and domain requirements, ensuring that the system is effectively tailored to different settings. By making our configurations and code publicly available, this thesis enables other researchers to replicate and extend our approach, broadening the application of multimodal vision-language technologies across diverse linguistic landscapes. .. /s
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'Grammar Enhancer']
Original BC scores: AI: 0.9996999502182007, HUMAN: 0.00030010007321834564
Calibration BC scores: AI: 0.8490566037735849, HUMAN: 0.15094339622641506
Starting MC
MC Score: {'OpenAI GPT': 0.8490566033714566, 'Mistral': 3.536609388101585e-11, 'CLAUDE': 7.886521620700199e-11, 'Gemini': 1.8226352022777583e-10, 'Grammar Enhancer': 1.05633615012623e-10}
Original BC scores: AI: 0.9997455477714539, HUMAN: 0.0002544422750361264
Calibration BC scores: AI: 0.8490566037735849, HUMAN: 0.15094339622641506
Input Text: sThis thesis addresses the challenge of enhancing the performance of vision-language retrieval systems for low-resource languages. While existing models like CLIP demonstrate robust capabilities in high-resource environments, they often falter when applied to languages with sparse data. We introduce a novel framework that adapts multimodal vision-language models to effectively process and retrieve information across diverse linguistic contexts. The framework integrates advanced techniques such as machine translation, and lightweight transformers to generate synthetic datasets in low-resource languages, which are crucial for training. Our methodology involves a comparative analysis of various encoder models, emphasizing cost-effective training strategies without compromising on computational efficiency. Experiments conducted demonstrate that our adapted models achieve significant improvements in retrieval accuracy. This thesis enhances the field of multimodal vision-language retrieval systems for under resourced languages by adapting the typically resource-heavy CLIP models for use with Azerbaijani, a language with limited computational resources. This adaptation involves customizing transformer architectures and implementing memory-efficient training methods, which dramatically reduce computational and memory demands while maintaining high performance levels. Additionally, this work provides a detailed methodology for adapting these technologies to other low-resource languages. It clearly outlines the steps for modifying base models to meet specific linguistic and domain requirements, ensuring that the system is effectively tailored to different settings. By making our configurations and code publicly available, this thesis enables other researchers to replicate and extend our approach, broadening the application of multimodal vision-language technologies across diverse linguistic landscapes. .. /s
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'Grammar Enhancer']
Original BC scores: AI: 0.9997455477714539, HUMAN: 0.0002544422750361264
Calibration BC scores: AI: 0.8490566037735849, HUMAN: 0.15094339622641506
Starting MC
MC Score: {'OpenAI GPT': 0.84905660336483, 'Mistral': 3.521894448908252e-11, 'CLAUDE': 8.364791167016474e-11, 'Gemini': 1.808296200586307e-10, 'Grammar Enhancer': 1.0905832835325274e-10}
Original BC scores: AI: 0.9988322854042053, HUMAN: 0.0011677537113428116
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
Input Text: sThis thesis addresses the challenge of enhancing the performance of vision-language retrieval systems for low-resource languages. While existing models like CLIP demonstrate robust capabilities in high-resource environments, they often falter when applied to languages with sparse data. We introduce a novel framework that adapts multimodal vision-language models to effectively process and retrieve information across diverse linguistic contexts. The framework integrates advanced techniques such as machine translation, and lightweight transformers to generate synthetic datasets in low-resource languages, which are crucial for training. Our methodology involves a comparative analysis of various encoder models, emphasizing cost-effective training strategies without compromising on computational efficiency. Experiments conducted the demonstrate that our adapted models achieve significant improvements in retrieval accuracy. This thesis enhances the field of multimodal vision-language retrieval systems for under resourced languages by adapting the typically resource-heavy CLIP models for use with Azerbaijani, a language with limited computational resources. This adaptation involves customizing transformer architectures and implementing memory-efficient training methods, which dramatically reduce computational and memory demands while maintaining high performance levels. Additionally, this work provides a detailed methodology for adapting these technologies to other low-resource languages. It clearly outlines the steps for modifying base models to meet specific linguistic and domain requirements, ensuring that the system is effectively tailored to different settings. By making our configurations and code publicly available, this thesis enables other researchers to replicate and extend our approach, broadening the application of multimodal vision-language technologies across diverse linguistic landscapes. .. /s
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'Grammar Enhancer']
Original BC scores: AI: 0.9988322854042053, HUMAN: 0.0011677537113428116
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
Starting MC
MC Score: {'OpenAI GPT': 0.6614420059483542, 'Mistral': 2.7468719183314672e-11, 'CLAUDE': 6.551506247421843e-11, 'Gemini': 1.408843518782721e-10, 'Grammar Enhancer': 8.737004349819536e-11}
Original BC scores: AI: 0.9986097812652588, HUMAN: 0.0013902162900194526
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
Input Text: sThis thesis addresses the challenge of enhancing the performance of vision-language retrieval systems for low-resource languages. While existing models like CLIP demonstrate robust capabilities in high-resource environments, they often falter when applied to languages with sparse data. We introduce a novel framework that adapts multimodal vision-language models to effectively process and retrieve information across diverse linguistic contexts. The framework integrates advanced techniques such as machine translation, and lightweight transformers to generate synthetic datasets in low-resource languages, which are crucial for training. Our methodology involves a comparative analysis of various encoder models, emphasizing cost-effective training strategies without compromising on computational efficiency. Experiments conducted the demonstrate that our adapted models achieve significant improvements in retrieval accuracy. This thesis enhances the field of multimodal vision-language retrieval systems for under resourced languages by adapting a typically resource-heavy CLIP models for use with Azerbaijani, a language with limited computational resources. This adaptation involves customizing transformer architectures and implementing memory-efficient training methods, which dramatically reduce computational and memory demands while maintaining high performance levels. Additionally, this work provides a detailed methodology for adapting these technologies to other low-resource languages. It clearly outlines the steps for modifying base models to meet specific linguistic and domain requirements, ensuring that the system is effectively tailored to different settings. By making our configurations and code publicly available, this thesis enables other researchers to replicate and extend our approach, broadening the application of multimodal vision-language technologies across diverse linguistic landscapes. .. /s
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'Grammar Enhancer']
Original BC scores: AI: 0.9986097812652588, HUMAN: 0.0013902162900194526
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
Starting MC
MC Score: {'OpenAI GPT': 0.6614420059505294, 'Mistral': 2.7797601589577552e-11, 'CLAUDE': 6.390007485578449e-11, 'Gemini': 1.388099927783187e-10, 'Grammar Enhancer': 8.855552924614072e-11}
{'This thesis addresses the challenge of enhancing the performance of vision-language retrieval systems for low-resource languages.': -0.022032804085780223, 'While existing models like CLIP demonstrate robust capabilities in high-resource environments, they often falter when applied to languages with sparse data.': -0.013539232075658832, 'We introduce a novel framework that adapts multimodal vision-language models to effectively process and retrieve information across diverse linguistic contexts.': -0.008850095600076838, 'The framework integrates advanced techniques such as machine translation, and lightweight transformers to generate synthetic datasets in low-resource languages, which are crucial for training.': -0.001126126307431862, 'Our methodology involves a comparative analysis of various encoder models, emphasizing cost-effective training strategies without compromising on computational efficiency.': 0.009559146105111271, 'Experiments conducted the demonstrate that our adapted models achieve significant improvements in retrieval accuracy.': -0.02109800482142602, 'This thesis enhances the field of multimodal vision-language retrieval systems for under resourced languages by adapting a typically resource-heavy CLIP models for use with Azerbaijani, a language with limited computational resources.': -0.03558557401150948, 'This adaptation involves customizing transformer architectures and implementing memory-efficient training methods, which dramatically reduce computational and memory demands while maintaining high performance levels.': 0.02043055115893942, 'Additionally, this work provides a detailed methodology for adapting these technologies to other low-resource languages.': 0.009171094810027019, 'It clearly outlines the steps for modifying base models to meet specific linguistic and domain requirements, ensuring that the system is effectively tailored to different settings.': -0.02269609733901005, 'By making our configurations and code publicly available, this thesis enables other researchers to replicate and extend our approach, broadening the application of multimodal vision-language technologies across diverse linguistic landscapes...': -0.01883132254427542} bc
Original BC scores: AI: 0.9975274205207825, HUMAN: 0.002472545485943556
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
WARNING:  Invalid HTTP request received.
WARNING:  Invalid HTTP request received.
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
/home/aliasgarov/copyright_checker/predictors.py:259: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  probas = F.softmax(tensor_logits).detach().cpu().numpy()
WARNING:  Invalid HTTP request received.
Input Text: sThis thesis addresses the challenge of enhancing the performance of vision-language retrieval systems for low-resource languages. While existing models like CLIP demonstrate robust capabilities in high-resource environments, they often falter when applied to languages with sparse data. We introduce a novel framework that adapts multimodal vision-language models to effectively process and retrieve information across diverse linguistic contexts. The framework integrates advanced techniques such as machine translation, and lightweight transformers to generate synthetic datasets in low-resource languages, which are crucial for training. Our methodology involves a comparative analysis of various encoder models, emphasizing cost-effective training strategies without compromising on computational efficiency. Experiments conducted the demonstrate that our adapted models achieve significant improvements in retrieval accuracy. This thesis enhances the field of multimodal vision-language retrieval systems for under resourced languages by adapting a typically resource-heavy CLIP models for use with Azerbaijani, a language with limited computational resources. This adaptation involves customizing transformer architectures and implementing memory-efficient training methods, which reduce computational and memory demands while maintaining high performance levels. Additionally, this work provides a detailed methodology for adapting these technologies to other low-resource languages. It clearly outlines the steps for modifying base models to meet specific linguistic and domain requirements, ensuring that the system is effectively tailored to different settings. By making our configurations and code publicly available, this thesis enables other researchers to replicate and extend our approach, broadening the application of multimodal vision-language technologies across diverse linguistic landscapes. .. /s
Models to Test: ['OpenAI GPT', 'Mistral', 'CLAUDE', 'Gemini', 'Grammar Enhancer']
Original BC scores: AI: 0.9975274205207825, HUMAN: 0.002472545485943556
Calibration BC scores: AI: 0.6614420062695925, HUMAN: 0.3385579937304075
Starting MC
MC Score: {'OpenAI GPT': 0.6614420059482446, 'Mistral': 2.7920614083030055e-11, 'CLAUDE': 6.29600495648708e-11, 'Gemini': 1.37968494059753e-10, 'Grammar Enhancer': 9.249861160750203e-11}
{'This thesis addresses the challenge of enhancing the performance of vision-language retrieval systems for low-resource languages.': -0.0223993784479603, 'While existing models like CLIP demonstrate robust capabilities in high-resource environments, they often falter when applied to languages with sparse data.': -0.015338944725661599, 'We introduce a novel framework that adapts multimodal vision-language models to effectively process and retrieve information across diverse linguistic contexts.': -0.0077758584511692505, 'The framework integrates advanced techniques such as machine translation, and lightweight transformers to generate synthetic datasets in low-resource languages, which are crucial for training.': -0.000431512871781027, 'Our methodology involves a comparative analysis of various encoder models, emphasizing cost-effective training strategies without compromising on computational efficiency.': 0.006743625380536846, 'Experiments conducted the demonstrate that our adapted models achieve significant improvements in retrieval accuracy.': -0.022862481288874203, 'This thesis enhances the field of multimodal vision-language retrieval systems for under resourced languages by adapting a typically resource-heavy CLIP models for use with Azerbaijani, a language with limited computational resources.': -0.036494040198384196, 'This adaptation involves customizing transformer architectures and implementing memory-efficient training methods, which reduce computational and memory demands while maintaining high performance levels.': 0.02177353263451164, 'Additionally, this work provides a detailed methodology for adapting these technologies to other low-resource languages.': 0.012405979561028763, 'It clearly outlines the steps for modifying base models to meet specific linguistic and domain requirements, ensuring that the system is effectively tailored to different settings.': -0.022644418003719777, 'By making our configurations and code publicly available, this thesis enables other researchers to replicate and extend our approach, broadening the application of multimodal vision-language technologies across diverse linguistic landscapes...': -0.017087079499633357} bc
{'Founded in 1899 by a group of Swiss, Catalan, German, and English footballers led by Joan Gamper, the club has become a symbol of Catalan culture and Catalanism, hence the motto "Més que un club" ("More than a club").': 0.003235688081863714, '[2] Unlike many other football clubs, the supporters own and operate Barcelona.': -0.14938091290909186, "It is the third-most valuable football club in the world, worth $5.51 billion, and the world's fourth richest football club in terms of revenue, with an annual turnover of €800.1 million.": 0.3658677971047907, '[3][4] The official Barcelona anthem is the "Cant del Barça", written by Jaume Picas and Josep Maria Espinàs.': -0.23088013599360915, '[5] Barcelona traditionally play in dark shades of blue and garnet stripes, hence nicknamed Blaugrana.': -0.36542606113642334} bc
{'Founded in 1899 by a group of Swiss, Catalan, German, and English footballers led by Joan Gamper, the club has become a symbol of Catalan culture and Catalanism, hence the motto "Més que un club" ("More than a club").': 0.38582236484888827, '[2] Unlike many other football clubs, the supporters own and operate Barcelona.': 0.2606849287384725, "It is the third-most valuable football club in the world, worth $5.51 billion, and the world's fourth richest football club in terms of revenue, with an annual turnover of €800.1 million.": 0.060964775302539256, '[3][4] The official Barcelona anthem is the "Cant del Barça", written by Jaume Picas and Josep Maria Espinàs.': 0.08375754673911556, '[5] Barcelona traditionally play in dark shades of blue and garnet stripes, hence nicknamed Blaugrana.': -0.05391279244127709} quillbot
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)

correcting text..:   0%|          | 0/1 [00:00<?, ?it/s]
correcting text..: 100%|██████████| 1/1 [00:01<00:00,  1.00s/it]
correcting text..: 100%|██████████| 1/1 [00:01<00:00,  1.01s/it]

correcting text..:   0%|          | 0/1 [00:00<?, ?it/s]
correcting text..: 100%|██████████| 1/1 [00:00<00:00,  1.49it/s]
correcting text..: 100%|██████████| 1/1 [00:00<00:00,  1.49it/s]

correcting text..:   0%|          | 0/1 [00:00<?, ?it/s]
correcting text..: 100%|██████████| 1/1 [00:00<00:00,  1.15it/s]
correcting text..: 100%|██████████| 1/1 [00:00<00:00,  1.15it/s]

correcting text..:   0%|          | 0/1 [00:00<?, ?it/s]
correcting text..: 100%|██████████| 1/1 [00:00<00:00,  1.45it/s]
correcting text..: 100%|██████████| 1/1 [00:00<00:00,  1.45it/s]
/usr/lib/python3/dist-packages/requests/__init__.py:87: RequestsDependencyWarning: urllib3 (2.2.1) or chardet (4.0.0) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "