README.md

---
license: apache-2.0
---
# Cross-Encoder for MS MARCO - EN-DE (ONNX)

This repository hosts the ONNX format of the cross-lingual Cross-Encoder model for EN-DE, originally developed for passage re-ranking. It's based on the model trained using the [MS Marco Passage Ranking](https://github.com/microsoft/MSMARCO-Passage-Ranking) dataset.

The ONNX version maintains the efficiency and compatibility required for deployment in various environments, facilitating easier integration with platforms supporting ONNX models.

For the original model and further details, refer to [cross-encoder/msmarco-MiniLM-L12-en-de-v1](https://huggingface.co/cross-encoder/msmarco-MiniLM-L12-en-de-v1) on Hugging Face.

## Application
This model can be used for information retrieval tasks. For usage examples, see [SBERT.net Retrieve & Re-rank](https://www.sbert.net/examples/applications/retrieve_rerank/README.html).

## Training Code
The training script used for the original model is available in this repository, see `train_script.py`.

## How to Use in ONNX
To load and use the model in ONNX format, ensure you have the appropriate ONNX runtime installed in your environment. For example, we run this model using Metarank.

## Model Evaluation

### Best Re-ranker for German Texts

Based on an internal evaluation conducted by our development team, the cross-encoder/msmarco-MiniLM-L12-en-de-v1 model has been identified as one of the most effective re-rankers for German texts.

#### Evaluation Context
The primary goal was to enhance chatbot response accuracy and improve document relevance in response to user queries. Reranking is vital as it enhances the search results by placing the most relevant documents at the top, ensuring the chatbot delivers the most accurate responses based on user inputs.

#### Decision and Reasoning
After comprehensive testing against various models such as corrius/cross-encoder-mmarco-mMiniLMv2-L12-H384-v1 and multiple mixedbread-ai variants, the cross-encoder/msmarco-MiniLM-L12-en-de-v1 model demonstrated superior performance in processing and reranking German-language texts. It proved to be highly efficient in terms of accuracy and computational performance during benchmark tests involving complex query responses.

#### Advantages
- **Improved Accuracy**: Leads to more precise responses from the chatbot, significantly enhancing user experience.
- **Market Relevance**: Offers a competitive edge in German-speaking markets by effectively managing complex queries.

# Integrating an ONNX Model with Metarank for German Text Re-ranking

## Setup Local Metarank Instance

To run Metarank locally, you need to create a `config.yml` file and set up a Docker container. Below are the detailed steps:

### Step 1: Create Configuration File

Create a `config.yml` file in the root of your Metarank project:

```yaml
inference:
  msmarco:
    type: cross-encoder
    model: UlanYisaev/msmarco-MiniLM-L12-en-de-v1
```

### Step 2: Run Metarank using Docker
```bash
cd metarank
docker run --name metarank -p 8080:8080 -v $(pwd):/opt/metarank metarank/metarank:latest serve --config /opt/metarank/config.yml
```

### Step 3: Configure Metarank Docker Image to Include config.yml
Adjust the Docker configuration to include the config.yml in the build:
```bash
# Modify build.sbt in Metarank to include the custom script and config
new Dockerfile {
    add(new File("deploy/metarank_custom.sh"), "/metarank.sh")
    add(new File("config.yml"), "/opt/metarank/config.yml")
    entryPoint("/metarank.sh")
    cmd("--help")
}

# Build the Docker image
sbt docker
```

deploy/metarank_custom.sh:
```bash
#!/bin/bash

set -euxo pipefail
OPTS=${JAVA_OPTS:-"-Xmx1700m -verbose:gc"}

exec /usr/bin/java $OPTS -cp "/app/*" ai.metarank.main.Main serve --config /opt/metarank/config.yml
```

### Step 4: Run and Manage the Docker Container
```bash
docker run --name metarank -p 8080:8080 metarank/metarank:0.7.8-amd64
docker stop metarank
docker start metarank
```