facebook/bart-large-mnli · Need help speeding up headline categorization

Hi all,

I’m working on a project that needs to categorize 300 headlines into 9-16 dynamic categories every hour. I'm using the BART model via Huggingface's API. My current implementation in Python takes 3-5 seconds per headline, which is too slow.

def categorise(categories, item):
    API_URL = "https://api-inference.huggingface.co/models/facebook/bart-large-mnli"
    headers = {"Authorization": "Bearer <token>"}
    payload = {
        "inputs": item,
        "parameters": {"candidate_labels": categories},
    }
    response = requests.post(API_URL, headers=headers, json=payload)
    if response.status_code == 200:
        results = response.json()
        return {"category": results[0]['labels'][0], "confidence": results[0]['scores'][0]}
    else:
        return {"error": f"API request failed with status {response.status_code}"}

I call this function in a for loop for each headline. Is there a way to make this faster?