---
title: De-DSI
emoji: 👁
colorFrom: purple
colorTo: green
sdk: gradio
sdk_version: 4.26.0
app_file: app.py
pinned: false
arxiv: 2404.12237
license: gpl-3.0
short_description: Generative retrieval for URLs, magnet links, and BTC wallets
---

# De-DSI

Generative AI is increasingly influencing fields such as content discovery, relevance ranking, and financial transactions, showcasing its potential to disrupt various industries. 
The novel end-to-end generative architectures could pave the way for fully decentralized alternatives in social media, the movie industry, search engines, and financial sectors—mirroring the decentralization levels of Bitcoin and BitTorrent. 
This shift could significantly empower ordinary Internet users.
Explore the scientific foundation of this transformation in our paper presented at EuroMLSys 2024. 
The paper is available [here](https://huggingface.co/papers/2404.12237).

We invite you to contribute to and engage with our community at the International Workshop on Distributed Infrastructure for Common Good (DICG).


## Demo

For this demo, we trained an end-to-end generative Transformer on a small dataset (526 records) that comprises YouTube URLs, magnet links, and Bitcoin wallet addresses.
Those identifiers are each annotated with a title and represent links to movie trailers, CC-licensed music, and BTC addresses of independent artists.
Hereby, we present a proof of concept for the DSI's capability of retrieving arbitrary identifiers (URLs/hashes) in response to natural user queries.

The model is available under a permissive license and can be accessed [here](https://huggingface.co/tribler/dsi-search-on-toy-dataset).

### Please Note

This project represents both a groundbreaking advance and a preliminary exploration into decentralized systems. 
As a preliminary model, the project showcases a toy example rather than the full potential of its ultimate capabilities.
It serves as a proof of concept that invites further development and imagination.