They Said It Couldn’t Be Done
•
76
Anastasia Stasenko
anastasiastasenko
AI & ML interests
LLM for humanities, social sciences and public good
Ethical AI
Recent Activity
upvoted
an
article
29 days ago
They Said It Couldn’t Be Done
updated
a model
29 days ago
PleIAs/Pleias-Pico
updated
a model
29 days ago
PleIAs/Pleias-Nano
Articles
Organizations
anastasiastasenko's activity
upvoted
an
article
29 days ago
Article
They Said It Couldn’t Be Done
By
•
•
76
updated
5
models
29 days ago
upvoted
an
article
about 2 months ago
Article
Releasing the largest multilingual open pretraining dataset
By
•
•
98
upvoted
an
article
2 months ago
Article
VLM Art Analysis
By
•
•
11
upvoted
an
article
3 months ago
Article
wHy DoNt YoU jUsT uSe ThE lLaMa ToKeNiZeR??
By
•
•
38
updated
a
dataset
10 months ago
updated
a
collection
10 months ago
reacted to
clem's
post with ❤️
11 months ago
Post
In 2024, we're expanding from open weights to open EVERYTHING (datasets, training scripts,...).
Excited to see this dataset release in French by @Pclanglais @carbonbasedLLM @anastasiastasenko :
PleIAs/French-PD-Newspapers
"To give you an idea of the size, the full French Wikipedia is about 2 billon words. This is 40 times larger."
Excited to see this dataset release in French by @Pclanglais @carbonbasedLLM @anastasiastasenko :
PleIAs/French-PD-Newspapers
"To give you an idea of the size, the full French Wikipedia is about 2 billon words. This is 40 times larger."
reacted to
Pclanglais's
post with ❤️
11 months ago
Post
Hi everyone,
For my first post, I'm announcing a big release (in multiple ways): probably the largest open corpus in French to date, with 85 billion words in the public domain.
The dataset has been prepared in collaboration with Benoît de Courson and Benjamin Azoulay from Gallicagram (https://shiny.ens-paris-saclay.fr/app/gallicagram). Gallicagram is a major cultural analytics project in French, the open and better version of ngram viewer for large scale search of word and ngram occurrences.
The corpus is made of two different dataset for monographs (16B words) PleIAs/French-PD-Newspapers and newspapers/periodicals (69B) PleIAs/French-PD-Newspapers Along with the full text it also includes core provenance metadata.
Beyond research in digital humanities, the corpus can also be used to training open and reproducible LLMs. Being in the public domain means it can be released everywhere in any shape without restrictions.
The corpus is not perfect: digitization of cultural heritage is challenging and, especially for newspapers, we tackle with layout issues and a significant rate of optical character recognition mistake. Our conviction is that releasing corpus as a commons is the best way to improve on this. Sharing is caring.
For my first post, I'm announcing a big release (in multiple ways): probably the largest open corpus in French to date, with 85 billion words in the public domain.
The dataset has been prepared in collaboration with Benoît de Courson and Benjamin Azoulay from Gallicagram (https://shiny.ens-paris-saclay.fr/app/gallicagram). Gallicagram is a major cultural analytics project in French, the open and better version of ngram viewer for large scale search of word and ngram occurrences.
The corpus is made of two different dataset for monographs (16B words) PleIAs/French-PD-Newspapers and newspapers/periodicals (69B) PleIAs/French-PD-Newspapers Along with the full text it also includes core provenance metadata.
Beyond research in digital humanities, the corpus can also be used to training open and reproducible LLMs. Being in the public domain means it can be released everywhere in any shape without restrictions.
The corpus is not perfect: digitization of cultural heritage is challenging and, especially for newspapers, we tackle with layout issues and a significant rate of optical character recognition mistake. Our conviction is that releasing corpus as a commons is the best way to improve on this. Sharing is caring.
- 1 reply
liked
a
dataset
12 months ago
liked
a
model
12 months ago
liked
a
model
about 1 year ago
