Hugging Face's logo
Caleb Fahlgren's picture
TFLOPS icon 14.2 TFLOPS
60 28 101

Caleb Fahlgren PRO

cfahlgren1
severo's profile picture okasi's profile picture JoPmt's profile picture
·

AI & ML interests

None yet

Articles

Organizations

Hugging Face's profile picture Datasets Maintainers's profile picture Hugging Face OSS Metrics's profile picture Hugging Face TB Research's profile picture ChatDB's profile picture Cognitive Computations's profile picture nltpt-q's profile picture DuckDB Text-2-SQL Bench's profile picture

Posts 7

view post
Post
573
You can clean and format datasets entirely in the browser with a few lines of SQL.

In this post, I replicate the process @mlabonne used to clean the new microsoft/orca-agentinstruct-1M-v1 dataset.

The cleaning process consists of:
- Joining the separate splits together / add split column
- Converting string messages into list of structs
- Removing empty system prompts

https://huggingface.co/blog/cfahlgren1/the-beginners-guide-to-cleaning-a-dataset

Here's his new cleaned dataset: mlabonne/orca-agentinstruct-1M-v1-cleaned
View all posts

Collections 2