Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:2405.07883

Tokenizer Adaptation

Collection of research on tokenizers' adaptation to specific domains and/or languages. Special focus on sequence compression directions

Fast Vocabulary Transfer for Language Model Compression

Paper • 2402.09977 • Published Feb 15 • 2
Multi-Word Tokenization for Sequence Compression

Paper • 2402.09949 • Published Feb 15
Zero-Shot Tokenizer Transfer

Paper • 2405.07883 • Published May 13 • 4
Language Model Tokenizers Introduce Unfairness Between Languages

Paper • 2305.15425 • Published May 17, 2023 • 1

Papers-Fundamentals

RoFormer: Enhanced Transformer with Rotary Position Embedding

Paper • 2104.09864 • Published Apr 20, 2021 • 10
Attention Is All You Need

Paper • 1706.03762 • Published Jun 12, 2017 • 44
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences

Paper • 2404.03715 • Published Apr 4 • 60
Zero-Shot Tokenizer Transfer

Paper • 2405.07883 • Published May 13 • 4

Large Language Model (LLM) and NLP related papers.

LoRA+: Efficient Low Rank Adaptation of Large Models

Paper • 2402.12354 • Published Feb 19 • 6
The FinBen: An Holistic Financial Benchmark for Large Language Models

Paper • 2402.12659 • Published Feb 20 • 16
TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization

Paper • 2402.13249 • Published Feb 20 • 10
TrustLLM: Trustworthiness in Large Language Models

Paper • 2401.05561 • Published Jan 10 • 64

LLaMA Beyond English: An Empirical Study on Language Capability Transfer

Paper • 2401.01055 • Published Jan 2 • 54
YAYI 2: Multilingual Open-Source Large Language Models

Paper • 2312.14862 • Published Dec 22, 2023 • 13
Order Matters in the Presence of Dataset Imbalance for Multilingual Learning

Paper • 2312.06134 • Published Dec 11, 2023 • 2
TaCo: Enhancing Cross-Lingual Transfer for Low-Resource Languages in LLMs through Translation-Assisted Chain-of-Thought Processes

Paper • 2311.10797 • Published Nov 17, 2023

Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation

Paper • 2310.05737 • Published Oct 9, 2023 • 4
SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models

Paper • 2308.16692 • Published Aug 31, 2023 • 1
Towards General Text Embeddings with Multi-stage Contrastive Learning

Paper • 2308.03281 • Published Aug 7, 2023 • 1
ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings

Paper • 2305.11554 • Published May 19, 2023 • 2

Company

© Hugging Face

TOS Privacy About Jobs

Website

Models Datasets Spaces Pricing Docs