Will rwkv6-pile be released in the future?

#1
by gongxunyao - opened

I want to latest version of rwkv pile to train a horror story, but it seems that v4 is less excellent than v6

This model is trained more than Pile. RWKV-6-Pile will be weaker than it.

@ZhangRC But I want a pure model for training a novel gpt

  1. This is RWKV, not GPT.
  2. "novel GPT" is ambiguous: Did you mean a GPT optimized for narrative stories, or just a new GPT? In either case, RWKV-6-world should fit your requirements.

In general performance: v6 3B > v4 7B
You can finetune this model on stories, it's a base model

Sign up or log in to comment