Language Models do better when they're focused.
One strategy is to pass a relevant subset (chunk) of your full data. There are many ways to chunk text.
This is a tool to understand different chunking/splitting strategies.