I’m testing a model for extracting information from documents, and I’m running into something strange. It works reasonably well with short pdfs, but with longer presentation files the output becomes incomplete or sometimes stops halfway through. ive tried changing the max token length, but Im not sure if that’s actually the right thing to adjust. I’m testing it with presentation files downloaded from different sources, so the file structure isn’t always exactly the same. Has anyone dealt with this kind of issue? Is it more likely to be a context-length problem, or should I be looking at the pdf extraction step before the model receives the text?
Before tuning anything, measure one number: the token count of the extracted text just before it goes to the model. That single measurement tells you which of your two theories is right, and it’s the step almost everyone skips.
On the symptom itself, “stops halfway through” usually points at the output limit rather than context length. When you exceed context, the input gets truncated, so you typically lose the beginning of the document and get confidently wrong answers about it, not a response that cuts off mid-sentence. A response that just stops is almost always max_new_tokens or a stop sequence firing. Those are different knobs and worth separating.
That said, I’d look hard at the extraction step first, because presentation PDFs are genuinely awful to extract. Slide decks come out with text boxes interleaved in arbitrary reading order, tables flattened into runs of numbers, and speaker notes mixed into body text. So the model is often getting both a long input and a scrambled one, and the scrambling is what actually breaks it. Print the extracted text for one failing deck and read it yourself before blaming the model.
If you have the original .pptx rather than only the PDF, parse that instead. python-pptx gives you shapes and text frames directly with slide boundaries intact, which is far cleaner than recovering structure from PDF text.
And for long documents generally, don’t feed the whole thing. Chunk per slide or per page, extract per chunk, then merge the results. Per-chunk extraction is more reliable, it parallelises, and when one chunk fails you can see exactly which one instead of getting a truncated answer for the whole document.