BYTEDANCE: QA TRAINING BEATS TRANSCRIPTION FOR LLMS

ByteDance researchers found that training large language models through question-answering outperforms transcription methods for processing long, image-heavy documents. A 7B model trained this way matched larger models' performance on documents four times longer than its training data.