Jerry Liu (@jerryjliu0)

Today, @llama_index is an applied research lab for document OCR and document work. Our core focus is investing in building the best-in-class models+ agents at the pareto frontier of accuracy/cost/latency for document processing. To do this, we have to invest in all the different components around the research infra stack: 1. Building comprehensive and fair benchmarks. Today we have ParseBench, ExtractBench, and ExtractBench Pro. They're already the most popular benchmarks in parsing and extraction today, and we're just getting started. 2. Investing in data curation and synthetic data generation. 3. Investing in compute and our training pipeline. This has produced models like OCRia-1. 4. Investing in agent optimization - tune everything around the models to further hillclimb cost and accuracy. 5. Closing the loop based on feedback - whether from active usage of our product, implicit/explicit customer feedback, or automated verification. We are accelerating our capabilities at a crazy pace. Today, our work is documented in our research page: There is a lot more to come!

i used to do this a lot. but then i felt like i was getting progressively dumber at writing. i think there's some value in forcing your own brain to sharpen your thinking, instead of offloading it to AI what i found helpful is to manually type out my own understanding in distilled bullet points after reading the LLM output

We built the fastest PDF -> markdown parser in the world 🚀⚡️ AND it’s more accurate than any other open-source, model-free parser (pymupdf4llm, opendataloader, pdf-inspector, markitdown) on 3 standardized benchmarks: olmOCR0-bench, opendataloader-bench, ParseBench Introducing LiteParse v2.1. The v2 base version was already the fastest document->text parser on the planet, and with this new release we’ve introduced markdown. It is fully open-source (Apache 2.0) and free, is usable from CLI/Rust/Node/Python/WASM, and is also installable as a one-click agent skill. Check it out: Come check out LiteParse: