ArXiv Query

[{"_1":2,"_128":-5,"_129":-5},"loaderData",{"_3":4,"_125":126},"root",{"_5":6,"_7":8,"_15":16,"_46":47,"_80":81,"_87":88,"_118":-5,"_119":45,"_120":121,"_122":123,"_124":42},"isBlog",true,"tenant",{"_9":10,"_11":12,"_13":14},"type","subdomain","identifier","pdf-vector","projectId","pdfvector","site",{"_17":14,"_18":19,"_20":19,"_21":22,"_23":24,"_25":-5,"_26":27,"_28":29,"_30":-5,"_31":32,"_33":34,"_39":-5,"_40":-5,"_41":42,"_43":44},"id","name","PDF Vector","businessName","description","PDF Vector is an AI powered document processing and academic search platform that provides a unified API for parsing PDFs, Word files, Excel spreadsheets, images, and invoices into clean text or structured data. It also lets developers and no code users ask questions about documents, extract custom fields, and search or fetch over 5 million academic papers from multiple research databases to power RAG systems, research tools, and document centric applications.","favicon","https://pdfvector.com/favicon.ico","logo","brandColor","#339AF0","url","https://pdfvector.com","publicBlogTitle","publicBlogDescription","Fast, accurate PDF/Word/Image/Excel parsing API for building reliable, scalable document automation and AI workflows","blogNavLinks",[35],{"_28":36,"_37":38},"/","label","Home","blogCtaButton","blogFont","hidePoweredBy",false,"allowedDomains",[45],"pdf-vector.firstclick.so","blogData",{"_48":49,"_49":50,"_55":56},"page","category",{"_17":51,"_18":52,"_53":54,"_21":-5},"89db66d1-f96a-4c5d-850e-a0e9f6135bf8","Developer","slug","developer","articles",[57],{"_17":58,"_59":60,"_53":61,"_62":60,"_63":64,"_65":66,"_67":68,"_69":70,"_71":72,"_73":74,"_75":76,"_77":78,"_49":79},"2ac5a344-4ab7-429c-b485-344b680139fe","title","Search ArXiv Papers Without XML Parsing Headaches","11-search-arxiv-papers-without-xml-parsing-headaches","keyword","metaDescription","Drowning in XML while exploring ArXiv for automation ideas? Learn simpler ways to find relevant papers that directly support invoice and statement workflows.","content","If you've ever tried to search ArXiv programmatically, you've probably stared at XML responses wondering why something so simple has to be so complicated. Those nested namespaces, the Atom 1.0 format, and the constant worry about whether your XML parser will handle the next edge case correctly.\n\nWe've all been there. You just want to search for papers about \"quantum computing\" and get back a nice JSON array. Instead, you're debugging xmlns attributes at 2 AM.\n\n## Understanding the ArXiv XML Challenge\n\nArXiv's API returns results in Atom 1.0 format, which made sense in 2007 when the API was designed. Today, it creates unnecessary complexity for developers who expect JSON responses from modern APIs.\n\nHere's what a typical ArXiv API response looks like:\n\n```xml\n\n\n \n ArXiv Query\n \n http://arxiv.org/abs/2301.00001v1\n 2023-01-01T00:00:00Z\n 2023-01-01T00:00:00Z\n Quantum Computing Applications\n \n John Doe\n \n \n\n```\n\nThe multiple namespaces and nested structure make parsing error-prone. Miss one namespace declaration and your entire parser breaks.\n\n## Method 1: Traditional ArXiv API with XML Parsing\n\nLet's look at the traditional approach using the ArXiv API directly:\n\n```javascript\nimport { XMLParser } from 'fast-xml-parser';\n\nasync function searchArxivTraditional(query: string) {\n const url = `http://export.arxiv.org/api/query?search_query=${encodeURIComponent(query)}&max_results=10`;\n\n const response = await fetch(url);\n const xmlText = await response.text();\n\n const parser = new XMLParser({\n ignoreAttributes: false,\n removeNSPrefix: false\n });\n\n const result = parser.parse(xmlText);\n const entries = result.feed.entry || [];\n\n // Transform to clean JSON\n return (Array.isArray(entries) ? entries : [entries]).map(entry => ({\n id: entry.id,\n title: entry.title,\n authors: Array.isArray(entry.author)\n ? entry.author.map(a => a.name)\n : [entry.author?.name],\n published: entry.published,\n summary: entry.summary\n }));\n}\n\n// Usage\nconst papers = await searchArxivTraditional(\"quantum computing\");\nconsole.log(papers);\n```\n\n**Pros:**\n\n- Direct access to ArXiv API\n- No third-party API keys needed\n- Free to use\n\n**Cons:**\n\n- Complex XML parsing logic\n- Namespace handling is fragile\n- No built-in error handling for malformed XML\n- Must handle rate limiting manually (3 second delays)\n\n## Method 2: Python ArXiv Library\n\nThe official Python library simplifies things somewhat:\n\n```python\nimport arxiv\nimport json\n\ndef search_arxiv_python(query):\n search = arxiv.Search(\n query=query,\n max_results=10,\n sort_by=arxiv.SortCriterion.SubmittedDate\n )\n\n results = []\n for result in search.results():\n results.append({\n \"id\": result.entry_id,\n \"title\": result.title,\n \"authors\": [author.name for author in result.authors],\n \"published\": result.published.isoformat(),\n \"summary\": result.summary\n })\n\n return json.dumps(results)\n\n# Usage\npapers = search_arxiv_python(\"quantum computing\")\nprint(papers)\n```\n\n**Pros:**\n\n- Official library handles XML parsing\n- Cleaner code than manual parsing\n- Automatic retry logic\n\n**Cons:**\n\n- Python-only solution\n- Still requires JSON transformation\n- Not suitable for JavaScript/TypeScript projects\n\n## Method 3: PDF Vector Academic Search API\n\nPDF Vector provides a modern alternative with native JSON responses:\n\n```javascript\nasync function searchArxivWithPDFVector(query: string) {\n const response = await fetch('https://www.pdfvector.com/v1/api/academic-search', {\n method: 'POST',\n headers: {\n 'Authorization': 'Bearer pdfvector_xxx',\n 'Content-Type': 'application/json'\n },\n body: JSON.stringify({\n query: query,\n providers: ['arxiv'],\n limit: 10,\n fields: ['title', 'authors', 'year', 'abstract', 'pdfURL']\n })\n });\n\n const data = await response.json();\n return data.results;\n}\n\n// Usage\nconst papers = await searchArxivWithPDFVector(\"quantum computing\");\nconsole.log(papers);\n\n// Clean JSON response:\n// [\\\n// {\\\n// \"title\": \"Quantum Computing Applications in Machine Learning\",\\\n// \"authors\": [\\\n// { \"name\": \"John Doe\" },\\\n// { \"name\": \"Jane Smith\" }\\\n// ],\\\n// \"year\": 2023,\\\n// \"abstract\": \"We explore the intersection of quantum computing...\",\\\n// \"pdfURL\": \"https://arxiv.org/pdf/2301.00001.pdf\"\\\n// }\\\n// ]\n```\n\nWant to search across multiple databases? Just add more providers:\n\n```javascript\nconst results = await fetch('https://www.pdfvector.com/v1/api/academic-search', {\n method: 'POST',\n headers: {\n 'Authorization': 'Bearer pdfvector_xxx',\n 'Content-Type': 'application/json'\n },\n body: JSON.stringify({\n query: \"quantum computing applications\",\n providers: ['arxiv', 'semantic-scholar', 'pubmed'],\n limit: 20\n })\n});\n```\n\n**Pros:**\n\n- Clean JSON responses with no XML parsing needed\n- Search multiple academic databases simultaneously\n- Consistent data structure across all providers\n- No rate limiting issues for reasonable usage\n- [TypeScript SDK available](https://www.npmjs.com/package/pdfvector)\n\n**Cons:**\n\n- Requires API key (free tier available with 100 credits/month)\n- [Credit-based system](https://www.pdfvector.com/#pricing) (2 credit per search)\n\n## Comparing the Approaches\n\n| Aspect | ArXiv Direct | Python Library | PDF Vector |\n| --- | --- | --- | --- |\n| Response Format | XML | Python objects | JSON |\n| Parsing Complexity | High | Medium | None |\n| Error Handling | Manual | Built-in | Built-in |\n| Multiple Databases | No | No | Yes |\n| Setup Time | 1 day | 5 minutes | 5 minutes |\n\n## Making the Right Decision\n\n**Use ArXiv Direct API when:**\n\n- You're building a one-off script or prototype\n- You have existing XML parsing infrastructure\n- You need unlimited free queries\n- You're comfortable handling XML namespaces\n- Rate limiting won't affect your use case\n\n**Use Python arxiv library when:**\n\n- You're already in a Python environment\n- You need the official implementation\n- You want built-in error handling\n- You can work within the rate limits\n- You prefer Python objects over raw XML\n\n**Use PDF Vector when:**\n\n- You want clean JSON responses without XML parsing\n- You need to search multiple academic databases\n- You value development speed over free access\n- You're building a production application\n- You need consistent data structure across providers","coverImage","https://pub-697921b125bd45e59218b6c661706939.r2.dev/projects/pdfvector/articles/2ac5a344-4ab7-429c-b485-344b680139fe/cover-1765187809458-3h0mzl.jpg","funnelStage","TOFU","status","COMPLETED","publishDate","2025-08-29T15:36:00.000Z","createdAt","2025-12-08T09:56:35.917Z","updatedAt","2025-12-08T10:41:28.445Z",{"_17":51,"_18":52,"_53":54,"_21":-5},"categories",[82,83],{"_17":51,"_18":52,"_53":54,"_21":-5},{"_17":84,"_18":85,"_53":86,"_21":-5},"13cb48d2-4566-4842-8bd1-f447f5fce875","No-Code","no-code","metaTags",[89,91,93,100,103,106,109,111,114,116],{"_59":90},"Developer - PDF Vector",{"_18":21,"_65":92},"Browse all Developer articles on PDF Vector",{"_94":95,"_96":97,"_98":99},"tagName","link","rel","canonical","href","https://www.pdfvector.com/blog/category/developer",{"_101":102,"_65":90},"property","og:title",{"_101":104,"_65":105},"og:description","Browse all Developer articles",{"_101":107,"_65":108},"og:type","website",{"_101":110,"_65":99},"og:url",{"_18":112,"_65":113},"twitter:card","summary_large_image",{"_18":115,"_65":90},"twitter:title",{"_18":117,"_65":105},"twitter:description","structuredData","host","blogHost","www.pdfvector.com","basePath","/blog","siteNotFound","catch-all",{"_127":6},"isBlogHost","actionData","errors"]