Multimodal document understanding
Reading and extracting information from visually rich documents with multimodal LLMs, including document layout, tables, charts and multi-page, multi-document question answering.
Hi, I'm
Debanjan Mahata
Senior Machine Learning Research Engineer, Bloomberg AI, New York
I work on multimodal document understanding and agentic AI: systems that retrieve, read and reason over large collections of complex documents. My background is in natural language processing and information retrieval.
About me
I am a Senior Machine Learning Research Engineer in Bloomberg AI in New York, where I am part of the AI Search team and work on multimodal document understanding: retrieval, extraction and agentic reasoning over text-heavy and visually rich documents such as filings, reports, presentations, tables and charts.
On AI Search, I work on multimodal document retrieval; our research on this is published as MIDR (EMNLP 2026). Previously, I was a member of the core agent team behind ASKB, the agentic conversational interface on the Bloomberg Terminal, where I worked on conversation memory and context management and on ASKB Workflows. I am interested in vision-based retrievers for enterprise-scale Document AI and in memory systems that help agents manage context across long interactions.
I enjoy taking research all the way to production. I work with engineers, researchers, product managers and financial analysts, and mentor research interns. I also collaborate closely with Prof. Hamed Zamani (UMass Amherst, CIIR) and Prof. Mohit Bansal (UNC Chapel Hill). Before this I led the AI organization for Moody's Analytics' KYC business, was a Senior Research Scientist in Bloomberg AI, and was a Senior Research Associate at Infosys in Palo Alto. I have also served as Adjunct Faculty at IIIT-Delhi. Earlier research covered keyphrase extraction, summarization and computational social science.
I hold a PhD in Information Science from the University of Arkansas at Little Rock, and BS and MS degrees in Computer Science from Banaras Hindu University, India. I speak Bengali, Hindi and English, and a little Sanskrit. Outside work I play the tabla and cricket, and enjoy music, food and travel.
Recent updates
-
2026
MIDR: Enrichment-Augmented Indexing for Multimodal Document Retrieval accepted at the EMNLP 2026 Main Conference.
-
2026
Serving as Area Chair for EACL 2027, in both the main track and the Industry Track.
Current research
Retrieval & RAG
Multimodal and vision-based retrieval, enrichment-augmented indexing, hybrid sparse and dense retrieval, and retrieval-augmented generation over large document collections.
Agentic AI & memory
Agents that plan and act over research workflows, tool use and MCP, and conversation memory and context management for long user–assistant interactions.
Experience
Senior Machine Learning Research Engineer @ Bloomberg AI
- AI Search: research on multimodal document retrieval, including MIDR (EMNLP 2026), an enrichment-augmented indexing approach that uses multimodal LLMs to turn document pages into structured text fields combined with dense retrieval.
- ASKB: previously a core member of the agent team behind ASKB on the Bloomberg Terminal. Owned conversation memory and context management, and helped launch ASKB Workflows, which turn multi-step research tasks into structured outputs.
- Document intelligence: designed an end-to-end RAG pipeline for extracting structured financial information from multimodal documents, working with a team of ML engineers alongside engineering, UI, product and financial analysts.
- Research: research on multimodal document understanding published at EMNLP, ICCV and ACL-family venues; mentor research interns; collaborate closely with Prof. Hamed Zamani (UMass Amherst, CIIR) on information retrieval and Prof. Mohit Bansal (UNC Chapel Hill) on multimodal document understanding.
Director, Machine Learning @ Moody's Analytics
- Led the AI organization for Moody's KYC (Know Your Customer) business, delivering production solutions including Compliance Catalyst and AI Review.
- Designed the ML infrastructure: model versioning (DVC), training lifecycle (MLflow), deployment (FastAPI on Kubernetes) and CI/CD (CML and GitHub Actions).
- Migrated legacy NLP pipelines to multilingual transformer models for detecting risk-relevant news, and added ELK observability for 600+ models on AWS EKS.
- Interviewed for Hugging Face's Director of Machine Learning Insights (Finance Edition).
Senior Research Scientist @ Bloomberg AI
- Designed, implemented and trained the open-source KBIR and KeyBART language models, using new pre-training objectives for keyphrase representation learning (NAACL 2022; model on Hugging Face).
- Delivered production entity-sentiment and topic-summarization models for Bloomberg Event-Driven Feeds and the Bloomberg Terminal.
- Worked with domain experts to turn news and media use cases into concrete ML problems and deployed solutions; research on keyphrases, citation worthiness and conversation summarization.
Senior Research Associate @ Infosys
- R&D team of the Infosys Information Platform.
- Developed a patented system for ranked keyword extraction from unstructured text (US 11,334,608 B2), the basis of Key2Vec. Received the Infosys Award for Innovation (2017).
- Led Solr-based search improvements: indexing, query expansion with dense embeddings and learned ranking.
- Built internal NLP microservices and a Python backend on spaCy, and mentored research interns.
Adjunct Faculty @ IIIT-Delhi
- Collaborated with the MIDAS Lab and advised students on NLP, speech and multimodal research.
- Co-authored work on code-switched and offensive language, mental health and harassment disclosures on social media, visual speech recognition, keyphrase extraction and citation text generation.
Graduate roles @ University of Arkansas at Little Rock
- Graduate Assistant, Center of Innovation and Commercialization (Jan 2013 – Aug 2015): assessed new inventions and startup ideas, identified markets, and helped faculty, students and staff commercialize their innovations.
- Graduate Teaching Assistant (Aug – Dec 2012): taught and graded HTML, CSS and JavaScript for an undergraduate class of 32 students.
- Graduate Research Assistant (Aug 2011 – Aug 2012): built social media crawlers and performed social network analysis, data mining and sentiment analysis.
Data Science Intern @ MyCityWay
- Built core analytics components for personalization and contextual ad targeting.
- Improved data quality and sentiment-driven ranking for mobile apps; worked on the early MobileROI platform.
Programmer Analyst @ Cognizant Technology Solutions
- Developed Java and COBOL components implementing new business logic for an insurance product suite.
Talks & resources
Datasets, models and code from my work, and talks I have given.
Datasets
-
M3DocVQA
Open-domain document VQA benchmark over 3,000+ PDFs and 40,000+ pages (ICCV 2025).
-
LDKP: Long Document Keyphrases
Keyphrase identification from long scientific documents: LDKP3K (~100K papers) and LDKP10K (~1.3M papers) with full text and metadata (DL4SR @ CIKM 2022).
-
GupShup
Hindi-English (Hinglish) code-switched conversational dataset of 6,831 conversations with summaries in English and Hinglish (EMNLP 2021). The dataset is available on request from Laiba Mehnaz.
-
Citation Worthiness of Sentences in Scholarly Articles
Over 2.7 million sentences from ACL Anthology papers labelled for whether they need a citation (NAACL-HLT 2021).
-
Transport Complaint Data
3,700 tweets about transport, annotated as complaints or non-complaints (AAAI 2020).
-
MeTooMA
9,973 tweets about the MeToo movement annotated for relevance, stance, hate speech, sarcasm and dialogue acts (ICWSM 2020).
-
Keyphrase Extraction using Contextual Embeddings
Data for Keyphrase Extraction from Scholarly Articles as Sequence Labeling using Contextualized Embeddings (ECIR 2020).
-
BHAAV
The first and largest Hindi corpus for emotion analysis in stories: 20,304 sentences from 230 short stories across 18 genres, labelled anger, joy, suspense, sad or neutral.
-
Hindi Discourse
10,472 sentences from 53 stories by 11 20th-century authors, labelled with discourse modes: argumentative, descriptive, dialogic, informative and narrative (LREC 2020).
-
Hindi NLI Data
The first recast natural language inference datasets for Hindi, built from three affective text classification datasets: 43K, 17K and 203K premise–hypothesis pairs (AACL-IJCNLP 2020).
Models & code
-
KBIR & KeyBART
Language models pre-trained for keyphrase extraction (KBIR) and keyphrase generation (KeyBART), from Learning Rich Representation of Keyphrases from Text (Findings of NAACL-HLT 2022).
-
M3DocRAG
Multimodal RAG framework for multi-page, multi-document understanding; the repository also contains the M3DocVQA benchmark.
-
GupShup summarization models
Models for summarizing code-switched conversations.
Talks & tutorials
-
From Fundamentals to Recent Advances: A Tutorial on Keyphrasification
ECIR 2022 tutorial with Rui Meng and Florian Boudin. All resources are available and linked on the website.
-
Keyphrases in Building NLP-Driven Financial Products
Feb 2023, Muffin Workshop @ AAAI 2023, USA.
-
Identifying Keyphrases from Text Documents: From Heuristics to Language Models
April 2022, Data Science Seminar, University of Utah, USA.
-
Automatic Identification of Keyphrases from Text
Feb 2022, IIT Guwahati, India.
-
Learning to Identify Keyphrases from Text
March 2021, MIDAS Lab, IIIT-Delhi, India.
-
Keyphrase Extraction as Sequence Labeling using Contextualized Embeddings
ECIR 2020.
-
Search Powered by Deep Learning
Feb 2017, Smart Data Conference, San Francisco, USA.
-
Big Data and NoSQL Databases
Jan 2013, ISACA Central Arkansas Chapter, USA.
Mentoring
I have been fortunate to work with many talented students.
Research interns at Bloomberg AI
Students I worked with at IIIT-Delhi
Education
2011 – 2015
PhD, Information Science
University of Arkansas at Little Rock, USA
Thesis: A framework for collecting, extracting and managing event identity information from textual content in social media.
2005 – 2010
BS & MS, Computer Science
Banaras Hindu University, India
Thesis: Mining the blogosphere from a socio-political perspective.
Honors & service
Honors
- 40 Under 40 Data Scientists, Analytics India Magazine (2025)
- Award for Innovation, Infosys (2017)
- Outstanding PhD Graduate, Information Quality Program, UA Little Rock (2015)
- Outstanding Publication Award, Integrated Computing Program, UA Little Rock (2015)
- First Place, Engineering and IT, Student Research and Creative Works Expo, UA Little Rock (2015)
- Outstanding First Year Graduate Student Award, Donaghey College of Engineering and IT (2011)
- All India Rank 1, Postgraduate Entrance Test (Computer Science), Banaras Hindu University (2008)
Service
- Area Chair, EACL 2027 (main track and Industry Track)
- Multimodal AI track, Bloomberg Data Science Ph.D. Fellowship
- Senior PC: AAAI 2021, 2022; IJCAI 2013
- PC / reviewer: ACL Rolling Review, ACL, EMNLP, NAACL, EACL, AACL, COLING, AAAI, IJCAI, ECIR, LREC, The Web Conference
- Mentor, NAACL SRW 2021 and ACL SRW 2020
- Co-chair, MR2AMC workshops (IEEE MIPR, IEEE ISM 2018); Publicity Chair, IEEE BigMM 2019
Let's connect
Happy to hear about research collaborations and mentoring. The easiest way to reach me is LinkedIn.