COMMITTEE CHAIR: Dr. Xishuang Dong

TITLE: LARGE LANGUAGE MODELS FOR INFORMATION EXTRACTION

ABSTRACT: Large Language Models (LLMs) have emerged with remarkable capabilities in understanding, generating, and contextualizing natural language. These capabilities have significantly advanced information extraction (IE), enabling the extraction of meaningful information from both unstructured and structured data. This dissertation investigates two important IE tasks, named entity recognition (NER) and text-to-SQL (Text2SQL), and further explores interpretable knowledge distillation techniques to enable efficient deployment of LLMs in resource-constrained environments.For NER, this dissertation focuses on extracting clinically relevant information from electronic health records (EHRs), including medications, diseases, and their relationships. Limited availability of annotated clinical data remains a major challenge for developing high-performing biomedical IE systems. To address this issue, this dissertation investigates the use of ChatGPT for synthetic data generation and augmentation. Multiple pre-trained BERT models, originally trained on large corpora such as Wikipedia and MIMIC, are subsequently fine-tuned on the augmented datasets. Experimental results demonstrate that LLM-generated synthetic data effectively improves biomedical NER performance and facilitates accurate extraction of key clinical entities from EHRs. For Text2SQL, which enables non-expert users to query relational databases using natural language, this dissertation proposes several LLM-based frameworks for improving SQL generation quality and reliability. The first introduces a SQL quality evaluation mechanism that iteratively refines generated queries using feedback on syntactic correctness and semantic accuracy. The second integrates non-parametric attention and confidence-guided prompt refinement without relying on external knowledge, achieving a 6.5% improvement in execution accuracy over a GPT-4o baseline. Furthermore, an Inference-Time Bayesian Refinement Framework (IBRF) reformulates Text2SQL generation as an iterative process of generation, probabilistic error diagnosis, and targeted repair. On the BIRD benchmark, IBRF achieves 66.02% execution accuracy and 64.90% valid efficiency score, improving performance by an average of 49.7% over base models. Finally, this dissertation develops interpretable, structure-aware knowledge distillation methods that transfer both label dependencies and reasoning processes from large teacher models to compact students. These approaches recover nearly 80% of teacher performance with 19× fewer parameters and reduce computational costs by up to 42× while preserving meaningful reasoning capabilities, demonstrating the potential of efficient and trustworthy LLMs for real-world information extraction applications.

Keywords: LLMs, Text2SQL, Knowledge Distillation, Information Extraction

Room Location: Electrical and Computer Engineering Department Conference, Room 315D