Paper Type
Short
Paper Number
PACIS2025-1204
Description
Patent valuation plays a critical role in managing intellectual property by determining a patent’s strategic and financial worth. However, traditional methods often require significant time and resources and produce inconsistent results across jurisdictions. In this study, we introduce the DRAM Patent Benchmark (DPB), a domain-specific dataset of 1,708 DRAM patents graded by a commercial online evaluation system. We evaluate three state-of-the-art large language models (LLMs)—GPT-4o, LLaMA-3.1-8B, and Qwen2.5-14B-Instruct—by measuring their accuracy in pairwise ranking tasks and analyzing their reasoning through expert validation. The models achieve moderate alignment with the grading system (64–70% accuracy), but expert reviewers identify key weaknesses in their legal reasoning and contextual awareness. Specifically, the models overlook patents’ legal status and firm-level strategic factors. These results show that while LLMs can support initial assessments, researchers and practitioners must refine these tools and integrate expert oversight to ensure reliable and practical patent valuation.
Recommended Citation
Kim, Suhyeong; Yang, Seongjun; and Kim, Seongwoo, "Assessing Patent Value with LLMs: DRAM Dataset" (2025). PACIS 2025 Proceedings. 2.
https://aisel.aisnet.org/pacis2025/aiandml/aiandml/2
Assessing Patent Value with LLMs: DRAM Dataset
Patent valuation plays a critical role in managing intellectual property by determining a patent’s strategic and financial worth. However, traditional methods often require significant time and resources and produce inconsistent results across jurisdictions. In this study, we introduce the DRAM Patent Benchmark (DPB), a domain-specific dataset of 1,708 DRAM patents graded by a commercial online evaluation system. We evaluate three state-of-the-art large language models (LLMs)—GPT-4o, LLaMA-3.1-8B, and Qwen2.5-14B-Instruct—by measuring their accuracy in pairwise ranking tasks and analyzing their reasoning through expert validation. The models achieve moderate alignment with the grading system (64–70% accuracy), but expert reviewers identify key weaknesses in their legal reasoning and contextual awareness. Specifically, the models overlook patents’ legal status and firm-level strategic factors. These results show that while LLMs can support initial assessments, researchers and practitioners must refine these tools and integrate expert oversight to ensure reliable and practical patent valuation.
Comments
AI ML