Can AI Describe Art as We Do? A Case Study on a Pottery Collection
There are two capabilities of current large language models (LLM)-based AI systems that we attempt to evaluate for improving the discoverability of library and museum collections, which are often searched by using expert-defined keyword vocabularies through complex hierarchical categories: 1) Vector search: differing from the traditional keyword search, it improves discovery of word semantic relationships in a broader natural language domain and 2) Multimodal large language models (MLLM): combining computer vision processing images alongside LLMs, boosting understanding of the image both textually and visually. We explore how visual language models (VLM) and MLLMs can bridge vocabulary gaps in search between expert-generated descriptions and the public.
Read More