John R. Smith and Shih-Fu Chang
Department of Electrical Engineering and
Center for Image Technology for New Media,
Columbia University,
New York, N.Y. 10027
{jrsmith, sfchang}@itnm.columbia.edu
Center for Telecommunications Research
Technical Report # 459-96-25
We describe a prototype visual information system for searching for images and videos on the World-Wide Web. New visual information in the form of images, graphics, animations and videos is being published on the Web at an incredible rate. However, cataloging this visual data is beyond the capabilities of current text-based Web search engines. The key to cataloging it is the marriage of text-based processing and content-based visual analysis of the images and videos. In this paper, we describe a complete system by which visual information on the Web is (1) collected by automated agents, (2) processed in both text and visual feature domains, (3) catalogued and (4) indexed for fast search and retrieval. We introduce an image and video search engine which utilizes both text-based navigation and content-based technology for searching visually through the catalogued images and videos. Finally, we provide an initial evaluation based upon the cataloging of over one half million images and videos collected from the Web.
Keywords - content-based visual query, image and video storage and retrieval, World-Wide Web.