Keyword Extraction and Search Technique Using Transformation Angle Estimation to Candidate Character Set in Scene Image

This paper proposes an image search technology that processes character candidate sets by scanning scene images containing character strings such as signboard messages with a recognition dictionary. First, it extracts keywords composed of characters that are regularly aligned in the image by matching the word dictionary to the candidate character set. It then uses the user-input keywords in performing image search. We assume that the principal use of this technology is to retrieve images from data sets held by the user. The proposed technology has the feature that it can extract character strings even if they consist of shaded, three-dimensional characters, lie on a complex background, and are inclined, etc., attributes that cannot be offered by existing character recognition schemes for scene images. To overcome the problem of the background triggering the erroneous recognition of characters, we estimate the transformation angles of the candidate character from angular distributions of the gradient vector of pertinent areas, and add it to the limiting conditions used in determining the arrangement of character strings in the matching against the word dictionary process. Experiments on keyword extraction and image retrieval show that the proposed technology has practical performance in the application assumed, and that its estimation of the transformation angle of candidate characters is effective in decreasing keyword extraction errors.