Only released in EOL distros:
Package Summary
bases on literate_pr2 package from Menglong Zhu: menglong(at)seas.upenn.edu
- Author: Robert Heinze, Richard Bormann
- License: LGPL
- Source: git https://github.com/ipa320/cob_object_perception.git (branch: release_electric)
Contents
Detailed Documentation
Overview
Cob_read_text detects and recognizes text in natural images. The system was trained for indoors household environment with Care-O-bot's stereo vision colour cameras. Text localisation and OCR occur separately and can be evaluated using precision, recall. For locating text on images Stroke Width Transformation is used. Text whose baseline was found is transformed back into horizontal text using Bézier curves and given to OCR software to recognize text.
To use read text clone the repository to your disc and build it.
Usage
Single Image
To read text on a single image: roslaunch cob_read_text read_text.launch image:=path_to_image
read_text.launch -> run_detection -> text_detect
Image Stream
To read text from a camera stream: roslaunch cob_read_text read_text_from_camera.launch
read_text_from_camera.launch -> text_detect
Remap "image_color" to topic that publishes to be processed images.
Evaluation Database
To read text on images of database with ground truth data: roslaunch cob_read_text read_text_with_eval.launch xmlfile:=path_to_xmlfile
read_text_with_eval.launch -> run_detection -> text_detect
Parameters for read_text
Parameters are specified in launch/params.yaml.
Parameters
smoothImage (bool, default: false)- smoothing of whole input image before processing
- maximal number pixels of recognized strokes, good for big text: <50, small test: >50
- use rgb channels to compute edgeMap
- upper threshold for canny algorithm
- lower threshold for canny algorithm
- parameter when comparing gradients of opposing edges, in paper: 3.14 / 6
- parameter when comparing stroke widths between pixels
- parameter when comparing color between pixels, set to 255 to deactivate
- maximum height of component in pixels
- maximum variance of stroke width inside component
- diagonal of component has to be smaller than diagonalParameter*medianStrokeWidth
- number of pixels inside component belonging to letter has to be bigger than maxStrokeWidth * pixelCountParameter
- maximum number of foreign components inside component
- [turned off], width has to be smaller than heightParameter * height
- maximum color value difference
- maximum pythagorean distance between 2components
- maximum quotient of median sw of 2 components
- maximum diagonal ratio between 2 components
- maximum gray color value difference between components
- maximum single colour value difference between components
- maximum area ratio between components
- [turned off] maximum pixel number ratio (pixels belonging to letter)
- probability p
- maximum percentage of outliers in dataset
- minimum percentage of outliers in dataset
- maximum angle of bezier curve
- parameter, with which maximum distance between curve and point is calculated
- how strong is blur when sharpening with unsharp mask
- sets minimum brightness change that will be sharpened
- magnitude, how much contrast is added at the edges when sharpening
- which spell check method is used
- show gray colored edgemap image
- show swt map
- show connected components
- show connected components recognized as letter
- show pairs (2 letters belonging together)
- show all text segments after pairs were merged together
- show how text blocks are broken into several lines
- show all bezier models that are created within ransac
- show ransac result -> show best bezier model only
- show how neighbor texts are merged if they fit together and were accidentally separated before
- show results after ocr
- use bezier-transformed images (true) or normal rectangles around bezier lines
Additional Applications
read_evaluation
To evaluate cob_read_text the application read_evaluation is used. Usage: read_evaluation takes a .xml-file containing a list of images with ground-truth bounding boxes enclosing every text that can be found.
Example input file:
<?xml version="1.0" encoding="UTF-8"?> <tagset> <image> <imageName>example_image2.png</imageName> <taggedRectangles> <taggedRectangle center_x="605.894" center_y="636.016" width="94" height="40" angle="-20" text="Sprite" /> <taggedRectangle center_x="752.5" center_y="626.5" width="47" height="25" angle="0" text="Red" /> </taggedRectangles> </image> </tagset>
All rectangles are described via:
- their center point
- their width and height
- their rotation angle (clockwise)
- and the text written inside of them.
For every image referenced between <image> and </image> in the .xml file, read_evaluation calls cob_read_text and compares the results with the solution.
The evaluation is based on the ICDAR 2005 Robust Reading Competitions system using recall and precision as measure: "Precision p is defined as the number of correct estimates divided by the total number of estimates. Recall r is defined as the number of correct estimates divided by the total number of targets. We define the match mp between two rectangles as the area of intersection divided by the area of the minimum bounding box containing both rectangles. For each rectangle in the set of estimates we find the closest match in the set of targets, and vice versa. Hence, the best match m(r, R) for a rectangle r in a set of Rectangles R is defined as: m(r, R) = max mp (r, r') | r' ∈ R Then, our new more forgiving definitions of precision and recall, where T and E are the sets of ground-truth and estimated rectangles respectively:
p = ( Σr_e ∈ Em(r_e, T) ) / |E|
r = ( Σr_t ∈ Tm(r_t, E) ) / |T|
(S. M. Lucas. ICDAR 2005 text locating competition results. In Proc. 8th Int’l Conf. Document Analysis and Recognition, volume 1, pages 80–84, 2005)
Results are saved in folder 'results' inside the image containing folder.
Running the application:
roslaunch cob_read_text read_text_with_eval.launch xmlfile:=path_to_xmlfile
The .xml-file has to be specified in read_text_with_eval.launch.
LabelBox
LabelBox can be used to generate a .xml-file for read_evaluation. Taking an image or a folder with images inside, LabelBox produces a .xml-file as seen above.
Running the application:
$(rospack find cob_read_text)/bin/labelBox path_to_image
$(rospack find cob_read_text)/bin/labelBox path_to_folder/
labelBox Help
Allowed file types: 'png', 'PNG', 'jpg', 'JPG', 'jpeg', 'JPEG', 'bmp', 'BMP', 'tiff', 'TIFF', 'tif' or 'TIF'.
bases on literate_pr2 package from Menglong Zhu: menglong(at)seas.upenn.edu