1 Introduction

Modern text detectors are capable of capturing text in a variety of different challenging scenarios. However, they may still fail to detect text instances when dealing with extreme aspect ratios and different scales

In order to deal with these difficulties, we propose a new algorithm for scene text detection in this paper, which proposes a series of strategies to significantly improve the quality of text localization

EAST, a very representative one-stage scene text detector, has been proven to fail to detect text instances with extreme aspect ratios. (You can check out the EAST paper)

There are two main reasons for this:

1) The receptive field of the network is limited, so there is not enough information to accurately predict long text instances with spatial expansion.

2) In the NMS step of EAST, the detection fusion uses their text/non-text classification scores as weights, which ignores their quality differences caused by the limited receptive field of the network, and finally leads to biased geometric estimates

This paper proposes a Multi-Oriented Scene Text Detector (MOST) with localization refinement. The localization refinement part consists of a Textual Feature Alignment Module (TFAM) and a Position-Aware Non-Maximum Suppression (PA-NMS) module.

The former aligns image features with coarse detection results, which can dynamically adjust the receptive field of the localization prediction layer. On the other hand, the latter adaptively merges raw detections according to where they were predicted to focus on accurate predictions while discarding inaccurate ones.

2. Performance effect

In particular, they improve performance by 4.0% and 9.5% on the MLT17 validation set (for different IoU criteria), and by 5.1% on the MTW1 test set. Also, our proposed text detector maintains a simple pipeline and runs very fast.

The contribution of this paper has 4 aspects:

1. We propose TFAM, which dynamically adjusts the receptive field based on coarse detection.

2. The proposed PA-NMS further improves detection by fusing location-based trusted predictions.

3. We introduce an instance-level IoU loss to balance training on text instances of different scales.

4. Our proposed MOST achieves state-of-the-art or competitive performance at fast inference speeds.

Bottom-up methods divide scene text detection into two steps: 1) detect basic elements; 2) aggregate these elements to generate detection results

3. Approach strategy

Top-down methods usually follow a common object detection pipeline and output word/text line detection results directly. These methods can be further specified into two subclasses. One-stage text detectors such as TextBoxes, EAST, TextBoxes++ and RRD directly regress the text box parameters on the entire feature map and use NMS to produce the final result. Two-stage text detectors like the Mask TextSpotter series, on the contrary, follow the MaskRCNN type framework, by first using a region proposal network (RPN) to generate text boxes, and then returning the corresponding border parameters. These methods usually have a relatively simple post-processing algorithm that avoids complex aggregation steps.

LOMO proposes an iterative refinement module (IRM) to perceive the entire long text through iterative refinement. It extracts multiple RoI features based on the preliminary proposal to form a multi-level detector.

It extracts multiple RoI features based on the preliminary proposal to form a multi-level detector

PA-NMS stands for position-aware non-maximum suppression.

Experiment: Comparing SimCLR vs. Non-SimCLR for comparison

3.1 Supervised Learning

This work proposes a contrastive learning method: SeqCLR, for text recognition. Treating each feature map as a series of independent instances leads to contrastive learning at the sub-word level, e.g. extracting several positive pairs and multiple negative examples per image. In addition, to obtain effective visual representations for text recognition, new augmentation heuristics, different encoder architectures, and custom projection heads are further proposed.

Experiments on handwritten and scene texts show that the proposed method outperforms non-sequential alignment methods when training a text decoder with learned representations. Furthermore, when the amount of supervision is reduced, SeqCLR significantly improves performance compared to supervised training, and when fine-tuned with 100% labels, SeqCLR achieves state-of-the-art results on standard handwritten text recognition benchmarks.

3.2 Position-aware non-maximum suppression

Text and non-text can have scores S(p), S(q). And the corresponding weights are p(i) and q(i)

We propose position-aware NMS, which keeps accurate parts of detected boxes while removing inaccurate parts according to their positions during merging.

PA-NMS uses the value of the corresponding position-aware score, instead of the text/non-text classification score, as the weight of the box in the position-aware merging process, which can help to precisely locate the text boundary.

4. Experimental results

It can be clearly seen that either it is almost parallel to the baseline, or it is higher than the general baseline. But the speed exceeds the general training.

renderings