text

In the field of non-deep learning, heat map refers to simply aggregating large amounts of data, and using a progressive color band to elegantly express, to intuitively show the spatial density or frequency of spatial data. [1] In DCNN, heat maps help to understand which part of an image makes the neural network make the final classification decision. There are two ways to generate the heat map, one is the Gaussian heat map, and the other is the class-activated heat map generated by Grad-CAM (as shown in the figure below).

It is easy to see that the heat map can reflect the location of the detection target.

The following figure [2] shows a Gaussian heat map. The center Gaussian value is close to 1, and the edge is close to 0.

The description of the generated heat map in the CenterNet paper is as follows:

(Note that the concept of "heat map" here is different from the heat map of "the activation intensity of the input image to the category" in Grad-CAM mentioned later)

The heat map style generated in this way is roughly like this [3]:

Observe the Gaussian kernel formula used to map the heat map, where (x, y) is the position of the step block enumerated in the image to be detected, and (px, py) is the coordinate corresponding to the GT key point in the low-resolution image.

It can be seen that when the position of the enumeration block and the coordinates of the GT key point are close to coincide, the output value of the Gaussian kernel is close to 1; when the position of the enumeration block and the GT key point are very different, the output value of the Gaussian kernel is close to 0.

In this way, each key point (block) Gaussian heat map after Gaussian kernel mapping is similar in data representation (this figure [4] is only an example):

In the Gaussian kernel used for mapping, _ ^ 2 is "target-scale adaptive variance". The "target scale" here is not the size of the final bbox box, but the size and distribution of each point in the generated Gaussian heat map.

The value of the denominator in the fractional formula is 0 ~ m. M depends on the resolution of the feature map. Therefore, the output range of _x corresponds to 1 ~ n, n is close to 0 but not 0.

When _ ^ 2 takes a larger value, the Gaussian nuclear output distribution is concentrated at the category peak point, which makes the generated heat map "bright spot" is smaller; when _ ^ 2 takes a smaller value, the "bright spot" diverges to the periphery.

In fact, because the network predicts the center of the object (block R2), the heat map generated by it expresses how well each segmented image fits the center of the object of the determined category.

"At the time of inference, extract the peak points of each category on the heat map separately. The practice is to compare all the response points on the heat map with the 8 neighboring points connected to it. If the response value of this point is greater than or equal to its eight neighbors The point value is retained, and finally we retain all the first 100 peak points that meet the previous requirements. "

The subsequent treatment of this place in the paper is actually a 3 * 3 maximum pooling, and the Gaussian kernel result value between (0,1) is converted into a [0,1] hit flag.

References

1.The principle and implementation of HeatMap

2.Generate Gaussian images

3.Interpretation of CenterNet (1) papers

4.Generation of heatmap and vectormap in human bone key point detection

———— 2020-4-25 @ ————