Global automatic detection method employing multi-level constraints for circular markers in high-speed videogrammetry

Shouzhu Zheng; Sicong Liu; Zhen Ye; Xiaolong Ma; Baohang Wang; Jianxia Zhang

doi:10.1117/1.JRS.19.016501

28 November 2024 Global automatic detection method employing multi-level constraints for circular markers in high-speed videogrammetry

Shouzhu Zheng, Sicong Liu, Zhen Ye, Xiaolong Ma, Baohang Wang, Jianxia Zhang

Author Affiliations +

Journal of Applied Remote Sensing, Vol. 19, Issue 1, 016501 (November 2024). https://doi.org/10.1117/1.JRS.19.016501

Abstract

In recent years, algorithms for fast and accurate recognition and detection of circular markers have become crucial in the field of high-speed videogrammetry. However, most existing techniques often necessitate a manually selected region of interest that encompasses the full information of the circular marker. This manual box selection method is inefficient and unsuitable for practical engineering applications. To address this issue, we propose a global automatic recognition and detection approach that employs multi-level constraints for identifying circular markers in high-speed videogrammetry. First, an edge detection method based on the Canny algorithm is employed to extract candidate edges containing all circular markers. Subsequently, two geometric constraints—general geometric condition and roundness metric constraint—are applied to eliminate a large number of non-circular marker edges. Finally, pseudo-edges of circular markers are removed, and the corresponding accurate edges are retained by applying an extrema point condition constraint. The performance of the proposed method is evaluated using several high-speed videogrammetry image datasets. Experimental results demonstrated that our method can accurately detect and recognize all circular markers, outperforming comparable methods. The proposed method holds promise for efficient and wide-ranging applications in the recognition and detection of circular markers in high-speed videogrammetry.

1. Introduction

High-speed videogrammetry, as a specialized form of close-range photogrammetry, has garnered considerable attention in recent years. This technique is especially advantageous for monitoring the spatial information of moving objects across various engineering applications such as bridge monitoring, wind tunnel tests, and civil engineering construction.¹^–⁵ One of its key benefits is its ability to provide non-contact, high-precision measurements. Typically, moving objects are affixed with circular markers at strategic locations to facilitate tracking. These markers are then detected and recognized to analyze the movement trajectory in either two- or three-dimensional space. Consequently, fast and accurate target recognition and circular marker detection are critical steps in the high-speed videogrammetry process. The quality of these initial steps significantly impacts the accuracy and efficiency of subsequent tasks, including object tracking and three-dimensional reconstruction.

Recently, target recognition and detection are widely used in various fields of remote sensing optical image,⁶^–⁸ synthetic aperture radar (SAR) image,⁹^–¹¹ multi- and hyper-spectral image,¹²^–¹⁴ video image,¹⁵^,¹⁶ etc. Moreover, in the field of high-speed videogrammetry, target recognition and detection of circular markers have also gained significant attention in the field of image processing, as the resulting data offer valuable insights into geometric measurements. The accuracy and computational efficiency of target recognition and detection are critical, especially in applications requiring high precision. Consequently, this area has become a focal point for research to enhance performance metrics. Kim et al.¹⁷ employed the fast line extraction algorithm in conjunction with the least squares fitting approach to provide a rapid and robust method for ellipse extraction. Mai et al.¹⁸ introduced a hierarchical strategy for robust ellipse extraction, leveraging line segments, connectedness, curvature requirements, and the random sample consensus algorithm. Their method demonstrated excellent performance in handling occlusions and overlapping ellipses through experiments on both synthetic and real images. Von Gioi et al.¹⁹ developed a line segment detector (LSD) method that required no parameter adjustment and included false detection control. Experimental findings indicated that this approach yielded more accurate results than prior methods, with a lower rate of false positives and false negatives. Chia et al.²⁰ proposed an ellipse detector with a self-correcting capability based on edge-following, outperforming comparative methods in terms of recall and precision scores under challenging conditions such as background clutter, salt-and-pepper noise, and partial occlusions. Fornaciari et al.²¹ addressed the real-time performance trade-off between detection effectiveness and execution time, proposing a fast and effective ellipse detector for real-world images. Ellipse and line segment detector, with continuous formulation (ELSDc), introduced by Pătrăucean et al.,²² employed an enhanced LSD version that improved candidate generation, validation, and model selection, significantly reducing the false-positive rate. Furthermore, ELSDc based on the given possible region was adopted by Ye et al.²³ and Tong et al.²⁴ in the ellipse detection of videogrammetric monitoring. However, ELSDc’s performance in processing complex primitives remains limited owing to its candidate generation stage. Lu et al.²⁵ proposed a circle detection method that refines Arc-Support line segments using a combination of gradient direction, polarity, region restriction, mean shift clustering, and twice circle fitting. The centroid search algorithm in the PhotoModeler software²⁶^–³⁰ and other adaptive methods based on image block with interest region⁴^,³¹^–³³ offer accurate identification of circular marker edges based on manual box selection in high-speed videogrammetry; therefore, they may cause the failure of high-efficiency detection. Lu et al.³⁴ introduced the Arc-Support detection framework, which employed Arc-Support line segments to detect ellipses through four stages: Arc-Support group formation, initial ellipse set generation, clustering, and candidate verification. Quantitative studies revealed that this method successfully balanced accuracy and efficiency. Furthermore, Liu et al.³⁵ employed the Arc-Support method to detect the ellipse target of the region of interest (ROI) in the high-speed videogrammetric measurements. However, the Arc-Support method is sensitive to complex backgrounds and noise. Meng et al.³⁶ proposed an arc adjacency matrix–based ellipse detection (AAMED) technique that employed cumulative factor–based matrices and adaptive constraints. Experimental results across nine datasets showed that AAMED outperformed 12 comparative methods in terms of recall, precision, $F$ -measure, and time consumption. However, the AAMED method cannot effectively detect small ellipses and may easily yield false small ellipses.

There are several limitations with most existing methods for target recognition and detection. These often necessitate a manually selected ROI and dependence on synthetic and simplistic background settings, among other factors. Such requirements make it challenging to achieve efficiency and precision under complicated conditions commonly encountered in real-world applications of high-speed videogrammetry. Alternatively, methods employing deep learning for target recognition and detection require the generation of large datasets, which can be impractical without large sample datasets. To address these challenges, this study introduces an automatic recognition and detection method utilizing multi-level constraints based on a single image. This approach aims to rapidly and reliably identify all circular markers within a high-speed videogrammetry image. First, the Canny detection algorithm is employed to extract candidate edges. Subsequently, two geometric constraints based on the general characteristic of the ellipse are applied to remove edge information related to non-circular markers. Finally, an extrema point condition constraint is further implemented to eliminate pseudo-edges associated with circular markers, retaining only the accurate edge information. The main contributions of this study are summarized as follows: (1) a global automatic detection method based on multi-level constraints is proposed for circular markers of a whole image in high-speed videogrammetry, (2) the region growth algorithm is introduced in general geometric constraint that effectively removes several non-candidate edges, and (3) an extrema point condition constraint based on the number and grayscale difference of extrema points is developed to accurately refine the identification of circular mark edges.

The remainder of this paper is organized as follows: Section 2 outlines the proposed method, detailing the edge detection and multi-level constraints, which encompass geometric and grayscale constraints based on the region growth algorithm, morphological methods, roundness metrics, and extrema point condition constraints. Section 3 presents the experiments conducted using the proposed method and compares them to the current state-of-the-art methods. Section 4 presents the conclusions based on this study.

2. Methodology

The current methods for target recognition and detection, particularly those focused on circular markers (Fig. 1) in high-speed videogrammetry, often rely on manually selecting the ROI and require algorithms specifically tailored for circular marker recognition. As shown in Fig. 1, two varieties of circular markers, which have been extensively utilized in the previous studies,⁵^,³¹^,³³^,³⁵^,³⁷^–⁴⁴ were tested in this work. The circular markers shown in Figs. 1(a) and 1(b) are often regarded as tracking and control targets, respectively. The tracking target is a white circle on a black background, with a size of $\sim 20 \times 20$ to $40 \times 40 pixels$ in the obtained image. Based on the tracking target, the control target incorporates a modest cross-wire, an inner-circle configuration, and retro-reflective material at the center of the designated region. The two types of circular markers exhibit notable differences in the gray value within and outside the circle. This necessitates that the pixel value within the circle exceeds that outside the circle, a requisite for the subsequent detection process, which aims to extract the complete edge contour of circular markers.

Fig. 1

Circular markers in high-speed videogrammetry. (a) Tracking target. (b) Control target.

These approaches, often semi-automated or requiring manual intervention, are inefficient and do not meet the demands for high-efficiency processing in practical high-speed videogrammetry applications. To address these limitations, we propose a global automatic recognition and detection method with multi-level constraints, which operates on the entire image to identify circular markers. This approach incorporates two geometric constraints and one grayscale condition constraint to automatically and accurately extract all circular markers within a high-speed videogrammetry image. The proposed method consists of four key processing steps: (1) Canny detection, (2) general geometric constraint, (3) a second geometric constraint based on roundness metric conditions, and (4) extrema point condition constraint. The detailed workflow of the proposed approach is illustrated in Fig. 2.

Fig. 2

Flowchart of the proposed method for circular mark in high-speed videogrammetry.

2.1.

Canny Detection

The Canny operator⁴⁵ is employed for detecting all circular markers across the image. This choice is influenced by its features, such as non-maximum suppression and dual-threshold processing, which are based on improved Sobel and Prewitt operators. The Canny detection process comprises several key steps: Gaussian filter smoothing, gradient magnitude and direction calculation, non-maximum suppression, and dual-threshold algorithm. Gaussian filter smoothing uses a two-dimensional Gaussian function [as shown in Eq. (1)] to smooth the image and reduce the effects of the noise. The gradient magnitude and direction calculation are defined as shown in Eqs. (2) and (3). Non-maximum suppression is used to accurately acquire a one-pixel width of each edge. The dual-threshold algorithm is adopted to reduce the influence of noise after the process of non-maximum suppression. Therefore, this approach swiftly and accurately approximates true edge information. Importantly, it ensures that candidate edges containing circular markers are effectively preserved, minimizing the likelihood of missing any circular markers.

Eq. (1)

G (x, y) = \frac{1}{2 π σ^{2}} e^{- \frac{x^{2} + y^{2}}{2 σ^{2}}},

Eq. (2)

M = \sqrt{G_{x}^{2} + G_{y}^{2}},

Eq. (3)

θ = \arctan (\frac{G_{x}}{G_{y}}),

where

σ

represents a parameter of the Gauss filter, and

G_{x}

and

G_{y}

represent the partial derivation of the

x

-direction and

y

-direction, respectively.

2.2.

General Geometric Constraint

In the imaging process, a circular mark is typically projected as an ellipse. The properties of this ellipse, such as length, radius, eccentricity, and region size, served as the initial constraint conditions. These constraints filtered out a large portion of non-edge information by iterating through all identified candidate edges. The initial constraint criteria for the length radius, eccentricity, and region size are as follows:

Eq. (4)

{\begin{cases} a_{\min} \leq a \leq a_{\max} \\ e_{\min} \leq e \leq e_{\max} \\ R_{\min} \leq R \leq R_{\max} \end{cases},

where

a, e

, and

R

represent length radius, eccentricity, and region size, respectively. The corresponding min and max separately represent the minimum and maximum values of the variable set, respectively.

In addition, a region growth algorithm is employed to assess the grayscale value of the eight neighboring regions around each pixel in a region. This is used to ascertain whether the edge points in a region exhibit closed connectivity, thereby further eliminating irrelevant edge information. For the region growth algorithm, the upper-left corner point of each region is automatically selected as the initial seed point. The same gray-level count in eight neighboring regions is determined based on the circle’s closed characteristics. The condition for closed connectivity is that the same gray number of the eight neighborhood regions around each seed point is greater than or equal to 2. This looping process iteratively filters all additional edge regions, preserving only those that meet the closed connectivity condition. As a result, a significant amount of non-candidate edges is removed.

2.3.

Second Geometric Constraint Based on Roundness Metric Condition

Some pseudo-edges may persist even after applying the general geometric constraint and region growth algorithm. To address this issue, we employ a two-pronged approach: morphological reconstruction based on geodesic distance⁴⁶ and a roundness metric condition. These techniques further refine the edge information by identifying and removing pseudo-edges that mimic the characteristics of circular markers.

The morphological reconstruction method, grounded in geodesic distance, fills in any gaps in the edge information of the preceding binary image. Concurrently, the roundness metric condition is applied to evaluate the geometric characteristics of the filled area. This condition helps ascertain whether the filled region approximates a circular shape, allowing for more accurate and robust detection of circular markers.

2.3.1.

Morphological reconstruction based on geodesic distance

First, the geodesic distance of closed connectivity is defined. Suppose that the connected region is $A$ , and the path $P$ between points c and d in the connected region $A$ is completely included in $A$ ; then, the geodesic distance between two points $c$ and $d$ can be defined as $d_{A} (c, d)$ . At this time, the geodesic distance between two points c and d in the connected region A is the shortest path among all paths. If $P$ does not exist, then $d_{A} (c, d) = \infty$ . Then, the geodesic expansion is calculated according to the geodesic distance defined earlier. Suppose that a discrete set $X$ is defined and the equation is to satisfy $Y \subseteq X, Y$ is the identification image, and $X$ is the mask image. For scale $n$ ( $n ≧ 0$ ) in $X$ , the result $δ_{X}^{(n)} (Y)$ of geodesic expansion of $Y$ is the set of pixels whose geodesic distance from point $p$ to $Y$ in $X$ is less than or equal to $n$ . The corresponding equation is as follows:

Eq. (5)

δ_{X}^{(n)} (Y) = {p \in X | d_{X} (p, Y) \leq n} .

For the geodesic expansion $δ_{X}^{(n)} (Y)$ of scale $n$ , the geodesic expansion with a unit of scale 1 can be obtained by iterating $n$ times.

Eq. (6)

{\begin{cases} δ_{X}^{(n)} (Y) = δ_{X}^{(1)} \circ δ_{X}^{(1)} \dots \circ δ_{X}^{(1)} \\ δ_{X}^{(1)} = (Y \oplus B) \cap X \end{cases},

where

B

is a

3 \times 3

connection matrix (four connection matrix or eight connection matrix),

\oplus

is the expansion operation, and

\cap

is the intersection operation.

Therefore, the image reconstruction based on geodesic distance can obtain the result of filling the hole through the iterative calculation of the basic geodesic expansion.

2.3.2.

Roundness metric condition

A roundness threshold is defined on the basis of the circular characteristics. Subsequently, the roundness metric is applied to evaluate the quality of each filled area generated through morphological reconstruction. This evaluation determines whether the shape is approximately circular, thereby facilitating the removal of pseudo-edges from circular markers. The equation for this judging condition is as follows:

Eq. (7)

m = \frac{4 π S}{P^{2}},

where

m

represents roundness,

S

represents the area of each region, and

P

represents the perimeter of each region.

2.4.

Extrema Point Condition Constraint

The extrema point condition constraint aims to refine the identification of circular mark edges by judging the number of extrema points and grayscale differences with inside and outside pixel values for each candidate region. First, the extremal method proposed by Haralick and Shapiro⁴⁷ is employed to identify eight distinct extremal pixels for each candidate region. These extremal points are situated at various positions around the oriented bounding rectangle of the region, specifically at the top-left, top-right, right-top, right-bottom, bottom-right, bottom-left, left-bottom, and left-top corners. Figure 3 illustrates the extrema of two different regions. Each extrema point is distinct in the region on the left of Fig. 3(a). In the region on the right of Fig. 3(b), certain extrema points (such as top-right and right-top) are identical. Typically, ellipses and circles have eight different extreme points [similar to Fig. 3(a)]. Subsequently, the centroid coordinates for each candidate region are calculated using the binary centroid algorithm.⁴⁸ For each region, eight straight-line equations are constructed by connecting the centroid to each extremal point. Two points are then selected on either side of each extremal point along these lines, as detailed in Eq. (8). The arrangement of these eight straight lines for each candidate region is illustrated in Fig. 4. To finalize the identification process, the pixel values at the two selected points around each extremal point are compared. Specifically, the condition for identifying a region as a circular mark is that the pixel value closer to the centroid must be greater than the pixel value further away. In this case, it can effectively be determined that the gray value within the circle of eight distinct is consistently greater than the corresponding gray value outside the circle, with the constraint of eight extrema points exhibiting a relatively uniform distribution. This ensures the reliability and robustness of the judgment. This guarantees that only legitimate circular markers are considered, thereby enhancing the precision of the overall detection process. Consequently, the circular marks can be obtained accurately through the constraint of the number of extrema points and grayscale difference based on the extrema point condition.

Eq. (8)

{\begin{cases} y = k x = b \\ k = \frac{y_{1} - y_{0}}{x_{1} - x_{0}} \\ b = y_{0} - k x_{0} \end{cases},

where (

x_{0}, y_{0}

) represents the pixel position of the centroid, (

x_{1}, y_{1}

) represents the pixel position of each extremal point, (

x, y

) represents the pixel position of two points on both sides of each extremal point,

k

represents the slope of the straight-line equation, and

b

represents the intercept of the straight line equation.

Fig. 3

Extrema of two different regions. (a) Distinct situation. (b) Identical situation.

Fig. 4

Judgment process of circular mark edge. (a) Candidate edge (yellow). (b) Extremal points (red) and centroid position (magenta). (c) Straight line construction (cyan). (d) Two selected points (orange). (e) Circular mark edge (red).

3. Results and Discussion

Experiments were conducted using five real-world high-speed videogrammetry image datasets to assess the performance of the proposed method; the experiments include (1) a five-layer frame model, (2) a three-story frame model, (3) a vibration generator experiment, (4) a high-rise building model, and (5) a brick and concrete structure wall model. All datasets were captured by one of two CamRecord CL600 high-speed cameras (Optronis, Kehl, Germany), which has a resolution of $1280 \times 1024 pixels$ . Here, the settings of high-speed cameras often meet the following requirements: (1) the field of view of measuring range should cover the specified range structure, and (2) the position of the camera should not be affected by the external environment of the structure.

3.1.

Datasets Descriptions

3.1.1.

Five-layer frame model (FFM) dataset

This dataset was generated during a shaking table experiment involving an FFM and was captured using the Optronis CamRecord CL600 camera. The model is a core tube structure composed of reinforced concrete, with dimensions measuring 2.42, 1.62, and 4.5 m in the $X$ , $Y$ , and $Z$ directions, respectively. A total of 16 circular markers were strategically placed on the surface and periphery of the structure. The dataset includes images of regions from the second to the third stories and surrounding areas, as shown in Fig. 5(a).

Fig. 5

Experiments on five different datasets. (a) Five-layer frame model dataset. (b) Three-story frame model dataset. (c) Vibration generator experiment dataset. (d) High-rise building model dataset. (e) Brick and concrete structure wall dataset.

3.1.2.

Three-story frame model (TSF) dataset

This dataset, depicted in Fig. 5(b), features a TSF model constructed of concrete. The base dimensions are $4.5 m \times 1.8 m$ , and the model has an overall height of 4.26 m, with each story measuring 1.42 m in height. Approximately 12 circular markers were attached to the surface of the model. In addition, six stable iron rods were positioned around the model, and 10 circular markers were affixed to its surface.

3.1.3.

Vibration generator experiment (VGE) dataset

This dataset focuses on a vibration generator with a base measuring $˜ 0.86 m \times 0.55 m$ and a maximum height of 0.9 m. The generator was placed on a platform measuring $1.2 m \times 0.8 m$ . Fourteen circular markers were affixed to the vibration generator; their distribution is illustrated in Fig. 5(c).

3.1.4.

High-rise building model (HRB) dataset

This dataset, shown in Fig. 5(d), is taken from the lower half of two high-rise building model. The plane size of each high-rise building model is $9.84 m \times 3.2 m$ , with a height exceeding 10 m. On the surface of the structure model, 22 circular markers were affixed at their corresponding key position.

3.1.5.

Brick and concrete structure wall (BCSW) dataset

This dataset was obtained through a shaking table experiment utilizing a brick and concrete structure wall model. In addition, the model was also a scaling-down model structure of an actual wall and was also captured using the Optronis CamRecord CL600 camera. A total of 27 circular markers were affixed to the corresponding surface of the wall.

3.2.

Experimental Setup and Parameter Settings

To further validate the effectiveness of the proposed method, two state-of-the-art ellipse detection methods—AMMED³⁶ and Arc-Support³⁴—were chosen for comparative analysis. These methods are among the most commonly used for ellipse detection in image processing and computer vision. Performance metrics such as precision, recall, and $F$ -measure³⁴^,³⁶ were used to evaluate detection and recognition performance. Precision = TPs/(TPs + FPs), recall = TPs/(TPs + FNs), and F-measure = $2 / ({precision}^{- 1} + {recall}^{- 1})$ . Here, TPs, FPs, and FNs represent true positives, false positives, and false negatives, respectively. The publicly available source codes of AMMED and Arc-Support were used to ensure a reliable comparative analysis. All experiments were conducted on a computer equipped with an Intel(R) Xeon(R) W-2235 CPU at 3.80 GHz and 64 GB of memory, running Windows 11 64-bit. The proposed method was implemented in MATLAB. In the five datasets, detailed parameters of the proposed method include length radius ( $a$ ), eccentricity ( $e$ ), region size ( $R$ ), and roundness ( $m$ ), where $15 pixel \leq a \leq 60 pixel$ , $0 < e \leq 0.7$ , $20 pixel \leq R \leq 150 pixel$ , and $m$ $\geq 0.915$ . In addition, to verify whether the extracted edge of the circular markers are accurate, the extracted edges of aforementioned methods are to further obtain the corresponding center coordinates. These coordinates are then to be compared with the results of the centroid search algorithm in the PhotoModeler software.²⁶^–³⁰

3.3.

Results and Analysis

A detailed analysis of the proposed method’s recognition process was conducted to offer an intuitive assessment its performance, as depicted in Figs. 6 Fig. 7–10. Both qualitative and quantitative analyses were conducted to compare the detection and recognition results of the proposed method against AMMED and Arc-Support. Detailed comparative outcomes are presented in Figs. 11 Fig. 10–13 and Tables 1 and 2.

Fig. 6

Recognition process results of FFM. (a) Canny. (b) General geometric properties constraint. (c) Roundness metric constraint. (d) Grayscale condition constraint.

Fig. 7

Recognition process results of TSF. (a) Canny. (b) General geometric properties constraint. (c) Roundness metric constraint. (d) Grayscale condition constraint.

Fig. 8

Recognition process results of VGE. (a) Canny. (b) General geometric properties constraint. (c) Roundness metric constraint. (d) Grayscale condition constraint.

Fig. 9

Recognition process results of HRB. (a) Canny. (b) General geometric properties constraint. (c) Roundness metric constraint. (d) Grayscale condition constraint.

Fig. 10

Recognition process results of BCSW. (a) Canny. (b) General geometric properties constraint. (c) Roundness metric constraint. (d) Grayscale condition constraint.

Fig. 11

Recognition result obtained by the Arc-Support on different datasets. (a) FFM. (b) TSF. (c) VGE. (d) HRB. (e) BCSW.

Fig. 12

Recognition result obtained by the AAMED on different datasets. (a) FFM. (b) TSF. (c) VGE. (d) HRB. (e) BCSW.

Fig. 13

Recognition result obtained by the proposed method on different datasets. (a) FFM. (b) TSF. (c) VGE. (d) HRB. (e) BCSW.

Table 1

Statistical results obtained by three considered methods on five datasets.

Dataset	Evaluation metric	AAMED (%)	Arc-Support (%)	Proposed method (%)
FFM	Precision	100.00	100.00	100.00
	Recall	81.25	75.00	93.75
	$F$ -measure	89.66	85.71	96.77
TSF	Precision	100.00	100.00	100.00
	Recall	40.91	59.09	100.00
	$F$ -measure	58.07	74.28	100.00
VGE	Precision	75.00	100.00	100.00
	Recall	42.86	85.71	100.00
	$F$ -measure	54.55	92.31	100.00
HRB	Precision	100.00	100.00	91.30
	Recall	9.09	50.00	95.45
	$F$ -measure	16.67	66.67	93.33
BCSW	Precision	100.00	100.00	100.00
	Recall	70.37	81.48	100.00
	$F$ -measure	82.61	89.80	100.00

Table 2

Time consumption results obtained by three considered methods on five datasets (time/s).

Dataset	AAMED	Arc-Support	Proposed method
FFM	0.25	2.08	0.59
TSF	0.30	1.38	0.68
VGE	0.71	1.09	0.54
HRB	0.32	3.45	0.32
BCSW	0.33	5.08	0.45

3.3.1.

Recognition process analysis of the proposed method

Figure 6 provides a detailed overview of the optimal recognition process for the dataset FFM using the proposed method. Figure 6(a) reveals extensive edge information captured using the Canny detector. A general geometric constraint was applied to effectively filter out numerous non-candidate edges, as shown in Fig. 6(b). Accurate detection results of all circular markers obtained using the roundness metric constraint are depicted in Fig. 6(c). Figure 6(d) further confirms that only those candidate edges meeting the grayscale condition constraint are retained. The recognition results for the dataset TSF, displayed in Fig. 7, align closely with those from Fig. 6. Figures 7(a)–7(c) detail the process involving Canny detection, general geometric constraints, and roundness metric constraints, respectively. The final ensemble of edges corresponding to all circular markers is presented in Fig. 7(d).

With regard to the third experiment using the VGE dataset, Fig. 8(a) illustrates the initial edge detection results using the Canny detector. This includes both candidate and non-candidate edges. Figure 8(b) shows the removal of numerous non-candidate edges via general geometric constraints. Figure 8(c) indicates that a few residual non-candidate edges are further removed, although some pseudo-edges, highlighted by red rectangles, remain. These residual pseudo-edges are ultimately eliminated in Fig. 8(d) through the application of the grayscale condition constraint.

For the last two datasets of the high-rise building model (HRB) and BCSW datasets, as illustrated in Figs. 9(a) and 10(a), respectively, demonstrate a comprehensive presentation of edge information extracted by the Canny detector. Figures 9(b) and 10(b) illustrate the elimination of all non-candidate edges from both datasets in accordance with the general geometric constraints. Figures 9(c) and 9(d) and Figs. 10(c) and 10(d) illustrate the further judgement processing using the roundness metric constraints and the extrema point condition constraint, respectively.

3.3.2.

Recognition accuracy comparison

To assess the recognition and detection performance of the proposed method, its performance was compared against two other well-established techniques: Arc-Support and AAMED. The final recognition results from all three methods are displayed in Figs. 11 Fig. 12–13 for visual interpretation. Subsequently, evaluation metrics—including precision, recall, and F-measure—were employed to quantitatively analyze these results.

A review of the experimental outcomes across the three datasets, as depicted in Figs. 11 Fig. 12–13, demonstrates that the proposed method outperforms Arc-Support and AAMED in terms of recognition accuracy. As illustrated in Fig. 11, instances of undetected features are apparent in the results generated by the Arc-Support method [as indicated by the yellow rectangles in Figs. 11(a)–11(e)]. A similar pattern emerges in the results yielded by AAMED, where both missed detections and false positives are present [noted within yellow rectangles in Figs. 12(a)–12(e)]. Specifically, two blue rectangles in Fig. 12(c) illustrate false detections by AAMED in the VGE dataset, which are likely the result of an oversight regarding circular mark features. In contrast, the proposed method demonstrated high detection accuracy in the FFM and HRB dataset, with only one missed detection of a yellow rectangle and two false detections of a blue rectangle. Furthermore, it successfully identified all circular markers in the other three datasets, as presented in Fig. 13. This superior performance can be attributed to the method’s comprehensive utilization of both the geometric and grayscale characteristics of the markers.

In addition, a detailed statistical analysis of the recognition results across the five datasets was performed, with the results presented in Table 1. The results demonstrated that both AAMED and Arc-Support yielded relatively low values for precision, recall, and $F$ -measure. In particular, Arc-Support registered precision values of 100.00%, 100.00%, 100.00%, 100.00%, and 100.00% across the five experiments. The corresponding recall figures were 75%, 59.09%, 85.71%, 50.00%, and 81.48%, with $F$ -measure values of 85.71%, 74.28%, 92.31%, 66.67%, and 89.80%, respectively. These suboptimal metrics are primarily attributable to missed detections inherent to the Arc-Support method. Similarly, for AAMED, the precision values were 100%, 100%, 75.00%, 100.00%, and 100.00%, with the recall values at 81.25%, 40.91%, 42.86%, 9.09%, and 70.37%, and the $F$ -measure values at 89.66%, 58.07%, 54.55%, 16.67%, and 82.61%, respectively, across the five datasets. The shortcomings of AAMED can be attributed to both false detections and missed detections observed in the experiments. In contrast, the proposed method successfully identified almost all circular markers in each of the five experiments. This included 15 markers (with only one missed detection) in the FFM dataset, 22 markers in the TSF dataset, 14 markers in the VGE dataset, 23 markers (with only one missed detection and two false detections) in the HRB dataset, and 27 markers in the BCSW dataset. It is noteworthy that the method’s statistical values for precision, recall, and $F$ -measure exhibited a marked superiority over those of the other two methods, reflecting a high level of accuracy. Specifically, significant improvements were observed in both recall and $F$ -measure metrics. This provides substantial evidence to support the effectiveness of the proposed method for the recognition of circular markers in high-speed videogrammetry applications.

Moreover, the computational time required for each method on the three datasets is summarized in Table 2. The execution time for Arc-Support on the datasets was 2.08, 1.38, 1.09, 3.45, and 5.08 s. In comparison, AAMED required 0.25, 0.30, 0.71, 0.32, and 0.33 s. In contrast, the proposed method required a shorter amount of time, with processing times of 0.59, 0.68, 0.54, 0.32, and 0.45 s for each dataset. Therefore, our proposed method demonstrates superior time efficiency compared with Arc-Support and AAMED. Nevertheless, the processing efficiency of our proposed method represents a significant improvement over manual box selection methods.

Finally, to further analyze the edge results of circular markers obtained by the above detection methods, the least squares fitting method is used to obtain the center coordinates of the common circular markers (illustrated in Fig. 14) and evaluate their accuracy, which are then compared with the results obtained through the centroid search algorithm. The corresponding results are shown in Fig. 15 and Table 3. As illustrated in Fig. 14, the center coordinates obtained for the common circular markers across all experimental datasets are consistent with those obtained by the centroid search algorithm. The corresponding RMSE results of all experimental datasets in Fig. 15 are largely within a 0.5-pixel range, which suggests that the edges extracted by the various methods are of superior accuracy. In particular, the title $N$ for the $x$ axis represents the point number of the common circular markers in Fig. 15. However, it is evident that the center coordinates yielded by our proposed method (such as point 5 of the VGE dataset in Fig. 16) are superior to those obtained by other methods, due to the minimal deviation observed in the edges extracted by the Arc-Support and AAMED methods. Table 3 also presents the mean and standard deviation (STD) values for the results of each dataset in Fig. 15. The mean and STD values for our proposed method are all smaller than those of other methods, indicating that the center coordinates obtained by our proposed method are more accurate and robust than those of other comparison methods.

Fig. 14

Common circular markers obtained by all comparison methods. (a) FFM dataset. (b) TSF dataset. (c) VGE dataset. (d) HRB dataset. (e) BCSW dataset.

Fig. 15

Center coordinates results obtained by all compared methods. (a) FFM dataset. (b) TSF dataset. (c) VGE dataset. (d) HRB dataset. (e) BCSW dataset.

Table 3

Center coordinates results obtained by three considered methods on five datasets (pixels).

Dataset	AAMED		Arc-Support		Proposed method
Dataset	Mean	STD	Mean	STD	Mean	STD
FFM	0.219	0.227	0.169	0.142	0.130	0.140
TSF	0.092	0.065	0.294	0.342	0.086	0.043
VGE	0.690	0.667	0.513	0.494	0.074	0.041
HRB	0.057	0.014	0.294	0.113	0.123	0.001
BCSW	0.114	0.136	0.105	0.123	0.099	0.089

Fig. 16

Edge results of point 5 in the VGE dataset obtained by all comparison methods. (a) Arc-Support. (b) AAMED. (c) Proposed method.

4. Conclusions

This study presents a global automatic detection method employing multi-level constraints designed to accurately recognize and detect circular markers in high-speed videogrammetry. The proposed approach fully integrates both geometric features and grayscale characteristics of circular markers to mitigate false and missed detections. Experimental results on three high-speed videogrammetry datasets validated the efficacy of the p roposed method, demonstrating its ability to overcome the limitations associated with state-of-the-art methods. Thus, the proposed method holds promise for efficient and wide-ranging applications in the recognition and detection of circular markers in high-speed videogrammetry. For future work, the proposed method will be further improved and tested to automatically detect circular markers under different complex scenes in high-speed videogrammetry.

Disclosures

The authors declare no conflicts of interest.

Code and Data Availability

Data sharing is not applicable to this paper due to copyright and commercial restrictions.

Acknowledgments

The authors would like to thank the Tongji University for providing the high-speed videogrammetry images. This research was funded by the Natural Science Foundation of Fujian Province, China (Grant Nos. 2023J011415 and 2023J05257); the Young and Middle-Aged Teacher Education Research Project of Fujian Province, China (Grant No. JAT220327); and the Startup Foundation for Introducing Talent of Minjiang University (Grant Nos. MJY22018 and MJY23002).

References

1.

S. W. Kim et al., “Vision-based monitoring system for evaluating cable tensile forces on a cable-stayed bridge,” Struct. Health Monit., 12 440 –456 https://doi.org/10.1177/1475921713500513 (2013). Google Scholar

2.

H. S. Park et al., “A new position measurement system using a motion-capture camera for wind tunnel tests,” Sensors, 13 12329 –12344 https://doi.org/10.3390/s130912329 SNSRES 0746-9462 (2013). Google Scholar

3.

X. L. Liu et al., “Videogrammetric technique for three-dimensional structural progressive collapse measurement,” Measurement, 63 87 –99 https://doi.org/10.1016/j.measurement.2014.11.023 0263-2241 (2015). Google Scholar

4.

X. H. Tong et al., “Tri-camera high-speed videogrammetry for three-dimensional measurement of laminated rubber bearings based on the large-scale shaking table,” Remote Sens., 10 1902 https://doi.org/10.3390/rs10121902 (2018). Google Scholar

5.

X. H. Tong et al., “Acceleration of object tracking in high-speed videogrammetry using a parallel OpenMP and SIMD strategy,” Photogramm. Rec., 34 174 –197 https://doi.org/10.1111/phor.12279 PGREAY 0031-868X (2019). Google Scholar

6.

X. Y. Yu et al., “Oriented object detection in aerial images based on area ratio of parallelogram,” J. Appl. Rem. Sens., 16 034510 https://doi.org/10.1117/1.JRS.16.034510 (2022). Google Scholar

7.

T. Y. Zhang et al., “Generalized few-shot object detection in remote sensing images,” ISPRS-J. Photogramm. Remote Sens., 195 353 –364 https://doi.org/10.1016/j.isprsjprs.2022.12.004 (2023). Google Scholar

8.

J. Yang and T. Y. Wang, “Small object detection model for remote sensing images combining super-resolution assisted reasoning and dynamic feature fusion,” J. Appl. Rem. Sens., 18 028503 https://doi.org/10.1117/1.JRS.18.028503 (2024). Google Scholar

9.

S. Tu et al., “Automatic target recognition scheme for a high-resolution and large-scale synthetic aperture radar image,” J. Appl. Rem. Sens., 9 096039 https://doi.org/10.1117/1.JRS.9.096039 (2015). Google Scholar

10.

D. He et al., “Occluded target recognition in SAR imagery with scattering excitation learning and channel dropout,” IEEE Geosci. Remote Sens. Lett., 20 1 –5 https://doi.org/10.1109/LGRS.2023.3266395 (2023). Google Scholar

11.

H. Wang et al., “Buried target detection method for ground penetrating radar based on deep learning,” J. Appl. Rem. Sens., 16 018503 https://doi.org/10.1117/1.JRS.16.018503 (2022). Google Scholar

12.

J. Zhou, C. M. Kwan and B. Ayhan, “Improved target detection for hyperspectral images using hybrid in-scene calibration,” J. Appl. Rem. Sens., 11 035010 https://doi.org/10.1117/1.JRS.11.035010 (2017). Google Scholar

13.

R. X. Li and S. Latifi, “Improving hyperspectral subpixel target detection using hybrid detection space,” J. Appl. Rem. Sens., 12 015022 https://doi.org/10.1117/1.JRS.12.015022 (2018). Google Scholar

14.

F. Meng et al., “Visual detection and association tracking of dim small ship targets from optical image sequences of geostationary satellite using multispectral radiation characteristics,” Remote Sens., 15 2069 https://doi.org/10.3390/rs15082069 (2023). Google Scholar

15.

M. Gong and Y. Shu, “Real-time detection and motion recognition of human moving objects based on deep learning and multi-scale feature fusion in video,” IEEE Access, 8 25811 –25822 https://doi.org/10.1109/ACCESS.2020.2971283 (2020). Google Scholar

16.

S. Li et al., “A multitask benchmark dataset for satellite video: object detection, tracking, and segmentation,” IEEE Trans. Geosci. Remote Sens., 61 1 –21 https://doi.org/10.1109/TGRS.2023.3278075 IGRSD2 0196-2892 (2023). Google Scholar

17.

E. Kim, M. Haseyama and H. Kitajima, “Fast and robust ellipse extraction from complicated images,” in Proc. 1st Int. Conf. Inf. Technol. Appl., 357 –362 (2002). Google Scholar

18.

F. Mai et al., “A hierarchical approach for fast and robust ellipse extraction,” Pattern Recognit., 41 2512 –2524 https://doi.org/10.1016/j.patcog.2008.01.027 (2008). Google Scholar

19.

R. G. Von Gioi et al., “LSD: a fast line segment detector with a false detection control,” IEEE Trans. Pattern Anal. Mach. Intell., 32 722 –732 https://doi.org/10.1109/TPAMI.2008.300 ITPIDJ 0162-8828 (2010). Google Scholar

20.

A. Y. S. Chia et al., “A split and merge based ellipse detector with self-correcting capability,” IEEE Trans. Image Process., 20 1991 –2006 https://doi.org/10.1109/TIP.2010.2099127 IIPRE4 1057-7149 (2011). Google Scholar

21.

M. Fornaciari et al., “A fast and effective ellipse detector for embedded vision applications,” Pattern Recognit., 47 3693 –3708 https://doi.org/10.1016/j.patcog.2014.05.012 (2014). Google Scholar

22.

V. Pătrăucean et al., “Joint a contrario ellipse and line detection,” IEEE Trans. Pattern Anal. Mach. Intell., 39 788 –802 https://doi.org/10.1109/TPAMI.2016.2558150 ITPIDJ 0162-8828 (2017). Google Scholar

23.

Z. Ye et al., “An improved subpixel phase correlation method with application in videogrammetric monitoring of shaking table tests,” Photogramm. Eng. Remote Sens., 84 579 –592 https://doi.org/10.14358/PERS.84.9.579 (2018). Google Scholar

24.

X. H. Tong et al., “Liquid-level response measurement using high-speed videogrammetry with robust multiple sphere tracking,” Measurement, 228 114290 https://doi.org/10.1016/j.measurement.2024.114290 0263-2241 (2024). Google Scholar

25.

C. S. Lu et al., “Circle detection by arc-support line segments,” in Proc. IEEE Int. Conf. Image Process., 76 –80 (2017). Google Scholar

26.

A. M. Sims, S. C. Kienle and J. Leifer, “Videogrammetry-based position monitoring using inherent surface targets,” in Conf. Proc. Soc. Exp. Mech. Ser., 741 –753 (2008). Google Scholar

27.

, “PhotoModeler scanner,” http://www.photomodeler.com/products/scanner/ (). Google Scholar

28.

L. M. Galantucci et al., “New method to calibrate and validate a high-resolution 3D scanner, based on photogrammetry,” Precis. Eng.-J. Int. Soc. Precis. Eng. Nanotechnol., 38 279 –291 https://doi.org/10.1016/j.precisioneng.2013.10.002 (2014). Google Scholar

29.

S. Del Pizzo et al., “A vision-based navigation system for landing procedure,” ACTA IMEKO, 7 102 –109 https://doi.org/10.21014/acta_imeko.v7i2.533 (2018). Google Scholar

30.

U. Papa, Embedded Platforms for UAS Landing Path and Obstacle Detection, 63 –79 Springer, Cham (2018). Google Scholar

31.

S. Z. Zheng et al., “A high-precision elliptical target identification method for image sequences,” in IGARSS 2018. 2018 IEEE Int. Geosci. and Remote Sens. Symp., 3374 –3377 (2018). https://doi.org/10.1109/IGARSS.2018.8517497 Google Scholar

32.

X. L. Liu et al., “High-speed videogrammetric measurement of the deformation of shaking table multi-layer structures,” Measurement, 154 107486 https://doi.org/10.1016/j.measurement.2020.107486 0263-2241 (2020). Google Scholar

33.

X. L. Liu et al., “EET-Hamming monocular high-speed measurement for long-span bridge structure displacement on a shaking table,” Measurement, 211 112591 https://doi.org/10.1016/j.measurement.2023.112591 0263-2241 (2023). Google Scholar

34.

C. S. Lu et al., “Arc-Support line segments revisited: an efficient, high-quality ellipse detection,” IEEE Trans. Image Process., 29 768 –781 https://doi.org/10.1109/TIP.2019.2934352 IIPRE4 1057-7149 (2019). Google Scholar

35.

X. L. Liu et al., “High-speed videogrammetry with mutually guided target tracking under occlusion for masonry building structure displacement on a shaking table,” Buildings, 13 2959 https://doi.org/10.3390/buildings13122959 (2023). Google Scholar

36.

C. Meng et al., “Arc adjacency matrix-based fast ellipse detection,” IEEE Trans. Image Process., 29 4406 –4420 https://doi.org/10.1109/TIP.2020.2967601 IIPRE4 1057-7149 (2020). Google Scholar

37.

X. H. Tong, “Monitoring a progressive collapse test of a spherical lattice shell using high-speed videogrammetry,” Photogramm. Rec., 32 230 –254 https://doi.org/10.1111/phor.12202 PGREAY 0031-868X (2017). Google Scholar

38.

S. Gao et al., “Development of a robust high-speed videogrammetric technique for measuring large-scale shaking table tests,” Proc. SPIE, 11051 1105104 https://doi.org/10.1117/12.2523314 PSISDG 0277-786X (2019). Google Scholar

39.

S. Gao et al., “Development of a high-speed videogrammetric measurement system with application in large-scale shaking table test,” ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., IV-2/W7 33 –38 https://doi.org/10.5194/isprs-annals-IV-2-W7-33-2019 (2019). Google Scholar

40.

B. K. Wang et al., “3D deformation monitor with a distributed high-speed videogrammetry system based on shaking table experiment,” Proc. SPIE, 11051 110510K https://doi.org/10.1117/12.2524691 PSISDG 0277-786X (2019). Google Scholar

41.

X. L. Liu, “High-speed videogrammetry for seismic performance of the spherical reticulated shell structure on the shaking table,” Buildings, 13 553 https://doi.org/10.3390/buildings13020553 (2023). Google Scholar

42.

Z. K. Jia, X. L. Liu and R. J. Wang, “Displacement measurement of urban ramp bridge model using monocular high-speed camera,” J. Phys.: Conf. Ser., 2519 012056 https://doi.org/10.1088/1742-6596/2519/1/012056 JPCSDZ 1742-6588 (2023). Google Scholar

43.

H. B. Shi et al., “High-speed videogrammetric measurement of the displacement of suspendome structure node,” Proc. SPIE, 12980 129802K https://doi.org/10.1117/12.3020958 PSISDG 0277-786X (2024). Google Scholar

44.

X. H. Tong, “Dynamic measurement of a long-distance moving object using multi-binocular high-speed videogrammetry with adaptive-weighting bundle adjustment,” Photogramm. Rec., 39 294 –319 https://doi.org/10.1111/phor.12485 PGREAY 0031-868X (2024). Google Scholar

45.

J. A. Canny, “Computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell., PAMI-8 (6), 679 –698 https://doi.org/10.1109/TPAMI.1986.4767851 ITPIDJ 0162-8828 (1986). Google Scholar

46.

P. Soille, Morphological Image Analysis: Principles and Applications, 219 –238 Springer-Verlag, Berlin (2004). Google Scholar

47.

R. M. Haralick and L. G. Shapiro, Computer and Robot Vision, 647 –653 Addison-Wesley Longman Publishing Co., Boston, Massachusetts (1992). Google Scholar

48.

M. R. Shortis, T. A. Clarke and T. Short, “Comparison of some techniques for the subpixel location of discrete target images,” Proc. SPIE, 2350 239 –250 https://doi.org/10.1117/12.189136 PSISDG 0277-786X (1994). Google Scholar

Biography

Shouzhu Zheng is currently an assistant professor at the College of Geography and Oceanography, Minjiang University, Fuzhou, China. His research interests include photogrammetry and remote sensing, video object detection, and tracking.

Sicong Liu is currently an associate professor with the College of Surveying and Geo-Informatics, Tongji University, Shanghai, China. His research interests include multitemporal data analysis, change detection, and multi-spectral/hyperspectral remote sensing in Earth observation and planetary exploration.

Zhen Ye (Member, IEEE) received his PhD degree in cartography and geoinformation from Tongji University, Shanghai, China, in 2018. He was a postdoctoral researcher with the Chair of Photogrammetry and Remote Sensing, Technical University of Munich, Munich, Germany, from 2018 to 2020. He is currently an associate professor with the College of Surveying and Geoinformatics, Tongji University. His research interests include photogrammetry and remote sensing, high-resolution satellite image processing, and planetary 3-D mapping.

Xiaolong Ma received his ME degree in geographic information engineering from the Chinese Academy of Surveying and Mapping, Beijing, China, in 2014, and his PhD in surveying and mapping from Tongji University, Shanghai, China, in 2019. He is currently an associate research fellow with the Institute of Cartography and Geographic Information System, Chinese Academy of Surveying and Mapping. His research interests include nighttime light remote sensing data analysis, urban information extraction, and geographic information security protection.

Baohang Wang received his MS and PhD degrees in Geodesy and Surveying Engineering from Chang’an University, Xi’an, China, in 2018 and 2022, respectively. He is currently a lecturer of geodesy and survey engineering with the College of Geography and Oceanography, Minjiang University, Fuzhou, China. His research interests include the development of different InSAR methods for artificial infrastructure facilities, land subsidence, and landslide.

Jianxia Zhang received his PhD in cartography and geographical information system from Capital Normal University, Beijing China, 2010. He is currently an assistant professor with the College of Geography and Oceanography, Minjiang University, Fuzhou, China. He is specialized in the aerial photogrammetry applications.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 International License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Shouzhu Zheng, Sicong Liu, Zhen Ye, Xiaolong Ma, Baohang Wang, and Jianxia Zhang "Global automatic detection method employing multi-level constraints for circular markers in high-speed videogrammetry," Journal of Applied Remote Sensing 19(1), 016501 (28 November 2024). https://doi.org/10.1117/1.JRS.19.016501

Received: 16 July 2024; Accepted: 14 November 2024; Published: 28 November 2024

Access the abstract

JOURNAL ARTICLE
19 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

Subscribe to Digital Library

Receive Erratum Email Alert

1.

Introduction

2.

Methodology

Fig. 1

Fig. 2

2.1.

Canny Detection

Eq. (1)

Eq. (2)

Eq. (3)

2.2.

General Geometric Constraint

Eq. (4)

2.3.

Second Geometric Constraint Based on Roundness Metric Condition

2.3.1.

Morphological reconstruction based on geodesic distance

Eq. (5)

Eq. (6)

2.3.2.

Roundness metric condition

Eq. (7)

2.4.

Extrema Point Condition Constraint

Eq. (8)

Fig. 3

Fig. 4

3.

Results and Discussion

3.1.

Datasets Descriptions

3.1.1.

Five-layer frame model (FFM) dataset

Fig. 5

3.1.2.

Three-story frame model (TSF) dataset

3.1.3.

Vibration generator experiment (VGE) dataset

3.1.4.

High-rise building model (HRB) dataset

3.1.5.

Brick and concrete structure wall (BCSW) dataset

3.2.

Experimental Setup and Parameter Settings

3.3.

Results and Analysis

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

Fig. 12

Fig. 13

Table 1

Table 2

3.3.1.

Recognition process analysis of the proposed method

3.3.2.

Recognition accuracy comparison

Fig. 14

Fig. 15

Table 3

Fig. 16

4.

Conclusions

Disclosures

Code and Data Availability

Acknowledgments

References

Biography

Keywords/Phrases

Search In:

Publication Years