Skip to main content

Trademark infringement recognition assistance system based on human visual Gestalt psychology and trademark design


Trademarks are common graphic signs in human society. People used this kind of graphic sign to distinguish the signs of representative significance such as individuals, organizations, countries, and groups. Under effective use, these graphic signs can bring maintenance and development resources and profits to the owner. In addition to maintenance and development, organizations that have obtained resources can further promote national and social progress. However, the benefits of these resources have also attracted the attention of unfair competitors. By imitating counterfeit trademarks that appear, unfair competitors can steal the resources of the original trademark. In order to prevent such acts of unfair competitors, the state has formulated laws to protect trademarks. In the past, there have also been researches on similar trademark searches to assist in trademark protection. Although the original trademark is protected by national laws, unfair competitors have recently used psychological methods to counterfeit the original trademark and steal its resources. Trademarks counterfeited through psychology have the characteristics of confuse consumers and do not constitute infringement under the law. Under the influence of such counterfeit trademarks, the original trademark is still not well protected. In order to effectively prevent such trademark counterfeiting through psychology, this article proposes new features based on trademark design and Gestalt psychology to assist legal judgments. These features correspond to a part of the process that is not fully understood in the human visual system and quantify them. In the experimental results, we used past cases to analyze the proposed assistance system. Discussions based on past judgments proved that the quantitative results of the proposed system are similar to the plaintiff or the judgment to determine the reasons for plagiarism. This result shows that the assistance system proposed in this article can provide visually effective quantitative data, assist the law to prevent malicious plagiarism on images by unfair competitors, and reduce the plagiarism caused by the similar design concepts of late trademark designers.


Trademarks are graphics that can be seen everywhere in human society. People use it as a recognition mark to distinguish countries, groups, and other representative signs that represent individuals or organizations [1]. These trademarks usually consist of various well-known entities or abstract graphics and symbols. In addition, they may also be hearing, smell, behavior, or graphics with distinctive features [2]. The origin of the trademark is that ancient craftsmen signed or marked their artwork or practical products to recognition the producer or guarantee the quality of the product. These signatures and marks have evolved to this day and become today’s trademark registration and protection system [3]. Trademarks have the following three common functions [4]:

  • Recognition source or ownership

  • Ensure that the goods or services have the same level of quality or characteristics

  • Advertising

After effectively performing the above functions, ordinary people can quickly understand the image and reputation of the organization or product through the trademark, and the organization with the trademark can also obtain the resources to maintain and develop the organization itself.

Under the condition of effective use of trademarks, it can obtain maintenance and development resources for organizations that own trademarks. However, these resources have also drew the attention of unfair competitors. Unfair competitors imitate similar trademarks to confuse people by its similarity. When people are confused, the resources that originally belong to the trademark owner will be stolen by unfair competitors. Moreover, unfair competitors damage the reputation of the trademark through inferior products or services. These conditions have led to the effect that trademark owners are unable to obtain sufficient resources to maintain or develop the organization. At the same time, it further prevents organizations from developing new technologies that contribute to the country and society. In order to advance the technology level of society and protect the right of the trademark owner, many countries allow trademark owners to register their trademarks. This method allows registered trademarks to be protected by national laws and prevents unfair competitors from indirectly preventing national progress.

While unfair competitors intend to steal the resources of the trademark organization, they will imitate similar trademarks to deceive people, steal the resources that the organization deserves, and damage the image and reputation of the trademark. Assuming that the behavior of unfair competitors is not prevented, the organization that owns the trademark would lost sufficient resources to maintain and develop itself. Furthermore, such behavior prevents the organization from developing novel technologies that contribute to the progress of the country and society. Therefore, many countries allow trademark-owned organizations to register their trademarks. After legitimate registration, trademark owners can claim their rights protected by national laws and prevents unfair competitors from stealing resources and profits.

However, even though registered trademarks seem already to be protected by law, there are still endless cases of trademark infringement in the world. In these cases, in addition to malicious infringement by unfair competitors, there are also some latecomers who violated the law since they are unfamiliar with local legislation. In addition, Zhu and Jin mentioned the difference in legal judgment between reverse confusion and forward confusion [5]. In their research, it is explained that forward confusion is the original trademark already well-known in the region. However, the reputation of the trademark was used by unfair competitors to steal the resources of the original trademark. And reverse confusion means that other trademarks similar to the original trademark are more well-known in the region than the original trademark, causing consumers to misunderstand that the original trademark is a subsidiary of the trademark. This situation makes it possible for unfair competitors to imitate well-known trademarks and then operate in areas not covered by well-known trademarks. When a well-known trademark wants to step into the area, they find that their future market has been plundered. The study believes that legal cases should be judged in accordance with the round and order in order to effectively regulate the infringement caused by reverse confusion. Therefore, the way of related work in the past was to search for similar features from a huge trademark database to check whether the trademark constitutes infringement for research purposes. In related works of searching for trademarks using digital images, a common method was to extract features from the high-frequency images and then find the nearest trademark through the distance between the features [6, 7]. In addition, there are also methods to directly use the binarized image as a feature to perform image matching search [8]. Meenalochini et al. [9] proposed the use of perceptual hashing algorithm to search for similar images. Showkatramani et al. [10] utilized the recently popular convolutional neural network (CNN) to train a variety of trademarks in different environments and implemented the computer vision trademark identification system of trademarks in different environments. Different from the use of images in trademark recognition, such as Anuar et al. [11, 12] proposed to use Tversky’s theory of similarity to search for the similarity between semantic meanings based on the semantic combination in trademarks. In the work of Yan et al. [13], they proposed to introduce multi-view deep neural network into the field of hash learning to design a high-performance trademark image retrieval model. In the experimental results, the NUS-WIDE and MS-COCO data sets are used to evaluate the model, and the model is better than the state-of-the-art single-view and multi-view hashing methods. In their research, Yan et al. believe that some trademarks are three-dimensional rather than flat. Therefore, they also proposed some three-dimensional image processing work [14, 15] to assist in trademark-related search processing. In addition, Yan et al. also discussed the research on the use of natural language in trademarks [16]. Abadi et al. [17] used artificial intelligence to classify trademarks registered by the US Trademark Office. Trappey et al. [18] used convolutional neural network and Siamese neural network to develop a method combining natural language and image similarity analysis. This method can search for potential infringements of trademark images and text and protect the intellectual property rights of trademark owners.

The related work described above has significant results in similar trademark inspections and classifications. Nevertheless, unfair competitors recently started to produce similar trademarks through Gestalt psychology. This kind of similar trademark not only confuses people with the original registered trademark, but also avoids the similar characteristics of the trademark, which make it difficult to judge whether they are plagiarism in related works. Related reports are as described in [19, 20] literature, this article uses Fig. 1 as an example. Suppose there is a well-known trademark as shown in Fig. 1a, and an unfair competitor wants to use the reputation of the well-known trademark to draw people’s attention. Considering the basis of legal judgment, unfair competitors refer to the existing cognition of Gestalt psychology and combine their own trademarks with well-known trademarks to produce infringing trademarks such as Fig. 1b. Under normal circumstances, people can easily distinguish the difference between the two trademarks. However, as shown in Fig. 1c, when the distance in the human visual system and the influence of the surrounding environment are considered, people will confuse the two trademarks due to the change of characteristics. Just when people are confused due to their existing cognition, infringing trademarks have already robbed benefits from well-known trademarks and at the same time damaged the reputation and image of well-known trademarks.

Fig. 1

Illustrate a hypothetical trademark case that uses visual psychology to confuse human vision. a Hypothetical well-known trademark. b Hypothetical counterfeit trademark. (c) Examples of possible confusion in environmental impact

In the past actual cases [21], since there was no suitable image analysis system to assist in legal judgments, most cases were judged based on the semantic meaning of trademarks. Also take Fig. 1 as an example. In semantic analysis, the trapezoid is a general figure and cannot be used as a basis for recognition. And the symbols appearing in the two trademarks cannot be regarded as similar trademarks since they are “Gesture” and “Glasses” respectively. Although in this case we had an objective and fair semantic judgment, we ignored that the judgment did not consider two trademarks from aspect of human visual system. This result has enabled unfair competitors in recent years to use Gestalt psychology to steal the reputation of the original trademark. Some consumers even misunderstood the counterfeit trademarks of unfair competitors as original trademarks.

Considering the influence of Gestalt psychology on trademark reputation, this paper refers to and attempts to implement a system that includes elements of attracting people in trademark design [22, 23]. In trademark design, these attractive elements are called “visual weight” [22, 23], and the details will be introduced in Section 2. Based on the elements mentioned in trademark design, this article realizes and extracts corresponding features in trademark image processing. We discuss the impact of these characteristics in the experimental results section and use past cases to discuss the correctness of the assistance system.

The main contributions of this article are as follows:

  • According to trademark design and Gestalt psychology, a new seven-feature visual judgment aid system is proposed. These features correspond to a part of the process that is not fully understood in the human visual system and quantify them.

  • Discuss the cases that have been finalized in the past based on the experimental results and discuss the differences before and after considering the similarity of visual images.

  • Provide a visual-based trademark judgment system to assist the law to prevent malicious plagiarism on images by unfair competitors and reduce plagiarism due to similar design concepts by less advanced trademark designers.

In the next section, we introduce trademark design and visual psychology. Subsequently, the “visual weight” element in the implementation of the trademark design is explained, and the system flowchart proposed in this article is explained. In the experimental results, we use some cases to verify the system proposed in this article. When verifying, we explain the relationship between existing trademark search methods, trademark design, and Gestalt psychology to explain why the existing methods cannot assist legal judgments. Finally, we will add the actual judgments of some cases to this system for discussion, explaining that visual image and semantic analysis are equally important factors in judging trademark similarity. At the same time, it explains that the system proposed in this article can help latecomers reduce plagiarism caused by similar design concepts.


Trademark design and Gestalt psychology

In the trademark design, Bradley [22, 23] proposed an influential element named “visual weight” based on human vision. The main purpose of “visual weight” is to make the designed trademark have characteristics that can draw human attention and represent the image of the organization or product. The original features mentioned by Bradley in “visual weight” are as follows:

  • Size. For human attention, large elements are more attractive than small elements.

  • Shape. Since the impression of irregular shapes is easily replaced by regular shapes, objects with regular shapes are more impressive than those with irregular shapes.

  • Color. Warm-colored objects are easily recognized as the foreground, while cool-colored objects are easy to blend with the background. In addition, red is considered the darkest color, and yellow is considered the lightest color.

  • Value. Dark elements are easier to notice than light elements.

  • Texture. Texture makes the element look like a three-dimensional object, giving the appearance of mass and physical weight. Therefore, textured elements draw human attention more than untextured objects.

  • Position. The foreground object is easier to become the main area of the logo than the background object. The farther an element is from the main area, the easier it will be noticed.

  • Orientation. In the display direction of the object, diagonal display with diagonal elements can give people the strongest impression, followed by vertical display, and horizontal display gives the lowest impression.

Gestalt psychologist Koffka explained the theory of human visual perception through the proposed perceptual organization [24]. Koffka believes that when everyone recognizes the images they see, including children and uncivilized people, they recognize the meaning of things in front of them based on organizational experience. In perceptual organization, the following points describe human visual perception:

  • The stronger the difference between the foreground graphic object and the background, the more it can become the focus of human vision.

  • Close graphic objects can easily form a whole.

  • Coordinated graphic objects are easy to form a whole, and uncoordinated graphic objects are easy to be separated.

  • When the distance between each part is equal, the graphic objects of the same color will form a whole.

  • According to people’s existing cognition, even if it changes drastically, it can be recognized from its characteristics.

  • If the graphic objects have directionality, graphic objects with the same directionality can easily form a whole.

Proposed trademark graphic recognition features and trademark similarity comparison system based on visual weight

From the description of the trademark design in the previous section, we can learn that how to draw human attention through vision, and to associate its image and reputation, has always been the purpose of a trademark. However, in the case of trademark judgments, due to the lack of objective image judgment methods, the judgments have been implemented through semantic meaning. This approach gives unfair competitors an opportunity to constantly use similar graphics but different semantics to counterfeit well-known trademarks as shown in Fig. 1. Due to the different semantics of counterfeit trademarks, trademarks cannot be protected by national laws. Visually, counterfeit trademarks confuse consumers and further steal the resources and profits of well-known trademarks. In order to more reliably protect the image and reputation of trademarks, there must be a system that can assist in the judgment of trademarks on graphics. The graphic judgment system is not to replace semantic judgment but to jointly deal with the problem of counterfeit trademarks to achieve the effect of trademark protection. Therefore, based on the visual perception theory and visual weight characteristics described in the previous section, this article proposes a system to assist in the judgment of trademark graphics.

Proposed trademark graphic recognition features based on visual weight

The first feature mentioned by visual weight is size. Using human vision to differentiate the size characteristics of the two trademarks is to compare the difference in size between the two. In image processing, the simplest and similar processing method is to take out the trademark that separates the background and calculate the difference between the two pixels. Since trademarks have different scales in each layout, this article uses all pixels in the region of interest (ROI) as the normalization parameter NSIZ of the feature. Considering that most images nowadays use lossy compressed images for storage, we use adaptive threshold [25] processing in the work of separating foreground and background. After obtaining the best foreground distance through basic image processing, the system would count its pixels to obtain the trademark pixel TSIZ. Through the normalization of formula (1), the normalized trademark size feature F1 is obtained.

$$ F_{1}=\frac{T_{SIZ}}{N_{SIZ}} $$

The second feature is shape. Also considering the changes of trademark images in various layouts, the shape feature does not use the perimeter or binarized area of the trademark object. This article extracts the shape feature F2 using circularity for quantification to reduce the impact of layout changes. When the circularity formula calculates a graphic object, it uses the area of the graphic object for quantitative calculation, so that the circle and the hollow ring present different values. Due to trademark objects, even if there is a gap between the graphic objects, each object is designed to have a certain degree of coordination. In the description of perceptual organization, these coordinated graphic objects make the trademark appear as a whole in human vision. Therefore, when we use the circularity formula to quantify the shape feature, we use the area of the trademark object as the variable for the quantitative calculation according to the previous description. The circularity calculation method is shown in formula (2), TP is the circumference of the trademark object.

$$ F_{2}=\frac{4\pi T_{SIZ}}{(T_{P})^{2}} $$

The third feature is color. When extracting the characteristics of color, in addition to the knowledge of visual perception theory and trademark design, the deactivation reaction of human vision should still be considered [26]. The visual deactivation reaction is that humans produce optical illusion such as complementary colors and persistence of vision based on the density or color difference between the foreground and background of the object. Considering the visual perception theory, trademark design, and visual deactivation, the color feature F3 is extracted as formula (3) in this article. Where Tr,c represent the pixels in the processing trademark, RTM and CTM are the rows and columns of the trademark object, respectively, and ch means the image channels.

$$ F_{3}=\sqrt{\sum_{ch} \left(\frac{\sum_{r}^{R_{TM}}\sum_{c}^{C_{TM}}(T_{r,c})}{R_{TM}\times C_{TM}} \right)^{2} } $$

The fourth characteristic is value. In the trademark design, value pointed out that darker elements are easier to be noticed than lighter elements. The main reason is that lighter elements give humans a lower sense of oppression than dark-colored elements, so that the elements in human society have more light-colored elements [23]. Among the many lighter elements, embedding an object with a darker element will make the object easy to be noticed. This kind of strongly discontinuous picture is called contrast in image processing. In general trademark design, in order to increase the chance of the trademark being noticed by people, the contrast between the background and the foreground is quite significant. Therefore, the feature F4 of the extracted value in this article focuses on the contrast of the foreground, and the calculation method is shown in formulas (4) and (5). Where the symbol in formula (4) is the convolution operation, and Tmsk is the coordinate of the trademark pixel. Formula (5) is the common gray-scale processing, and Pg is the defined gray-scale parameter [26].

$$ F_{4}=\frac{\max_{r,c}(G_{r,c}\ast T_{msk})-\min_{r,c}(G_{r,c}\ast T_{msk})}{\max_{r,c}(G_{r,c}\ast T_{msk})+\min_{r,c}(G_{r,c}\ast T_{msk})} $$
$$ G_{r,c}=\sum_{ch}\left(T_{r,c}(ch) \times P_{g}(ch) \right), \qquad P_{g}=[0.299, 0.587, 0.144] $$

The fifth feature is texture. In the description of visual weight, the texture feature combines trademarks with common things in human life to increase attractiveness. Common things include elements such as light sources, shadows, and three-dimensional perspectives, which make the logo graphics on the plane feel like three-dimensional objects. When the same figure is combined with different texture features, visual perception can not only recognize the figure, but also the type of combined texture features. From the perspective of the frequency domain of the signal system, the texture feature only changes the high-frequency information of the graphics. The original meaning of graphics at low frequencies has not been changed by texture features. In image processing, the basic feature of object recognition is to extract the corresponding high-frequency features from the object image. This feature is also considered in the related literature based on image processing in the related works [6, 7]. Therefore, the extraction method of texture feature F5 in this article uses the image high-frequency feature extraction method.

Position is the penultimate feature. Bradley described the position feature as a foreground object that easily becomes the main area of a trademark. Moreover, he also pointed out that when some elements are far away from the main area, these elements will become the visual focus. In this part of this article, we use formulas (6) and (7) to find the centroid (Cxi′,Cyi′) of each object. Then, combine the centroid of each object through formula (8). Finally, the position feature F6 is normalized and calculated by formula (9), where i is the index of objects, Sy(x) is the length of the y-axis at coordinate x, Sx(y) is the length of the x-axis at coordinate y, and OA is the area of the object.

$$ C'_{xi}=\frac{\int xS_{y} (x)dx}{O_{Ai}} $$
$$ C'_{yi}=\frac{\int yS_{x} (y)dy}{O_{Ai}} $$
$$ (C'_{xi},C'_{yi})=\left(\frac{\sum C'_{xi}O_{Ai}}{T_{SIZ}}, \frac{C'_{yi}O_{Ai}}{T_{SIZ}} \right) $$
$$ F_{6}=\sqrt{\sum_{a \in (x,y)} \left(\frac{C'_{a}}{\alpha_{TM}} \right)}, \qquad \begin{array}{c} a=x, \alpha=C\\ a=y, \alpha=R \end{array} $$

The last feature is orientation. Visual weight means that when a graphic is composed of multiple objects, objects with the same orientation are easily regarded as the same object by visual perception, and objects with different orientations are regarded as different objects. In this article, we use an ellipse to approximate the irregular shape of the trademark. Then, the angle between the main axis of the ellipse and the horizontal line is taken as the original orientation feature. Finally, the original orientation feature is normalized to [0, 1] as the trademark orientation feature F7.

Trademark similarity comparison system based on visual weight

After introducing the recognition features proposed based on visual weight, we introduce the proposed trademark similarity comparison system. Figure 2 shows the flow chart of the proposed system. First, the original and counterfeit trademark image data input based on actual cases. From the input image data, the seven normalized features of size, shape, color, value, texture, position, and orientation are extracted according to the description of the previous visual weight. From the features extracted from the case images, use formulas (10) and (11) to calculate the standard deviation σk. Where k represents the characteristic index, oi is sample index.

$$ D_{k}=\lVert F_{k}-\bar{F}_{k} \rVert_{2} $$
Fig. 2

Proposed the implementation flow chart of proposed trademark similarity comparison system

$$ \sigma_{k}=\sqrt{\frac{1}{M}\sum_{o_{i}=1}^{M} (D_{k}-\mu_{k})^{2}} $$

Next, we use formulas (12) and (13) to calculate the feature quantity threshold Tσ that takes into account human visual errors. After obtaining the characteristic quantity valve value Tσ, calculate the valve value \(\mathbb {T}\) according to formulas (14) to (16). In the formula, Mo(·) represents the mode, Q1(·) is the first interquartile difference, and wk is the weight of the feature. The finally obtained valve value \(\mathbb {T}\) is the valve value that assists the system to judge.

$$ T_{\sigma}=\text{Mo}(\mathbb{F}_{k}) $$
$$ \mathbb{F}_{k}=\lvert D_{k}<\sigma_{k} \rvert_{0}, \quad \forall \ k $$
$$ \mathbb{T}=Q_{1} (\mathcal{R}_{N}) $$
$$ \mathcal{R}_{N}=1-\omega+\sum_{k} D_{k} w_{k} $$
$$ \omega=\mathbb{F}_{k} \geq T_{\sigma} $$

Since in the human visual system [27], there are still biological and psychological processes that are not yet fully understood. Whether counterfeit trademarks cause human confusion is one of the parts we could discuss. At the same time, when humans recognize images through vision, if some of the features of the image are not obvious, the human brain will ignore the features and turn to use other organizational experience to recognize available features for identification [24]. Taking these factors into account, this article will further discuss the seven characteristics results in the experimental results and discuss it in conjunction with the previous known cases.

Experimental results and discussion

This implementation uses Graphis public trademark database [28], Trademark database of the Japan Platform for patent (JPP) [29], and Trademark database of the US Patent and Trademark Office (USPTO) [30]. The Graphis public trademark database has 5906 trademark images. There are 5450 JPEG images with a size of 670×670 pixels in the database, 254 images with a size less than 670×670 pixels, and 202 images with a size greater than 670×670 pixels. The trademark database of USPTO is a trademark database containing all trademarks registered in the USA.

Although there are still many cases related to trademark infringement in society today, it is still difficult to collect trademark images related to the case. Taking the difficulty of image search in actual cases into account, the counterfeit trademark images used in this article are mostly counterfeit trademarks similar to the original trademarks registered by the original trademark owners in advance. Such counterfeit trademarks are the mark that original trademark owners believe that they have the opportunity to confuse customers and further damage the reputation of the original trademark. Therefore, trademark owners first registered these numerous counterfeit trademarks to prevent unfair competitors from using similar methods to damage the reputation of the original trademarks.

Figure 3 is the statistics of the number of features that use pre-registered similar trademarks and different trademarks at a distance less than the standard deviation. There are 391 groups of pre-registered similar trademarks in the picture and 363 groups of different trademarks. It can be seen from Fig. 3 that among the seven features in similar trademarks, most of them have four or five feature distances that are less than the standard deviation. When the trademarks are not the same, there are usually two to three features whose distances are less than the standard deviation. Taking into account the error of the simulated human vision, the mode Mo(·) is used to identify the feature quantity threshold Tσ of similar trademarks.

Fig. 3

Statistics of the number of features whose distance is less than the standard deviation

Figure 4 shows the statistics of each feature whose distance is less than the standard deviation. In the figure, we can notice that the characteristic of F5, whether it is a similar trademark or not, is not below the standard deviation. This shows that compared with the past research using F5 feature, the search and recognition of digital trademark features are quite effective. F5 feature can clearly distinguish different trademarks and accurately search for similar trademarks by feature distance. Since the F5 feature is quite accurate, it cannot directly describe the confusion caused by the human visual system seeing the trademark. On the standpoint that trademarks are designed to attract consumers, this advantage has made it impossible for past research to be applied to assist the law and the recognition of similar trademarks.

Fig. 4

Statistics where the distance of each feature is less than the standard deviation

In addition, Fig. 4 allows us to further analyze the results of Fig. 3. From Fig. 4, we can notice the different trademark’s features in F3,F4, and F7, and the feature distance is still easily smaller than the standard deviation. The F3 feature is mainly attributed to color. If features of different shapes have the same color, the feature distance may be lower than the standard deviation. The F4 feature is mainly the contrast on the trademark. Most trademarks have the main goal of attracting human attention, and the mention of strong contrast in “visual weight” makes the trademark easy to attract attention. Therefore, the F4 feature is indeed easily smaller than the standard deviation in terms of distance comparison. The orientation of F7 features mainly depends on the ellipse encircled by the overall scope of the trademark. When the ellipse circled by the trademark arrangement position is similar, the calculated angle will be quite similar, resulting in the distance being easily smaller than the standard deviation.

Although the features of F3,F4, and F7 are easily misjudgmental, when analyzing trademark graphics in combination of features, they will become a key feature that makes human vision confused due to trademark similarity. Next, we will explain through actual case analysis.

In case [31], the proposed system only has a significant distance at F4 and F5, while other features are less than σk. The results of Table 1 show that it is as seen by human vision, that is, the two can be seen differently by careful observation, but other characteristics are quite similar, which is easy to cause confusion. The results of past judgments also indicated that the defendant’s trademark was judged in favor of the plaintiff due to it “may dilute the uniqueness of the plaintiff’s trademark.”

Table 1 Past trademark litigation cases were evaluated using the proposed image similarity resolution system

Case [32] is a recent case where the plaintiff was sentenced to lose. The results of Table 1 show that the proposed system is similar in features of F3 and F6. However, there is a huge gap in the characteristics of F2, which is like showing that the shape of the plaintiff and the defendant’s trademark is not easy to confuse people. In the actual judgment, the plaintiff in this case tended to file a complaint with similar meaning. The plaintiff believed that the meaning of the trademark “caught from the sea” was similar to the meaning of the defendant’s trademark “caught from the river.” Nevertheless, the judgment found that “sea” and “river” have a specific connection, but it will not confuse people. At the same time, after confirming that other items such as merchandise items and trademark signs would not confuse people, the final judgment indicated that the defendant did not constitute an infringement.

Case [33] is a case where the plaintiff was judged to lose the case many years ago. In this case, the plaintiff used the overall shape and color of the trademark design and imitation of some objects as the reasons for filing a lawsuit. The result of the judgment shows that the design of double concentric circles is a universal design figure and is less recognizable. As for the corrugated pattern in the trademark, the overall graphic of the plaintiff’s trademark represents long hair, but the overall graphic of the defendant’s trademark represents heat. Although the color difference was not described, the overall appearance, pronunciation, and semantics were completely different, which made the final judgment believe that there is no doubt of confusion. In the analysis results of Table 1 in this article, based on the similarities described by the plaintiff, the corresponding features are F2,F3, and F5. Feature F2 calculating the distance shows that although the whole object is visually double concentric circles, the pattern of the inner circle is different, and the area of the whole pattern makes the shape calculation result obviously different. From the analysis of feature F3, the plaintiff’s trademark is green and black, while the defendant is pure green. Normal people’s vision can easily detect the differences, just as the distance calculated from feature F3 is greater than 2σk. The calculation result of feature F5 is also greater than 2σk, indicating that the overall F5 difference between the two is far. The calculation results of the above three characteristics are in line with the results of human visual perception observing the trademarks of the two parties.

Case 1 and case 2 in [21] are also cases where the plaintiff lost. In case 1, the plaintiff believed that the defendant’s trademark was similar in shape, color, and design to the trademark owned by it, which violated the trademark rights. However, the verdict at that time indicated that “star” was a general term, with weak discernibility, and could not be used as a recognition criterion. “preya” and “bucks” have no semantic meaning. Pronunciation, “Starpreya” is pronounced as “Star Freya”, which is not similar in syllable to “Starbucks”. Moreover, the semantic meanings of the patterns in the trademarks were “Sirens-mermaids” and “Goddesses”, so that the final judgment was not considered to be similar trademarks. Similarly, the shape F2, color matching F3 and design pattern texture feature F5 recognized by the plaintiff were used to analyze the state of human attention to the trademark. When only looking at the shape feature F2, the calculation result is less than σk. However, it can be known from the judgment statement that “the design of double concentric circles is a universal design figure”, so the similarity cannot be recognized by this feature alone. Different from the previous case, in the F3 feature, due to the outer circle green and the inner circle black have the same color, the calculation result also less than σk. F5 texture feature is one of the few features larger than 2σk in this case. Although the verdict of this case was that the plaintiff lost the case, the characteristics described above and other characteristics of the system are less than 2σk. The combination of these results shows that the two trademarks are confused in human visual perception. However, the verdict was based on the difference between hearing and semantics, which lacked visual judgment. This verdict is similar to Fig. 1 in the introduction section. Without assistance in judging the pattern design standards, relying only on hearing, semantics, and other methods may enable unfair competitors to steal resources through similar methods.

In case 2, the plaintiff also filed a lawsuit against the defendant on the grounds of the shape and color of the trademark. The verdict in this case indicated that on the inner circle pattern, the trademark owned by the plaintiff was “Sirens-mermaids”, while the trademarks of the defendant were “mountain shape” and “Mt. RAINIER.” The semantics of the two trademarks are also completely different, so that the final judgment is not considered to be similar trademarks. Based on the reasons for reporting, the system proposed in this article also first examines the features F2,F3, and F5. The calculated result of feature F2 is less than σk, indicating that the two trademarks do have similar shapes visually. Feature F5 also shows the texture, the similarity is greater than 2σk. The difference from the previous case is the color scheme of feature F3. The F3 feature distance calculated in the proposed system is between 2σk and σk. In this case and the previous case, the color scheme is the same as green and black, but because of the different area of the color scheme, the human visual perception has a different perception of the combination of F2 and F3, which further recognizes the difference between the two.

Figure 5 shows two sets of similar trademark images in the USPTO trademark database [30]. Table 2 is an experimental comparison between the method in this article and the existing method using Fig. 5a and b trademark images. The values in the table are the normalized similarity distances. In Table 2, F1 to F7 are the seven features proposed in this article based on the principles of trademark design and the seven interpretation directions of Gestalt psychology on human visual subjective cognition. Corresponding to the size, shape, color, contrast, texture, position, and orientation distances between the two trademarks. The two columns of σk and 2σk are the standard deviation and twice the standard deviation of the database statistics used in this article.

Fig. 5

Illustrate a hypothetical trademark case that uses visual psychology to confuse human vision. a Ttwo similar data from [30] database. b Two different data from [30] database

Table 2 Comparison between the proposed method and the existing method in Fig. 5

In the test of Fig. 5a, the method of [6, 7] is similar to the F5 feature, so similar distances are obtained. After obtaining the distance using the [6, 7] method, the two brands can be obtained as similar images through the threshold. The method of [18] shows that the two images are similar images after passing the similarity threshold T. Each feature of the method in this article is different, and each feature can explain the human visual experience. The distance of F1 is between σk and 2σk, indicating that someone will see that the two images have a slight difference in size. The distance of F2 is greater than 2σk, which means that when a person sees two trademarks, they can quickly see that the shapes are very different. The distance of F3 is between σk and 2σk, which means that the color difference of the two trademarks is not very different, but we still feel that the colors of the two trademarks are slightly different. The distance of F4 is less than σk, indicating that there is no difference in the contrast between the two trademarks in attracting attention. The distance of F5 is greater than 2σk, which means that the texture of the two trademarks is different in human visual perception. The distance of F6 is less than σk, which means that the two trademarks are positioned in a similar position to human vision. The distance of F7 is less than σk, which means that the direction flow of the two trademarks is similar. The direction flow can explain whether the overall structure of the trademark is similar in human vision. Among the seven features, three characteristics are below the standard deviation and two characteristics are between the standard deviation and twice the standard deviation. This value can be obtained by formulas (12) to (16) with similar results for nearly 4 features. Like the statistical results of Fig. 3, the two trademarks can be given as similar trademarks.

In the test of Fig. 5b, the distance obtained by the related work [6, 7, 18] can be known through the similarity threshold, and it can be known that the image is not similar. In our proposed method, we can still explain the differences in human eyes between the two trademarks one by one according to seven features. At the end of the proposed method, the two features are less than the standard deviation. Also explained with Fig. 3, we get that the two trademarks are not similar.

The experimental result in Table 2 shows that the method proposed in this article and the past method can achieve the image retrieval function. However, in the actual verdict, even if the two trademarks are very similar in the eyes of the public, the judge’s understanding is still used as the basis of the judgment. The reason for this is that everyone has a subjective understanding of similar judgments of images. Although related work in the past has good methods and mathematical models and can accurately retrieve similar images, these methods and mathematical models can only show that the two trademarks are similar and cannot help explain which parts of the two trademarks are similar to cause confusion. The seven features of this article are based on the trademark design and Gestalt psychological design. In addition to the feature distance, these features also explain the human vision’s perception of image similarity. So that the features proposed in this article can assist the judgment in a more objective way in the future.

In the past systems used to recognition objects, the corresponding literature will explain the use of specified methods to combine features and then perform object identification based on the combined results. This article considers the visual perception theory of trademark design and Gestalt psychology. In trademark design and visual perception theories, they describe that when human vision recognizes objects or images, unobvious features will be ignored, and other organizations’ experience considers available features for identification [24]. Suppose, as in the cases of Table 1 [21, 33], case 1 and 2 combine the features through a specified method. Among the seven features, the F4 feature is the most similar feature by mathematical calculation. However, in actual human vision, the F4 feature of these cases is not considered a reference feature compared to the other two cases. In addition, in the similarities raised by the plaintiff in the case, not all features were raised to have similarities. Therefore, in the biological and psychological process that is not yet fully understood [27], this article quantifies and lists the characteristics of trademark design that meet the observation of human visual perception and provides quantitative data that simulate vision for design and legal decision-making reference.

Based on the visual perception theory of trademark design and Gestalt psychology, this article proposes seven features to assist judgment. Since these features are the defining elements of trademark design to attract human attention, the purpose of proposing features is to provide a basis for new trademarks or legal decisions when quantifying human visual perception through image processing, rather than making existing digital trademark search systems better. These features, which are combined with the design code search on the USPTO website, can help designers determine whether their new designs are similar to existing trademarks, and further prevent problems caused by misuse of others’ trademarks. At the same time, in the face of counterfeit trademarks of unfair competitors, it can also give quantitative and visual judgments similar to human visual perception, helping national laws to protect the reputation of the original trademark owners.


As individuals, organizations, groups, and other representative signs, trademarks are common graphics in human society. The resources and profits brought by trademarks can maintain and develop the organization and further promote national progress. However, these resources have attracted the attention of unfair competitors. They faked similar trademarks and defrauded the confused consumers of the resources that the original trademark owners should obtain. These stolen resources may prevent the organization from maintaining or developing itself, so that the country cannot progress. In order to promote national progress, most countries enact trademark laws to protect organizations that own trademarks. Even though trademarks are protected by law, there are still many infringement cases. Apart from some latecomers who caused infringements because of their unfamiliarity, there were also malicious plagiarism by unfair competitors. Therefore, most of the research on trademarks is to search for similar features from a huge trademark database to check whether the trademark constitutes plagiarism for research purposes. These studies have had significant results on similar trademark inspections and classifications.

However, in recent years, unfair competitors have begun to produce similar trademarks through psychological means. Such similar trademarks not only confuse people with the original registered trademarks, but also avoid the legal judgment of similar characteristics of the trademarks, making it difficult for law enforcement officers to judge whether they are plagiarism. Considering the influence of Gestalt psychology on trademark reputation, this article proposes a new seven-character visual judgment assistance system based on trademark design and Gestalt psychology. These features correspond to a part of the process that is not fully understood in the human visual system and quantify them. In the experiment, the trademark information used by the valve of this article to build the assistance system is Graphis public trademark database and USPTO database. During the verification test, we use the data in the USPTO database that is independent of the built-in valve value and the trademark images of past actual cases for testing. In the test results, the analysis of past cases shows that the system proposed in this article to assist in the judgment of visual perception is consistent to a certain extent with the reasons for the plaintiff’s litigation. This shows that the assistance system proposed in this article can effectively provide visual-based trademark judgment results. Therefore, the system proposed in this article can assist the law to prevent malicious plagiarism on images by unfair competitors, and reduce the plagiarism caused by the similar design concepts of late trademark designers.

Availability of data and materials

The trademark database is available online, the URL is as follows:



Region of interest. ROI are samples within a data set identified for a particular purpose


Abbreviation of trademark


Trademark database of the US Patent and Trademark Office. The database website of the registered trademark of the storage manufacturer in the USA


Trademark database of the Japan Platform for Patent. The database website of the registered trademark of the storage manufacturer in Japan


  1. 1

    B. A. Garner, Black’s Law Dictionary, 11th ed. (West Group, Saint Paul, 2019).

    Google Scholar 

  2. 2

    Taiwan Intellectual Property Office, Review Criteria: Risk of Confusion and Misidentification (Taiwan Intellectual Property Office, 2017). . Accessed 19 Nov 2020.

  3. 3

    G. Richardson, Brand names before the industrial revolution. Working Paper 13930, National Bureau of Economic Research (2008).

  4. 4

    R. Callmann, The Law of Unfair Competition, Trademarks and Monopolies, 4th ed (Callaghan, New Haven, 1998).

    Google Scholar 

  5. 5

    S. Zhu, Y. Jin, Empirical analysis of reverse confusion in trademark infringement. J. Nanjing Univ. Posts Telecommun., 5–39 (2019).

  6. 6

    T. D. Nguyen, H. H. H. Nguyen, T. H. Le, in The 2013 RIVF International Conference on Computing Communication Technologies - Research, Innovation, and Vision for Future (RIVF). Trademark image retrieval based on scale, rotation, translation invariant features, (2013), pp. 282–285.

  7. 7

    D. Agrawal, A. Jalal, R. Tripathi, in Proceedings of the 2013 International Conference on Information Systems and Computer Networks, ISCON 2013. Trademark image retrieval by integrating shape with texture feature, (2013), pp. 30–33.

  8. 8

    K. D. Joshi, S. N. Bhavsar, R. C. Sanghvi, Image retrieval system using intuitive descriptors. Procedia Technol.14:, 535–542 (2014).

    Article  Google Scholar 

  9. 9

    M. Meenalochini, K. Saranya, G. V. Rajkumar, A. Mahto, in 2018 3rd International Conference on Communication and Electronics Systems (ICCES). Perceptual hashing for content based image retrieval, (2018), pp. 235–238.

  10. 10

    G. Showkatramani, N. Khatri, A. Landicho, D. Layog, in 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA). Deep learning approach to trademark international class identification, (2019), pp. 608–612.

  11. 11

    F. M. Anuar, R. Setchi, Y. -K. Lai, A conceptual model of trademark retrieval based on conceptual similarity. Procedia Comput. Sci.22:, 450–459 (2013).

    Article  Google Scholar 

  12. 12

    F. M. Anuar, R. Setchi, Y. Lai, Semantic retrieval of trademarks based on conceptual similarity. IEEE Trans. Syst. Man Biol. Cybern. Syst.46(2), 220–233 (2016).

    Article  Google Scholar 

  13. 13

    C. Yan, B. Gong, Y. Wei, Y. Gao, Deep multi-view enhancement hashing for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell.43(4), 1445–1451 (2021).

    Article  Google Scholar 

  14. 14

    C. Yan, B. Shao, H. Zhao, R. Ning, Y. Zhang, F. Xu, 3d room layout estimation from a single rgb image. IEEE Trans. Multimed.22(11), 3014–3024 (2020).

    Article  Google Scholar 

  15. 15

    C. Yan, Z. Li, Y. Zhang, Y. Liu, X. Ji, Y. Zhang, Depth image denoising using nuclear norm and learning graph model. ACM Trans. Multimedia Comput. Commun. Appl.16(4) (2020).

  16. 16

    C. Yan, Y. Hao, L. Li, J. Yin, A. Liu, Z. Mao, Z. Chen, X. Gao, Task-adaptive attention for image captioning. IEEE Trans. Circ. Syst. Video Technol., 1–1 (2021).

  17. 17

    H. Habibollahi Najaf Abadi, M. Pecht, Artificial intelligence trends based on the patents granted by the united states patent and trademark office. IEEE Access. 8:, 81633–81643 (2020).

    Article  Google Scholar 

  18. 18

    C. Trappey, A. Trappey, S. Lin, Intelligent trademark similarity analysis of image, spelling, and phonetic features using machine learning methodologies. Adv. Eng. Inform.45: (2020).

  19. 19

    Y. Wu, Cheated! This shop is not the FamilyMart, customers almost ran in to buy drinks. written by Traditional Chinese (2016). Accessed 17 Nov 2020.

  20. 20

    Q. J. Chen, Y. X. Xu, Appearance similarity is as high as 90%, pet shop involved plagiarism PX Mart. written by Traditional Chinese (2019). Accessed 18 Nov 2020.

  21. 21

    T. Kim, Starbucks, A fierce battle to defend the logo (2019). Accessed 19 Jan 2020.

  22. 22

    S. Bradley, Design principles: visual weight And direction (2014). Accessed 07 Jan 2021.

  23. 23

    S. Bradley, Design Fundamentals - Elements, Attributes, & Principles (Vanseo Design, Boulder, 2018).

    Google Scholar 

  24. 24

    K. Koffka, Principles of Gestalt Psychology (Mimesis International, Sesto San Giovanni, 2014).

    Google Scholar 

  25. 25

    D. Bradley, G. Roth, Adaptive thresholding using the integral image. J. Graph. Tools. 12(2), 13–21 (2007).

    Article  Google Scholar 

  26. 26

    D. Eagleman, Visual illusions and neurobiology. Nat Rev Neurosci. 2:, 920–926 (2001).

    Article  Google Scholar 

  27. 27

    R. C. Gonzalez, R. E. Wood, Digital Image Processing, 4th Ed. (Pearson Education, London, 2009).

    Google Scholar 

  28. 28

    B. M. Pedersen, Logo database - Graphis. Graphis Inc. (1986). Accessed 21 Dec 2019.

  29. 29

    Japan Platform for Patent Information | J-PlatPat (JPP). Accessed 24 Jan 2021.

  30. 30

    USPTO United States Patent and Trademark Office. Accessed 23 Jan 2021.

  31. 31

    7-Eleven wins trademark infringement suit vs. Super-7 store. Convenience Store News (2015). Accessed 21 Dec 2020.

  32. 32

    Z. W. Li, Learned the lesson of being defeated to “Hodilao” in the lawsuit, Haidilao registered 200 trademarks in 2 days. written by Traditional Chinese (2020). Accessed 20 Jan 2021.

  33. 33

    Y. Li, G. Peng, Trademark battle! Starbucks defeated in lawsuit against e-coffee. TVBS News. written by Traditional Chinese (2007). Accessed 15 Jan 2021.

Download references


The authors would like to thank several anonymous reviewers and the editor for their comments and suggestions.


This research received no external funding.

Author information




Conceptualization, KMH. Methodology, KMH. Investigation, KMH, LMC, and TWC. Resources, LMC and TWC. Writing—original draft, KMH, LMC, and TWC. Writing—review and editing, KMH. Funding acquisition, KMH. Supervision, KMH. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Kuo-Ming Hung.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hung, KM., Chen, LM. & Chen, TW. Trademark infringement recognition assistance system based on human visual Gestalt psychology and trademark design. J Image Video Proc. 2021, 27 (2021).

Download citation


  • Trademark
  • Gestalt psychology
  • Trademark infringement