Open Access

Automatic detection method of bladder tumor cells based on color and shape features

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin 130033, P. R. China

University of Chinese Academy of Sciences, Beijing 100049, P. R. China

Search for more papers by this author

Yanbo Wang

Bethune First Hospital of Jilin University: The First Hospital of Jilin University, Changchun, Jilin 130061, P. R. China

E-mail Address: doctorwyb@126.com

Search for more papers by this author

Jiaqi Chen

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin 130033, P. R. China

University of Chinese Academy of Sciences, Beijing 100049, P. R. China

Search for more papers by this author

Mingjia Wang

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin 130033, P. R. China

Search for more papers by this author

Shulong Feng

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin 130033, P. R. China

Search for more papers by this author

Jin Yang

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin 130033, P. R. China

Search for more papers by this author

Nan Song

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin 130033, P. R. China

Search for more papers by this author

Jinyu Wang

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin 130033, P. R. China

University of Chinese Academy of Sciences, Beijing 100049, P. R. China

Search for more papers by this author

, and

Ci Sun

https://orcid.org/0009-0000-6482-2618

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin 130033, P. R. China

E-mail Address: 840714201@qq.com

Search for more papers by this author

https://doi.org/10.1142/S1793545824500056Cited by:0 (Source: Crossref)

Abstract

Bladder urothelial carcinoma is the most common malignant tumor disease in urinary system, and its incidence rate ranks ninth in the world. In recent years, the continuous development of hyperspectral imaging technology has provided a new tool for the auxiliary diagnosis of bladder cancer. In this study, based on microscopic hyperspectral data, an automatic detection algorithm of bladder tumor cells combining color features and shape features is proposed. Support vector machine (SVM) is used to build classification models and compare the classification performance of spectral feature, spectral and shape fusion feature, and the fusion feature proposed in this paper on the same classifier. The results show that the sensitivity, specificity, and accuracy of our classification algorithm based on shape and color fusion features are 0.952, 0.897, and 0.920, respectively, which are better than the classification algorithm only using spectral features. Therefore, this study can effectively extract the cell features of bladder urothelial carcinoma smear, thus achieving automatic, real-time, and noninvasive detection of bladder tumor cells, and then helping doctors improve the efficiency of pathological diagnosis of bladder urothelial cancer, and providing a reliable basis for doctors to choose treatment plans and judge the prognosis of the disease.

Keywords:

1. Introduction

Bladder cancer is a kind of malignant tumor that occurs in the bladder mucosa. 75–80% of bladder cancer is nonmuscle invasive bladder cancer, which is characterized by a high incidence and recurrence rate.^1,2 The diagnosis of bladder cancer mainly depends on noninvasive imaging coarse screening techniques such as color Doppler ultrasound, magnetic resonance imaging (MRI), computed tomography (CT),^3,4,5 and invasive precise examination techniques of endoscope,⁶ among which cystoscopy combined with tissue biopsy is the most reliable method to diagnose bladder cancer.⁷ At present, in clinical practice, most cystoscopes are rigid cystoscopes, and the examination process of patients is extremely painful; Moreover, in order to prevent recurrence, it is necessary to undergo endoscopic examinations several times a year, which seriously affects the quality of life of patients.^8,9 Traditional histopathological diagnosis technology has some problems, such as complicated operation, and it is time-consuming. Missed detection and false detection often occur in early screening, which makes it difficult to meet the requirements of early diagnosis and prognosis treatment of tumors.¹⁰ Due to there are some differences between tumor cells and normal cells in protein, nucleic acid, water content, and structure,¹¹ they are not only different in morphology but also different in the spectral curves. Compared with traditional detection methods, microscopic hyperspectral imaging technology can obtain rich spectral information and spatial information of samples at the same time, without complicated pre-experimental processing operations, saving manpower, and material resources. It has the advantages of fast detection speed and good real-time performance, providing a powerful tool for the detection of bladder tumor cells. The detection technology of bladder tumor cells based on microscopic hyperspectral imaging can assist medical diagnosis, thereby achieving early identification and early warning of tumor cells, avoiding missed detection of tumor cells caused by human factors, and reducing the probability of tumor cell deterioration.

With the popularization of computer analysis technology, image analysis combined with machine learning technology has been widely used to solve medical diagnosis problems, among which obtaining the color features of medical images for classification and recognition is one of the most effective methods. Color feature is a global feature, which describes the color properties of the target in the image, has good stability and is insensitive to the scaling and rotation of the image. The common methods for describing color features include color histograms, color moments, color sets, color aggregation vectors, and color correlation graphs. Color moment is a method proposed by Stricker and Orengo to describe the color distribution of images, and it has been experimentally proven that using color moments to describe the color features of HSV images for image retrieval is more effective than using color histograms.¹² Ghosh et al. achieved the early diagnosis and detection of glaucoma based on the color moments of RGB images, and classified normal and abnormal retinas at the same time. The accuracy of the experimental results was 87.47%.¹³ Aiming at the retrieval problem of medical images, Chen et al. achieved the retrieval of gastroscopic images based on color moments of the neighborhood of images.¹⁴ Ullah proposed a method for classification of brain MRI, which reduces the features of $32 \times 32 \times 3$ $32 \times 32 \times 3$ RGB images based on color moments. The remaining 9 color features after reduction were combined with K-NN model for classification, and the overall accuracy rate was 94.9745%.¹⁵ From the above studies, it can be found that the studies based on color moments mostly stay in low-dimensional space such as HSV color space and RGB color space, and lack the study on color moments in multi-dimensional spectral space.

In recent years, spectral technology has been widely developed in the field of biomedical science, which provides rapid, nondestructive and unmarked detection for genes, molecules, proteins, tissues, and so on.¹⁶ At the same time, the combination of computer image recognition technology and spectral technology also provides new methods for the detection and research of pathological images. Researchers use the comprehensive information obtained from spectral data to carry out qualitative and quantitative detection of molecular composition in substances, spectral diagnosis of related diseases, and comprehensive detection of prognosis. Wang and Li proposed a quantitative analysis method of liver tumors at different stages based on microscopic hyperspectral imaging technology. The morphological watershed algorithm was used to segment the grayscale images of liver pathology slices at different stages into multiple regions. Then, the support vector machine (SVM) was used for tumor identification, and the percentage of tumor area in liver sample slices was effectively calculated.¹⁷ Sun et al. constructed an automatic diagnosis model based on microscopic hyperspectral cholangiocarcinoma images by using a deep convolution neural network, which proved the advantages of hyperspectral images compared with traditional RGB images. They also proposed a spectral interval convolution and normalization method to further explore spectral information.¹⁸ Ortega et al. used a convolutional neural network model to automatically distinguish tumor tissues from hyperspectral images of glioma tissue slices. The average sensitivity and specificity were 88% and 77%, respectively, which were 7% and 8% higher than the results obtained using RGB images.¹⁹ In addition, some scholars have studied the detection of bladder tumor cells by using different spectral techniques. Kujdowicz et al. proposed to use Fourier transform infrared and Raman spectroscopy imaging as additional diagnostic tools to identify the chemical properties of bladder cancer, and studied the molecular differences between the nucleus and cytoplasm of bladder cancer at different stages and invasiveness.²⁰ Angeletti et al. classified multi-spectral images of urothelial cell images based on spectral space analysis using genetic algorithms, and the results showed that the average sensitivity and specificity were 85% and 95%, respectively.²¹ The above research confirms the advantages of spectral technology in the detection of tumor cells, and spatial information complementary to spectral information is also an important source of information. Some successful studies have shown that the combination of spectral features and spatial features provides the possibility of improving the performance of cell classification model.^22,23,24 Therefore, only extracting the spectral features of cells or only extracting the two-dimensional spatial features has certain limitations for tumor identification. Analyzing the spectral and spatial features of cells together has certain significance for the identification of bladder tumors.

Feature extraction is of great significance to the classification of tumor cells. Therefore, this study proposed a fusion feature method for hyperspectral images of urine smear samples, aiming to discuss the feasibility of bladder tumor cells identification. The main tasks are as follows: (1) Pre-identify the connected regions of cells to be analyzed in hyperspectral images, so as to ensure that tumor cells are extracted while minimizing the extraction of nontumor cells. (2) Combining recursive feature elimination (RFE) with SVM classifier. Then, the accuracy of the classification model with different K values of spectral feature numbers was discussed, and a reasonable K value was selected to achieve spectral dimensionality reduction. (3) Calculate the color features of each spectral channel after dimensionality reduction and fuse them with the shape features of cells. We obtained the fusion feature of ( $3 K + 1$ $3 K + 1$ )-dimension, and combined it with the SVM to construct a cell classification model. (4) Compare the performance of spectral feature (K-dimension), spectral and shape fusion feature (( $K + 1$ $K + 1$ )-dimension), and the fusion feature (( $3 K + 1$ $3 K + 1$ )-dimension) proposed in this paper on the SVM classification model. Through the above work, bladder tumor cells can be automatically identified and extracted, and the interference of nontumor cells such as normal cells, inflammatory cells and impurities in the samples can be effectively eliminated, which can provide valuable auxiliary diagnosis references for pathological laboratory doctors.

2. Materials and Methods

2.1. Data collection

Urine samples of patients suffering from bladder cancer were collected by the urology department of the first Bethune Hospital of Jilin University, approved by the ethics committee of the hospital, and informed consents were signed by each patient. Doctors prepared stained smears of samples, and manually marked representative areas in urine smears under electron microscope observation, which included three kinds of targets: tumor cells, normal cells, and other components (inflammatory cells, red blood cells, impurities, etc.). The types of cells marked by doctors were used as the standard values for model construction. The microscopic hyperspectral imaging system independently developed by our research group (CIOMP) was adopted in the experiment. As shown in Fig. 1, the system is composed of an Olympus microscope, a prism–grating (PG) light splitting module, a high-resolution CCD camera, a push-and-sweep mechanism, a control computer, a continuous spectrum halogen tungsten light source, etc. Hyperspectral image data can record both spectral information and spatial information of substances at the same time. The hyperspectral data cube of urine smear samples, which is composed of images with different wavelengths, is shown in Fig. 2, where the spatial dimension ( $X \times Y$ $X \times Y$ ) represents the size of each single-band image and the spectral dimension (Z) represents the total number of bands collected. The spectral range of this system is 400–1000nm, including 270 spectral bands, and the spectral resolution is better than 2.8nm. The objective lens with 40 times magnification was selected for imaging.

Fig. 1. Microscopic hyperspectral imaging system.

During the experiment, the hyperspectral image data of the representative areas marked in urine smears were collected, and the hyperspectral images of 150 urine smears were obtained. Each image showed three types of targets: tumor cells, normal cells and other components. The RGB images of the three types of targets are shown in Fig. 3(a), and the left side shows the initial RGB image of the urine smear sample. The outermost black part of the image is the background outside the field of view of the instrument objective, and the red circle next to the outer black is the manual mark of the doctor. The partial image of the initial image of this sample is on the right side of Fig. 3(a). At the same time, the average spectral curve and standard deviation of pixels in each target connected region are listed as shown in Fig. 3(b), where the blue curve represents impurity, pink represents tumor cells, green represents normal cells, and yellow represents inflammatory cells.

Fig. 3. Schematic diagrams of three kinds of targets and their corresponding initial spectral curves, (a) RGB schematic diagrams of urine smear samples, and (b) average spectral curves and standard deviations of tumor cells, normal cells and other components (impurities and inflammatory cells).

2.2. Pre-identification and processing

As there are not only tumor cells, but also normal urothelial cells, and other components in the hyperspectral images, the automatic detection algorithm of tumor cells needs to carry out the pre-identification step first, that is, pre-identify the connected regions (single cells or cell clusters) to be analyzed for subsequent classification. To extract the connected regions to be analyzed from the image, it is necessary to first use maximum between-cluster variance (OTSU) to threshold the RGB image generated by three wavelengths of hyperspectral image, then detect the contour and number of each connected region. The two morphological features of each connected region, the area size $τ$ $τ$ and the ratio of the shortest diameter to the longest diameter $ξ$ $ξ$ , were investigated. According to the values of $τ$ $τ$ and $ξ$ $ξ$ , the numbers of each connected region were arranged from large to small to form two ranking sets, and the first 30% numbers of the two ranking sets were reserved respectively and the intersection of the two sets was taken. The connected regions corresponding to this intersection were used as the connected regions of the cells to be analyzed for the next classification, and the shape feature $τ$ $τ$ of the connected regions of the cells to be analyzed was saved for subsequent feature fusion. In this way, inflammatory cells with relatively small areas and impurities with relatively narrow and long shapes can be effectively removed. This process needs to ensure that tumor cells are extracted while minimizing the extraction of nontumor cells, including normal cells and other components. As for a small number of nontumor cells extracted from this part, they can be distinguished from tumor cells by subsequent classification algorithms.

Generally speaking, normal urothelial cells are smaller than tumor cells and larger than inflammatory cells. However, due to inflammatory reaction, inflammatory cells in some areas are stacked, and their connected regions become larger, which leads to them being extracted as interference targets. In addition, some impurities on the slide are easily wrongly extracted due to their size and spectrum similar to the target to be analyzed. For example, Fig. 4 is an image in which the extracted cells to be analyzed are outlined in green and labeled. Seven connected regions to be analyzed are extracted from this image, in which the number 7 is a normal cell and the number 6 is a tumor cell. In addition, some other components are inevitably mixed in the extracted cells to be analyzed, for example, the numbers 2 and 4 are the connected regions of inflammatory cells, and the numbers 1, 3, and 5 are the connected regions of noncellular impurities, which need to be removed in the subsequent classification.

Fig. 4. Sample image after extracting cells to be analyzed.

Before feature extraction, it is necessary to calculate the relative transmittance of the transmission spectrum of the cells to be analyzed extracted from the image, and the relative transmittance is defined as follows :

T0=DN0−DNBDN1−DNB×100%,T0=DN0−DNBDN1−DNB×100%,<math display="block" altimg="eq-00013.gif"><msub><mrow><mi>T</mi></mrow><mrow><mn>0</mn></mrow></msub><mo>=</mo><mfrac><mrow><msub><mrow><mstyle><mtext mathvariant="normal">DN</mtext></mstyle></mrow><mrow><mn>0</mn></mrow></msub><mo>−</mo><msub><mrow><mstyle><mtext mathvariant="normal">DN</mtext></mstyle></mrow><mrow><mi>B</mi></mrow></msub></mrow><mrow><msub><mrow><mstyle><mtext mathvariant="normal">DN</mtext></mstyle></mrow><mrow><mn>1</mn></mrow></msub><mo>−</mo><msub><mrow><mstyle><mtext mathvariant="normal">DN</mtext></mstyle></mrow><mrow><mi>B</mi></mrow></msub></mrow></mfrac><mo>×</mo><mn>1</mn><mn>0</mn><mn>0</mn><mi>%</mi><mo>,</mo></math>(1)

where

T_{0}

$T_{0}$ and

{DN}_{0}

${DN}_{0}$ respectively, represent the spectral transmittance and transmission gray value of a pixel in an area to be analyzed,

{DN}_{1}

${DN}_{1}$ represents the transmission gray value of a blank background pixel of the sample, and

{DN}_{B}

${DN}_{B}$ represents the dark background gray value under the same experimental conditions. Therefore, it can solve the influence on the subsequent tumor cell identification accuracy caused by the changes of light source and environment, or the differences of samples background when obtaining hyperspectral images of different samples. Then the spectral transmittance of all pixels in the whole area of each connected region is preprocessed by smoothing and noise reduction, and the spectral transmittance of each connected region is saved for subsequent feature extraction, and the shape feature (

τ

$τ$ ) of the area of each connected region is obtained for subsequent feature fusion.

2.3. Feature extraction and classification

In this study, recursive feature elimination (RFE) is used to select spectral features. RFE is a feature selection method with good performance and strong generalization ability. By repeatedly training the selected machine learning model, it removes the least important features from the training data features set until the required number of K features remains, so as to reduce the dimension of hyperspectral data.²⁵ Among them, SVM was selected as the classification model. The SVM classification model based on RFE was proposed by Guyon et al. for cancer classification data. The basic principle of SVM classification model is to find the separation hyperplane with maximum interval in the sample space, so as to distinguish different types of samples.²⁶ In practical application, the sample spaces are mostly classified in nonlinear ways. By introducing the kernel function $K (x_{i}, x_{j})$ $K (x_{i}, x_{j})$ , the data in the low-dimensional sample space is mapped into the high-dimensional sample space through $φ (x)$ $φ (x)$ , and $K (x_{i}, x_{j}) = φ (x_{i}) \cdot φ (x_{j})$ $K (x_{i}, x_{j}) = φ (x_{i}) \cdot φ (x_{j})$ is satisfied, so that the data can be linearly separable in the high-dimensional space, and finally the optimal hyperplane can be found. Finding the optimal hyperplane problem is to solve the extreme value of the following equations :

L(λ)=−12N∑i=1N∑j=1λiλjyiyiK(xi,xj)+N∑i=1λi,L(λ)=−12N∑i=1N∑j=1λiλjyiyiK(xi,xj)+N∑i=1λi,<math display="block" altimg="eq-00022.gif"><mi>L</mi><mo stretchy="false">(</mo><mi>λ</mi><mo stretchy="false">)</mo><mo>=</mo><mo>−</mo><mfrac><mrow><mn>1</mn></mrow><mrow><mn>2</mn></mrow></mfrac><munderover accentunder="true" accent="true"><mrow><mo>∑</mo></mrow><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>N</mi></mrow></munderover><munderover accentunder="true" accent="true"><mrow><mo>∑</mo></mrow><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>N</mi></mrow></munderover><msub><mrow><mi>λ</mi></mrow><mrow><mi>i</mi></mrow></msub><msub><mrow><mi>λ</mi></mrow><mrow><mi>j</mi></mrow></msub><msub><mrow><mi>y</mi></mrow><mrow><mi>i</mi></mrow></msub><msub><mrow><mi>y</mi></mrow><mrow><mi>i</mi></mrow></msub><mi>K</mi><mo stretchy="false">(</mo><msub><mrow><mi>x</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>,</mo><msub><mrow><mi>x</mi></mrow><mrow><mi>j</mi></mrow></msub><mo stretchy="false">)</mo><mo>+</mo><munderover accentunder="true" accent="true"><mrow><mo>∑</mo></mrow><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>N</mi></mrow></munderover><msub><mrow><mi>λ</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>,</mo></math>(2)

s.t. N \sum i = 1 λ i y i = 0, (λ i \geq 0) . s.t. N \sum i = 1 λ_{i} y_{i} = 0, (λ_{i} \geq 0) . <math display="block" altimg="eq-00023.gif"><mstyle><mtext mathvariant="normal">s.t.</mtext></mstyle><mtext> </mtext><munderover accentunder="true" accent="true"><mrow><mo>\sum</mo></mrow><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>N</mi></mrow></munderover><msub><mrow><mi>λ</mi></mrow><mrow><mi>i</mi></mrow></msub><msub><mrow><mi>y</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>=</mo><mn>0</mn><mo>,</mo><mspace width="1em"></mspace><mo stretchy="false">(</mo><msub><mrow><mi>λ</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>\geq</mo><mn>0</mn><mo stretchy="false">)</mo><mo>.</mo></math> (3)

The optimal classification function of the final optimal hyperplane is

H (x) = sgn N \sum i = 1 λ i y i K (x i, x) + b, H (x) = sgn N \sum i = 1 λ_{i} y_{i} K (x_{i}, x) + b, <math display="block" altimg="eq-00024.gif"><mi>H</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo><mstyle><mtext mathvariant="normal">sgn</mtext></mstyle><munderover accentunder="true" accent="true"><mrow><mo>\sum</mo></mrow><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>N</mi></mrow></munderover><msub><mrow><mi>λ</mi></mrow><mrow><mi>i</mi></mrow></msub><msub><mrow><mi>y</mi></mrow><mrow><mi>i</mi></mrow></msub><mi>K</mi><mo stretchy="false">(</mo><msub><mrow><mi>x</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>,</mo><mi>x</mi><mo stretchy="false">)</mo><mo>+</mo><mi>b</mi><mo>,</mo></math> (4)

where x refers to the sample features, y is their label, N is the number of samples, and

λ_{i}

$λ_{i}$ is the Lagrange coefficient.

Commonly used kernel functions are linear kernel, polynomial kernel, Gaussian kernel, Laplacian kernel, and sigmoid kernel. Due to the large number of hyperspectral data features in this study, there is the possibility of over-fitting and too much calculation, so the linear kernel is used as the kernel function. As shown in Eq. (5), the specific process of SVM-RFE feature selection is shown in Fig. 5.

K (x i, x j) = 〈 x i, x j 〉 . K (x_{i}, x_{j}) = ⟨ x_{i}, x_{j} ⟩ . <math display="block" altimg="eq-00026.gif"><mi>K</mi><mo stretchy="false">(</mo><msub><mrow><mi>x</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>,</mo><msub><mrow><mi>x</mi></mrow><mrow><mi>j</mi></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mo stretchy="false">〈</mo><msub><mrow><mi>x</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>,</mo><msub><mrow><mi>x</mi></mrow><mrow><mi>j</mi></mrow></msub><mo stretchy="false">〉</mo><mo>.</mo></math> (5)

Fig. 5. Feature selection process of SVM-RFE.

Since there are three types of sample cells to be analyzed, that is, tumor cells, normal cells, and other components, the ultimate goal is to identify tumor cells from the hyperspectral images of urine smears. Therefore, tumor cells are regarded as positive; Nontumor cells, that is, normal cells and other components are regarded as negative. The problem of binary classification is carried out.

Color moments are used to calculate the color distribution of the targets. Since the color information is mainly distributed in low-order moment,¹² the first-order moment (representing the average value of all pixels on the ith color channel), the second-order moment (representing the standard deviation of all pixels on the ith color channel) and the third-order moment (representing the cubic root of the slope of all pixels on the ith channel) are enough to express the color distribution of the image. The first-order moment reflects the overall brightness of the image, the second-order moment reflects the color distribution range of the image, and the third-order moment reflects the symmetry of the color distribution of the image. Color moments are widely used in the field of image recognition, usually calculated in RGB color space. In this study, hyperspectral images are used to calculate color moments in spectral dimension.

The hyperspectral data after RFE dimensionality reduction have a total of K bands, that is, the K-dimensional spectral feature vector is obtained: $x = [x_{1}, x_{2}, \dots, x_{K}]$ $x = [x_{1}, x_{2}, \dots, x_{K}]$ . Because the cell size and nuclear–cytoplasmic ratio of tumor cells, normal cells and inflammatory cells are very different, the color distribution is different. Usually, the nuclei of tumors are larger, irregularly shaped, and deeply stained. Therefore, the color moment features are extracted after dimensionality reduction of the spectrum of the connected regions of the samples, K-dimensional spectral features are taken as the K color components of the color moment, M represents the number of pixels in each cell to be analyzed, and the formulas for calculating the first-order (mean), second-order (variance), and third-order (skewness) color moments are as follows:

μi=1MM∑j=1pi,j,μi=1MM∑j=1pi,j,<math display="block" altimg="eq-00028.gif"><msub><mrow><mi>μ</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>=</mo><mfrac><mrow><mn>1</mn></mrow><mrow><mi>M</mi></mrow></mfrac><munderover accentunder="true" accent="true"><mrow><mo>∑</mo></mrow><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>M</mi></mrow></munderover><msub><mrow><mi>p</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>j</mi></mrow></msub><mo>,</mo></math>(6)

σi=(1MM∑j=1(pi,j−μi)2)12,σi=⎛⎝1MM∑j=1(pi,j−μi)2⎞⎠12,<math display="block" altimg="eq-00029.gif"><msub><mrow><mi>σ</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>=</mo><msup><mrow><mfenced separators="" open="(" close=")"><mrow><mfrac><mrow><mn>1</mn></mrow><mrow><mi>M</mi></mrow></mfrac><munderover accentunder="true" accent="true"><mrow><mo>∑</mo></mrow><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>M</mi></mrow></munderover><msup><mrow><mo stretchy="false">(</mo><msub><mrow><mi>p</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>j</mi></mrow></msub><mo>−</mo><msub><mrow><mi>μ</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo></mrow><mrow><mn>2</mn></mrow></msup></mrow></mfenced></mrow><mrow><mfrac><mrow><mn>1</mn></mrow><mrow><mn>2</mn></mrow></mfrac></mrow></msup><mo>,</mo></math>(7)

si=(1MM∑j=1(pi,j−μi)3)13,<math display="block" altimg="eq-00030.gif"><msub><mrow><mi>s</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>=</mo><msup><mrow><mfenced separators="" open="(" close=")"><mrow><mfrac><mrow><mn>1</mn></mrow><mrow><mi>M</mi></mrow></mfrac><munderover accentunder="true" accent="true"><mrow><mo>∑</mo></mrow><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>M</mi></mrow></munderover><msup><mrow><mo stretchy="false">(</mo><msub><mrow><mi>p</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>j</mi></mrow></msub><mo>−</mo><msub><mrow><mi>μ</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo></mrow><mrow><mn>3</mn></mrow></msup></mrow></mfenced></mrow><mrow><mfrac><mrow><mn>1</mn></mrow><mrow><mn>3</mn></mrow></mfrac></mrow></msup><mo>,</mo></math>(8)

where

$p_{i, j}$ represents the pixel value (

$i = 1, 2, \dots, K$ ) of the jth pixel in the ith color component.

After spectral analysis of tumor cells, normal cells, and impurities in the urine smear image in Fig. 3, their color moments respectively are calculated. As can be seen from the RGB image in Fig. 6(a), both impurities and tumor cells show dark black staining at different degrees, while normal cells have smaller nuclei and lighter cytoplasm staining. In Fig. 6(b), the spectral first-order moments of impurities and tumor cells are higher than those of normal cells, and they are similar in shape and intensity to some extent, and some spectral regions almost overlap, so it is impossible to classify them by using only the first-order moments. On this basis, the second-order moments and third-order moments (Figs. 6(c) and 6(d)) are added as features to represent the color distribution of the three types of targets, which can effectively distinguish them.

Fig. 6. The color moment of hyperspectral image of urine smear, (a) schematic diagrams of three kinds of targets, (b) first-order moment, (c) second-order moment, and (d) third-order moment.

According to the above equation, a 3K-dimensional color feature vector $F_{color}$ can be obtained :

F color = [μ 1, σ 1, s 1, \dots, μ K, σ K, s K] . <math display="block" altimg="eq-00034.gif"><msub><mrow><mi>F</mi></mrow><mrow><mstyle><mtext mathvariant="normal">color</mtext></mstyle></mrow></msub><mo>=</mo><mo stretchy="false">[</mo><msub><mrow><mi>μ</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>,</mo><msub><mrow><mi>σ</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>,</mo><msub><mrow><mi>s</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>,</mo><mo>\dots</mo><mo>,</mo><msub><mrow><mi>μ</mi></mrow><mrow><mi>K</mi></mrow></msub><mo>,</mo><msub><mrow><mi>σ</mi></mrow><mrow><mi>K</mi></mrow></msub><mo>,</mo><msub><mrow><mi>s</mi></mrow><mrow><mi>K</mi></mrow></msub><mo stretchy="false">]</mo><mo>.</mo></math> (9)

Finally, the final ( $3 K + 1$ )-dimensional shape feature and color feature fusion vector X was obtained :

X = [τ, μ 1, σ 1, s 1, \dots, μ K, σ K, s K] . <math display="block" altimg="eq-00036.gif"><mi>X</mi><mo>=</mo><mo stretchy="false">[</mo><mi>τ</mi><mo>,</mo><msub><mrow><mi>μ</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>,</mo><msub><mrow><mi>σ</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>,</mo><msub><mrow><mi>s</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>,</mo><mo>\dots</mo><mo>,</mo><msub><mrow><mi>μ</mi></mrow><mrow><mi>K</mi></mrow></msub><mo>,</mo><msub><mrow><mi>σ</mi></mrow><mrow><mi>K</mi></mrow></msub><mo>,</mo><msub><mrow><mi>s</mi></mrow><mrow><mi>K</mi></mrow></msub><mo stretchy="false">]</mo><mo>.</mo></math> (10)

Input the obtained fusion feature vector X into the SVM classifier, and obtain the optimal classification function $H (x)$ of the optimal hyperplane in Eq. (4), thus obtaining the final SVM classification result. The framework of the proposed automatic detection algorithm for bladder tumor cells is shown in Fig. 7.

Fig. 7. Automatic detection algorithm framework of bladder tumor cells.

3. Results

Sensitivity, specificity, and accuracy were used to evaluate the performance of the tumor cell detection model, and the equations are as follows:

sensitivity=TPTP+FN,<math display="block" altimg="eq-00038.gif"><mstyle><mtext mathvariant="normal">sensitivity</mtext></mstyle><mo>=</mo><mfrac><mrow><mstyle><mtext mathvariant="normal">TP</mtext></mstyle></mrow><mrow><mstyle><mtext mathvariant="normal">TP</mtext></mstyle><mo>+</mo><mstyle><mtext mathvariant="normal">FN</mtext></mstyle></mrow></mfrac><mo>,</mo></math>(11)

specificity=TNTN+FP,<math display="block" altimg="eq-00039.gif"><mstyle><mtext mathvariant="normal">specificity</mtext></mstyle><mo>=</mo><mfrac><mrow><mstyle><mtext mathvariant="normal">TN</mtext></mstyle></mrow><mrow><mstyle><mtext mathvariant="normal">TN</mtext></mstyle><mo>+</mo><mstyle><mtext mathvariant="normal">FP</mtext></mstyle></mrow></mfrac><mo>,</mo></math>(12)

accuracy=TP+TNTP+FN+TN+FP,<math display="block" altimg="eq-00040.gif"><mstyle><mtext mathvariant="normal">accuracy</mtext></mstyle><mo>=</mo><mfrac><mrow><mstyle><mtext mathvariant="normal">TP</mtext></mstyle><mo>+</mo><mstyle><mtext mathvariant="normal">TN</mtext></mstyle></mrow><mrow><mstyle><mtext mathvariant="normal">TP</mtext></mstyle><mo>+</mo><mstyle><mtext mathvariant="normal">FN</mtext></mstyle><mo>+</mo><mstyle><mtext mathvariant="normal">TN</mtext></mstyle><mo>+</mo><mstyle><mtext mathvariant="normal">FP</mtext></mstyle></mrow></mfrac><mo>,</mo></math>(13)

among them, TP stands for true positive class, indicating the number of samples correctly classified as tumor cells (positive class); TN stands for true negative class, indicating the number of samples correctly classified as nontumor cells (negative class); FP stands for false positive class, indicating the number of samples wrongly classified as tumor cells; FN stands for false negative class, indicating the number of samples wrongly classified as nontumor cells.

The counting rules of TP, TN, FP, and FN in this study were as follows: Take a smear sample as an example, when all tumor cells in this sample were correctly classified, TP was recorded as 1 and FP as 0; conversely, when the tumor cells in the smear sample were wrongly classified, TP was recorded as 0 and FP as 1. Similarly, when all nontumor cells in a smear sample were correctly classified, TN was recorded as 1 and FN as 0; on the contrary, when the nontumor cells in the smear sample were wrongly classified, TN was recorded as 0 and FN as 1.

Sensitivity is the evaluation of the true positive rate, indicating the proportion of all positive samples that are correctly classified, that is, the evaluation of the classification performance of tumor cells. Specificity is the evaluation of the true negative rate, which indicates the proportion of all negative samples that are correctly classified, that is, the evaluation of the classification performance of nontumor cells. Accuracy reflects the proportion of the number of correctly classified samples to the total. Combining sensitivity, specificity, and accuracy to evaluate the performance of the classification model can basically reflect the quality of the model.

Compared with the SVM classification results when the RFE method retains 1–20 spectral features, its sensitivity, specificity and accuracy are calculated by Eqs. (11)–(13) and drawn as shown in Fig. 8.

It can be found from Fig. 8 that with the increase of the number of features, the classification performance generally shows an upward trend. When the number of retained spectral features K reaches 13–15, the optimal classification performance is reached, and then the classification performance gradually decreases with the increase of the number of spectral features. Therefore, the value of K is 13 in this experiment.

In the experiment, the training set and test set were randomly divided each time from the hyperspectral image data of 150 urine smears, and the division ratio was 2:1. The model was repeatedly tested for 10 times. The results of TP, TN, FP, and FN of each test were counted, and the average values of TP, TN, FP, and FN were taken as the final results. The number of TP, TN, FP, FN, and the classification model evaluation under repeated tests are shown in Tables 1 and 2.

**Table 1. The statistics of test set samples classification results when $K = 13$ .**
	Correct classification number	Wrong classification number	Total number
Positive class	$TP = 48$	$FN = 2$	50
Negative class	$FP = 45$	$TN = 5$	50

**Table 2. The classification effect of test set samples when $K = 13$ .**
	Sensitivity	Specificity	Accuracy
SVM ((3 $K + 1$ )-dimensional fusion feature)	0.96	0.90	0.93

In order to verify the advantages of the proposed fusion feature method in feature selection, the SVM classification results of fusion feature ((3 $K + 1$ )-dimension) and all spectral features (270-dimension) were compared under the same samples and experimental parameters. At the same time, in order to verify the improvement of the subsequent classification performance of the fusion feature method proposed in this study, under the same samples and experimental parameters, the SVM classification results of spectral feature (K-dimension), spectral and shape fusion feature (( $K + 1$ )-dimension) and fusion feature ((3 $K + 1$ )-dimension) after dimensionality reduction are compared, and the numbers and accuracy of TP, TN, FP, and FN of different methods are summarized in Table 3.

**Table 3. Comparison of classification effects of different feature methods.**
Method	TP	TN	FP	FN	Sensitivity	Specificity	Accuracy
SVM (270-dimensional spectral feature)	39	42	8	11	0.78	0.84	0.81
SVM (K-dimensional spectral feature)	43	45	5	7	0.86	0.90	0.88
SVM (( $K + 1$ )-dimensional spectral and shape feature)	43	47	3	7	0.86	0.94	0.90
SVM ((3 $K + 1$ )-dimensional fusion feature)	48	45	5	2	0.96	0.90	0.93

4. Discussion

By comparing the results in Table 3, it can be found that the model constructed by using all spectral features has the worst classification accuracy, and its sensitivity, specificity and accuracy are 0.78, 0.84, and 0.81, respectively, which are much lower than other methods. Because there are some redundant bands and irrelevant information in 270 spectral bands, direct use in modeling will lead to problems of excessive calculation and precision reduction, thus affecting the performance of the model. Therefore, the tumor cell detection results of the other three feature selection methods in addition to all spectral features were discussed, as shown in Fig. 9, where tumor cell pixels were covered in pink and nontumor cell pixels were covered in green.

From the results of Table 3 and Fig. 9, it can be found that the classification effect of the method using only spectral features is the worst among the three methods, and its sensitivity, specificity and accuracy are 0.86, 0.9, and 0.88, respectively. Although spectral and shape features method has the strongest ability to classify negative class (nontumor cells), there are 47 negative classes correctly in 50 smear test samples, that is, the number of TN is 47, so the specificity value is the highest, which is 0.94; However, the classification ability of tumor cells is weak, and only 43 positive samples in 50 smears are correctly classified, that is, the number of TP is 43. The fusion feature method proposed in this study has the strongest classification ability for tumor cells, with the TP number of 48, and the sensitivity and accuracy of classification are 0.96 and 0.93, respectively, which are better than the classification accuracy of spectral feature, spectral and shape features methods, but the classification accuracy of nontumor cells is slightly worse than that of spectral and shape features methods, with its specificity of 0.9.

In addition, the fusion feature algorithm shows certain advantages in classifying impurity targets of nontumor cells on urine smears. As shown in (a) 5, (b) 5, (c) 5 and (d) 5 of Fig. 9, both spectral feature and spectral and shape feature classify impurity targets incorrectly, and only the fusion feature classifies them correctly.

Since the RFE dimensionality reduction algorithm first guarantees the importance and effectiveness of the remaining K-dimensional features, the proposed fusion feature method deeply mines the color features in each dimension on the basis of the original spectral features after dimensionality reduction and fuses the shape features of cells, so that the original K-dimensional features are enriched into (3 $K + 1$ )-dimensional features. Compared with the other two methods, the fusion feature method can deeply mine the image distribution features of nucleus and cytoplasm in the connected regions of the target to be analyzed, and fully extract and utilize the spatial information and hyperspectral color information of cells, with rich features and no redundancy.

5. Conclusions

Bladder urothelial tumor is one of the common malignant tumors, and its forming factors are complicated. The postoperative recurrence rate and metastasis rate are high. In order to solve the problems of low efficiency and high false positive rate in the existing pathological detection methods, this study is based on microscopic hyperspectral imaging technology and fuses the shape and color features of bladder tumor cells to achieve automatic detection of bladder tumor cells. This paper first extracts the target to be classified by OTSU threshold segmentation to remove the interference of nontumor cells on urine smears. The dimension of the targets to be classified is reduced by RFE spectral feature extraction, and the color features of each spectral band are calculated and fused with the shape features; we discuss the classification accuracy of spectral feature, spectral and shape feature, and the fusion feature proposed in this paper under the same classifier. In contrast, the fusion feature method proposed in this paper has the strongest feature extraction ability for tumor cells, which can bring more features to be input into the classification model. The sensitivity and accuracy of its classification are 0.96 and 0.93, respectively, which are better than the classification accuracy of spectral feature methods, spectral and shape feature methods. This study can provide new analysis ideas for cytology and histology auxiliary screening, improve the work efficiency of doctors, and has great application potential in clinical medical treatment and patient prognosis diagnosis.

In addition, in future research work, we will continue to enrich the sample size of urine smears of bladder cancer, and investigate the prognosis of bladder cancer patients with different ages, genders, smoking, and living habits, so as to establish a perfect system for bladder cancer analysis and prognosis.

Funding

Bethune Medical Engineering and Instrument Center Fund (E10133Y8H0) Jilin province science and technology development plan project (20210204216YY, 20210204146YY).

Acknowledgments

We thank the doctors in the Pathology Department of the first Bethune Hospital of Jilin University for their help.

Conflicts of Interest

The authors declare that there are no conflicts of interest relevant to this article.

ORCID

Zitong Zhao https://orcid.org/0009-0007-7040-1554

Ci Sun https://orcid.org/0009-0000-6482-2618

References

1. Y. Lotan and C. G. Roehrborn , “Cost-effectiveness of a modified care protocol substituting bladder tumor markers for cystoscopy for the followup of patients with transitional cell carcinoma of the bladder: A decision analytical approach,” J. Urol. 167(1), 75–79 (2002). Crossref, Web of Science, Google Scholar
2. A. Rodríguez-Alonso et al., “Multivariate analysis of survival, recurrence, progression and development of mestastasis in T1 and T2a transitional cell bladder carcinoma,” Cancer 94(6), 1677–1684 (2002). Crossref, Web of Science, Google Scholar
3. M. J. Dibb et al., “Ultrasonographic analysis of bladder tumors,” Clin. Imaging 25(6), 416–420 (2001). Crossref, Web of Science, Google Scholar
4. J. Chin et al., “Magnetic resonance imaging for detecting and treatment monitoring of orthotopic murine bladder tumor implants,” J. Urol. 145(6), 1297–1301 (1991). Crossref, Web of Science, Google Scholar
5. S. Jaume et al., “Tumor detection in the bladder wall with a measurement of abnormal thickness in CT scans,” IEEE Trans. Biomed. Eng. 50(3), 383–390 (2003). Crossref, Web of Science, Google Scholar
6. A. Collado et al., “Early complications of endoscopic treatment for superficial bladder tumors,” J. Urol. 164(5), 1529–1532 (2000). Crossref, Web of Science, Google Scholar
7. M. Sun and T. Quoc-Dien , “Diagnosis and staging of bladder cancer,” Hematol./Oncol. Clin. 29(2), 205–218 (2015). Crossref, Web of Science, Google Scholar
8. S. E. Wason et al., Ureteroscopy, StatPearls [Internet], StatPearls Publishing (2023). Google Scholar
9. W. Krajewski et al., “Patient comfort during flexible and rigid cystourethroscopy,” Videosurgery Other Miniinvasive Tech. 11(2), 94–97 (2016). Crossref, Web of Science, Google Scholar
10. Z. Zhang et al., “Application of electrochemical biosensors in tumor cell detection,” Thorac. Cancer 11(4), 840–850 (2020). Crossref, Web of Science, Google Scholar
11. N. M. Anderson and M. Celeste Simon . “The tumor microenvironment,” Curr. Biol. 30(16), R921–R925 (2020). Crossref, Web of Science, Google Scholar
12. M. A. Stricker and M. Orengo , Similarity of color images, Storage and Retrieval for Image and Video Databases III, Vol. 2420, SPiE (1995). Crossref, Google Scholar
13. A. Ghosh et al., “Grid color moment features in glaucoma classification,” Int. J. Adv. Comput. Sci. Appl. 6(9), 99–107 (2015). Google Scholar
14. Q. Chen, X. Tai, Y. Dong, S. Pan, X. Wang and C. Yin , “Medical image retrieval based on semantic of neighborhood color moment histogram,” 2008 2nd Int. Conf. on Bioinformatics and Biomedical Engineering, Shanghai, China, pp. 2221–2224 (2008). https://doi.org/10.1109/ICBBE.2008.886 Crossref, Google Scholar
15. Z. Ullah, “Features reductions using color moments and classification of brain MRI using K-NN,” (2019). Google Scholar
16. G. Lu and F. Baowei , “Medical hyperspectral imaging: A review,” J. Biomed. Opt. 19(1), 010901 (2014). Crossref, Web of Science, Google Scholar
17. J. Wang and Q. Li , “Quantitative analysis of liver tumors at different stages using microscopic hyperspectral imaging technology,” J. Biomed. Opt. 23(10), 106002 (2018). Crossref, Web of Science, Google Scholar
18. L. Sun et al., “Diagnosis of cholangiocarcinoma from microscopic hyperspectral pathological dataset by deep convolution neural networks,” Methods 202, 22–30 (2022). Crossref, Web of Science, Google Scholar
19. S. Ortega et al., “Hyperspectral imaging for the detection of glioblastoma tumor cells in H&E slides using convolutional neural networks,” Sensors 20(7), 1911 (2020). Crossref, Web of Science, Google Scholar
20. M. Kujdowicz et al., “In vitro spectroscopy-based profiling of urothelial carcinoma: A Fourier transform infrared and Raman imaging study,” Cancers 13(1), 123 (2021). Crossref, Web of Science, Google Scholar
21. C. Angeletti et al., “Detection of malignancy in cytology specimens using spectral–spatial analysis,” Lab. Invest. 85(12), 1555–1564 (2005). Crossref, Web of Science, Google Scholar
22. Y. Duan et al., “Leukocyte classification based on spatial and spectral features of microscopic hyperspectral images,” Opt. Laser Technol. 112, 530–538 (2019). Crossref, Web of Science, Google Scholar
23. Q. Ran et al., Spatial-spectral blood cell classification with microscopic hyperspectral imagery, AOPC 2017: Optical Spectroscopy and Imaging, Vol. 10461, SPIE (2017). Crossref, Google Scholar
24. X. Hu et al., “Spatial-spectral identification of abnormal leukocytes based on microscopic hyperspectral imaging technology,” J. Innov. Opt. Health Sci. 13(2), 2050005 (2020). Link, Web of Science, Google Scholar
25. I. Guyon, J. Weston, S. Barnhill and V. Vapnik , “Gene selection for cancer classification using support vector machines,” Mac. Learn. 46(1–3), 389–422 (2002). Crossref, Web of Science, Google Scholar
26. M. A. Hearst et al., “Support vector machines,” IEEE Intell. Syst. Appl. 13(4), 18–28 (1998). Crossref, Web of Science, Google Scholar

Vol. 17, No. 06

Metrics

Downloaded 294 times

History

Received 6 November 2023

Accepted 27 February 2024

Published: 4 May 2024

Information

This is an Open Access article. It is distributed under the terms of the Creative Commons Attribution 4.0 (CC-BY) License. Further distribution of this work is permitted, provided the original work is properly cited.

Keywords

PDF download