Title: RSNA 2024 Lumbar Spine Degenerative Classification
Chinese: Classification of degenerative lumbar spine lesions
Link to the official kaggle website contest questions:/competitions/rsna-2024-lumbar-spine-degenerative-classification/overview
Organization of the article
①, how to read dcm/dicom files with python
②, based on matplotlib visualization
(iii) Plotting frequency distribution histograms
④. Summary of codes
Document dependencies
#
# Python version 3.11.8
torch==2.3.1
torchvision==0.18.1
matplotlib==3.8.4
pydicom==2.4.4
numpy==1.26.4
pip install -r
Read dicom images and do preprocessing
summarize
In this paper it is takenpydicomThe package reads the dicom file with the key code format:
dcm_tensor = (dicom_file)
Note the path of the dataset, which holds the data for each patient under the train_images file, and for each patient contains three MRI images, each MRI image is stored as a folder.
It should be noted that MRI images are three-dimensional images (dicom format), and it is generally customary to save each of their slices as a separate dcm file, so a dicom image will be saved as a folder [as follows].
We can access the dicom file using the following path:
"./train_images/4003253/702807833"
Read Path
In order to read the dicom image, we need to write code to read all the dcm files in the folder
# dicom file path
dicom_dir = ". /train_images/4003253/702807833"
# Path where all dcm files are kept
dicom_files = [(dicom_dir, f) for f in (dicom_dir) if ('.dcm')]
-
: Returns all files under the dicom_dir path -
('.dcm'): Filter all files in dcm format -
: add dcm filename after dicom_dir
Schema: ". /hello "+""->". /hello/"
route sorting
The dataset given for this kaggle tournament has an iterative approach to file names:
、、...、、、、...
This gives us some trouble, because in os's filename sorting rules, the ASCII size of the higher letter is retrieved first to do the sorting, which means that it will be considered as the previous file.
In this regard, this paper uses regular expressions to realize the sorting based on the size of the numbers in the filename.
def extract_number(filepath).
# Get the name of the file (including the extension)
filename = (filepath)
# Extract the number part of the filename, assuming the filename ends in a number, e.g. ''
match = (r'(\d+)\.dcm$', filename)
return int((1)) if match else float('inf')
# Sort based on numeric handles
dicom_files.sort(key=extract_number)
The effect of this code is as follows:
Read image
In order to read the dicom image, we need to read each dcm file in turn and finally package it into a 3D tensor, which is implemented in the following code:
# Create an empty list to save all dcm files
dcm_list= []
# Iterate over each file
for dicom_file in dicom_files.
# Read the file
dcm = (dicom_file)
# Convert it to numpy format
image_data = dcm.pixel_array.astype(np.float32)
# Add the list of files
dcm_list.append(image_data)
# Stack the images into a 3D tensor
tensor_dcm = ([(image_data) for image_data in dcm_list])
Data preprocessing
There are two common types of pretreatment, theNormalizationcap (a poem)Quantization
-
normalize: The process of scaling data to some standard range. Common normalization methods include Min-Max Normalization, which normalizes the data to the range [0,1], and Z-score Normalization, which transforms the data into a standard normal distribution. The Min-Max scheme is used in this example.
-
quantize: Quantization is the process of degenerating the value domain of data to discrete values. Commonly used to reduce storage and computational costs, especially in neural network models. Quantization usually converts floating point values to integer values. Quantization is usually preceded by normalization.
The normalization is implemented as follows:
def norm_tensor(tensor_dciom).
# Find the maximum and minimum values of an image
vmin, vmax = tensor_dciom.min(), tensor_dciom.max()
# Normalize
tensor_dciom = (tensor_dciom - vmax ) / (max_val - vmin)
return tensor_dciom
Realization ofmethodThe handle selects the preprocessing method:
if method == "norm".
# normalize
tensor_dcm = norm_tensor(tensor_dcm)
elif method == "uint8".
# normalize
tensor_dcm = norm_tensor(tensor_dcm)
# Quantize
tensor_dcm = (tensor_dcm * 255).clamp(0, 255).to(torch.uint8)
drawings
Since the dicom image is three-dimensional data, when visualizing it, we generally divide it into multiple slices on the z-axis to visualize it sequentially, and in this paper, we adopt a 5*5 grid to visualize up to 25 slices.
def show_dciom(tensor_dicom).
# Find the maximum and minimum values of the image
vmin, vmax = tensor_dicom.min(), tensor_dicom.max()
# Create a graphics window
fig, axes = (5, 5, figsize=(15, 15)) # 5x5 grid layout
count = 0
length = tensor_dicom.size()[0]
for i in range(25).
if count < length.
count += 1
else: return
count += 1 else: return
# Get the coordinates of the current image
ax = axes[i // 5, i % 5]
# Display the image
(tensor_dicom[i], cmap='gray') # , vmin=vmin, vmax=vmax
('off') # turn off the axes
plt.tight_layout() # avoid overlap
(f "Layer {i}")
()
One thing to be more aware of here is that the()In the function, we specify the vmin and vmax parameters; this is because when this parameter is not specified, the imshow function will automatically adjust the brightness of the points so that the point with the largest value corresponds to 255 brightness and the point with the smallest value corresponds to 0 brightness. Given that the maximum and minimum pixel values of neighboring slices may have large differences, this will make the brightness of the image of neighboring slices more abnormal, as shown in the following figure:
The upper left region of these two images is actually similar in brightness, but there is a large difference in the visualized images, which would be misleading to observe.
Visualization of frequency distribution histograms
The frequency distribution histogram of visualized MRI images has important significance in medical image processing, which mainly includes the following aspects:
-
Image Contrast Analysis: A frequency distribution histogram shows the distribution of different gray levels (or pixel intensities) in an MRI image. By analyzing the shape and extent of the histogram, the contrast of the image can be understood. For example, a wider distribution of the histogram indicates a higher contrast image that can better distinguish between different tissues or structures.
-
Image Equalization: The contrast of an image can be improved by histogram equalization, which makes the low-contrast areas clearer. The equalization process enhances the visual effect of an image by redistributing the pixel values in the image, resulting in a more even distribution of the histogram.
-
Organizational segmentation: Histograms of frequency distributions can help determine appropriatethresholds, to perform image segmentation. By analyzing the histogram, a suitable threshold can be selected to separate different tissues or lesions from the background.
-
Image Quality Assessment: Histogram analysis can reveal image quality problems, such as overly dark or bright images, or the effects of image noise. The shape of the histogram makes it possible to assess whether the image needs further processing or optimization.
Before plotting the histogram of the frequency distribution, it is necessary to spread the 3D vectors, and in this paper we use thefunction drawing
def show_hist(tensor_dicom).
# Flatten the pixel values of all images into a one-dimensional array
pixel_values = tensor_dicom.numpy().flatten()
# Draw the histogram
(figsize=(10, 6))
(pixel_values, bins=50, color='gray', edgecolor='black')
('Histogram of All Pixel Values')
('Pixel Value')
('Frequency')
(True)
()
The histogram presents the following step-by-step with a peak near val=0. This is due to the fact that most of the area in the MRI image does not contain human tissue and is null value 0.
If points other than zero are excessively concentrated at smaller values (<100), then it is likely that there is an extremely bright noise point in the MRI image, making the quality of the normalization with the brightness of this noise point as the maximum value poor, for which the maximum value can be replaced with the 99th percentile, and the 99th percentile can be normalized to a brightness of 200. (As opposed to normalizing to 255, which would allow the brightness of the distinction between the brightness values of the pixel points in the maximum 1%).
The images in this example are all of high quality, so no special processing is required.
Code Summary
code architecture
main function
#
# Import custom utility functions
from utils import read_one_dicom, show_dciom, show_hist
# Define the directory containing the DICOM images
dicom_dir = "./train_images/4003253/1054713880"
# Read the DICOM image into a tensor with uint8 data type
tensor_dciom = read_one_dicom(dicom_dir, method="uint8")
# Display the DICOM image slices in a 5x5 grid layout
show_dciom(tensor_dciom)
# Plot the histogram of pixel values from the DICOM image slices
show_hist(tensor_dciom)
# Convert the tensor to a NumPy array for further processing or inspection
np_img = tensor_dciom.numpy()
package file
from .preprocess import read_one_dicom
from .show import show_dciom
from .show import show_hist
Read& Preprocessing
#
import numpy as np
import torch
import os
import re
import pydicom
from tqdm import tqdm
def norm_tensor(tensor_dciom):
"""
Normalize the image tensor to the range [0, 1].
Args:
tensor_dciom (): Tensor containing image data.
Returns:
: Normalized image tensor.
"""
# Calculate the maximum and minimum values of the image tensor
vmin, vmax = tensor_dciom.min(), tensor_dciom.max()
# Normalize the image tensor to the range [0, 1]
tensor_dciom = (tensor_dciom - vmin) / (vmax - vmin)
return tensor_dciom
def extract_number(filepath):
"""
Extract the numeric part from the DICOM filename.
Args:
filepath (str): Path to the DICOM file.
Returns:
int: Extracted number from the filename. Returns float('inf') if not found.
"""
# Get the filename (including extension)
filename = (filepath)
# Extract numeric part from filename, assuming filenames end with digits, ., ''
match = (r'(\d+)\.dcm$', filename)
return int((1)) if match else float('inf')
def read_one_dicom(dicom_dir, method = "", bar_title = ""):
"""
Reads DICOM files from a directory and converts them into a PyTorch tensor.
Args:
dicom_dir (str): Directory containing DICOM files.
method (str): Optional method to process the tensor ('norm' for normalization, 'uint8' for normalization and conversion to uint8).
bar_title (str): Optional title for the progress bar.
Returns:
: PyTorch tensor containing image data from DICOM files.
"""
# Get all DICOM files and sort them based on numeric part of the filename
dicom_files = [(dicom_dir, f) for f in (dicom_dir) if ('.dcm')]
dicom_files.sort(key=extract_number)
# Create an empty list to store image data
dcm_list = []
# Initialize tqdm progress bar
with tqdm(total=len(dicom_files), desc='Processing DICOM files', unit='dcm', unit_scale=True, unit_divisor=1000000) as pbar:
# Iterate over each DICOM file and read image data
for count, dicom_file in enumerate(dicom_files, start=1):
# Read the DICOM file
dcm = (dicom_file)
# Extract and convert image data to a NumPy array
image_data = dcm.pixel_array.astype(np.float32)
# Add the image data to the list
dcm_list.append(image_data)
# Update progress bar description
pbar.set_description(bar_title + 'Reading')
# Update progress bar
(1)
# Convert the list of image data to a PyTorch tensor and stack into a 3D tensor
tensor_dcm = ([(image_data) for image_data in dcm_list])
if method == "norm":
# Normalize the image tensor
tensor_dcm = norm_tensor(tensor_dcm)
elif method == "uint8":
# Normalize the image tensor
tensor_dcm = norm_tensor(tensor_dcm)
# Scale the tensor values to the range [0, 255] and convert to uint8 type
tensor_dcm = (tensor_dcm * 255).clamp(0, 255).to(torch.uint8)
return tensor_dcm
Visualizing, plotting histograms
#
import numpy as np
import torch
import as plt
def show_dciom(tensor_dicom):
"""
Display MRI image slices in a 5x5 grid layout.
Parameters:
tensor_dicom (): Tensor containing MRI image slices, expected shape is (N, H, W),
where N is the number of slices, and H and W are the height and width of the images.
"""
# Calculate the minimum and maximum pixel values in the tensor
vmin, vmax = tensor_dicom.min(), tensor_dicom.max()
# Create a figure with a 5x5 grid layout
fig, axes = (5, 5, figsize=(15, 15)) # 5x5 grid layout
count = 0
length = tensor_dicom.size(0)
for i in range(25):
if count < length:
count += 1
else:
return
# Get the current subplot's axis
ax = axes[i // 5, i % 5]
# Display the image
(tensor_dicom[count - 1], cmap='gray', vmin=vmin, vmax=vmax)
('off') # Hide the axis
plt.tight_layout() # Adjust layout to prevent overlap
(f"Layer {i + 1}") # Title indicating the last displayed slice
()
def show_hist(tensor_dicom):
"""
Plot the histogram of pixel values for all MRI image slices.
Parameters:
tensor_dicom (): Tensor containing MRI image slices, expected shape is (N, H, W).
"""
# Flatten all image pixel values into a single 1D array
pixel_values = tensor_dicom.numpy().flatten()
# Plot the histogram
(figsize=(10, 6))
(pixel_values, bins=50, color='gray', edgecolor='black')
('Histogram of All Pixel Values')
('Pixel Value')
('Frequency')
(True)
()
Next preview.
Discuss the solution to this question
It's not easy to make, so please give me a free like, please!