From images to treatment: the role of artificial intelligence in modern radiotherapy

The Laboratory of Imaging Technologies (LST) at the Faculty of Electrical Engineering of the University of Ljubljana plays an important role in the advancement of modern radiotherapy. Researchers at LST focus on the development and rigorous evaluation of state-of-the-art methods for medical image analysis, with a particular emphasis on automated segmentation of critical organs. In close collaboration with clinical partners, they aim to improve the accuracy, efficiency and reliability of radiotherapy planning, contributing to safer treatment and a better quality of life for patients.

Image-guided radiotherapy

Radiotherapy is a well-established method for treating cancer, in which tumors and nearby lymph nodes are irradiated with ionizing radiation (usually high-energy X-rays in the megavoltage range). It is often used in combination with surgical and systemic treatment (chemotherapy) – for example, a patient may be referred for irradiation before or after surgery.

The main advantage of radiotherapy is its locality and lower invasiveness compared to other approaches, but this also requires extremely precise planning enabled by modern imaging diagnostics. Before treatment, patients usually undergo at least computed tomography (CT) imaging, and often also magnetic resonance imaging (MR) or positron emission tomography (PET).

On the CT image of the patient’s body, radiation oncologists manually delineate tumors and the surrounding healthy organs that are essential for normal body function – these are referred to as “critical organs”. These contours form the basis for precise radiotherapy planning. Based on the dose prescribed by the oncologist, medical physicists design the treatment plan so that the tumor receives the prescribed dose while critical organs receive as little radiation as possible. This reduces the likelihood of complications after treatment. Treatment planning is a demanding task that requires deep expertise and often involves finding a balance between treatment success (destroying cancer cells) and the patient’s quality of life afterwards. Irradiation is usually delivered in smaller doses (so-called fractions) spread over several weeks.

Manual delineation of critical organs is time-consuming and subject to variability

The manual delineation process is essentially similar to children’s coloring books – each organ must first be identified and then carefully “colored in”. In practice, this requires exceptional knowledge of anatomy and considerable experience, as the boundaries between organs are often poorly visible. Because CT and MR images are three-dimensional, the delineation must be repeated on every image slice. This is similar to coloring an entire coloring book for each patient – a single image may contain 100 or more slices, meaning that the entire process can take up to two hours per patient.

However, the development of computer vision, artificial intelligence and deep learning methods in recent decades has opened the door to automating even very demanding tasks in medicine, including the delineation of critical organs. This is one of the research areas of the Laboratory of Imaging Technologies (LST) at the Faculty of Electrical Engineering, where researchers develop and evaluate methods for medical image analysis across various applications: from radiotherapy to spinal surgery planning, diagnosis and monitoring of multiple sclerosis, cardiological measurements, aneurysm detection and brain age estimation.

Automatic segmentation of critical organs using artificial intelligence methods

In recent research, Asst. Prof. Dr Gašper Podobnik and Prof. Dr Tomaž Vrtovec from LST, in collaboration with the Institute of Oncology Ljubljana, focused on radiotherapy of the head and neck region. With the approval of an ethics committee, they created the first publicly available dataset of its kind, called HaN-Seg, which contains paired CT and MR images of patients referred for radiotherapy together with precise manual delineations of critical organs.

Using this dataset, they also conducted a study on inter-observer variability – in other words, they measured the extent to which the delineations of the same organ on the same image differ between two experts, for example between a resident and a specialist (Figure 2).

Variability between observers is expected and significant, and the study is particularly important for the objective evaluation of automatic segmentation methods for critical organs. Researchers at LST developed a multimodal segmentation method that automatically segments 30 critical organs in the head and neck region based on aligned CT and MR images of the same patient. In addition, they organized an international challenge, inviting research groups from around the world to address the same problem. The results are very encouraging: automatic methods achieve a quality comparable to manual delineation.

Multi-level evaluation of the quality of automatic segmentations

Errors in the segmentation of critical organs can have serious consequences for a patient’s life after treatment, even if cancer cells are successfully destroyed. The high risk of complications requires great responsibility and therefore comprehensive evaluation of automatic methods in comparison with manual delineations.

In most studies, automatic segmentations are evaluated using geometric metrics based on overlap and distances. In simple terms, this means placing the automatic “coloring” over the manual one and calculating the difference.

Researchers from LST extended this analysis with dosimetric and psychometric evaluation. In dosimetric evaluation, they examined whether a patient would receive a higher radiation dose when using automatic segmentations compared to manual delineations. In psychometric evaluation, experienced oncologists assessed the automatic segmentations on a scale from 1 to 5, where 1 represents unusable and 5 represents clinically acceptable segmentation.

On average, automatic methods achieved greater overlap and smaller distances from reference segmentations compared to the variability between observers. A simplified dosimetric analysis showed that 70% of automatic segmentations were dosimetrically acceptable, while the average psychometric score of 3.9 indicates that most automatic segmentations are clinically usable with minimal or even no corrections.

Looking to the future

The evaluation results demonstrate very strong performance of automatic methods for segmenting critical organs in the head and neck region. These findings can also be generalized to other anatomies, as the head and neck region is considered one of the most demanding due to the presence of many small and poorly visible structures. The success of these methods is also reflected in the growing number of commercial tools already entering clinical practice. Although the current workflow still requires verification of each automatic segmentation, we can expect fewer errors and even greater reliability in the future.

The future at the intersection of artificial intelligence and radiotherapy is therefore extremely promising. Automation of critical organ segmentation is only the beginning – research groups are increasingly working on automatic segmentation of tumors and lymph nodes, the development of tools supporting MR-only radiotherapy, and the use of advanced methods during the planning phase, for example to predict dose distribution, which can further accelerate the work of medical physicists. Researchers at LST are actively addressing several of these challenges and welcome collaboration with students interested in the interdisciplinary field of medical image analysis and artificial intelligence, either through extracurricular activities or within the master’s programme Biomedical Engineering.