|Year : 2020 | Volume
| Issue : 1 | Page : 29-35
A systematic review of methodology in prognostic and prediction modelling with radiomics for non-small cell lung cancer - Part IA systematic review of methodology in prognostic and prediction modelling with radiomics for non-small cell lung cancer – Part I
R Patil1, L Wee2, A Dekker2
1 Department of Radiation Oncology (MAASTRO), GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre (MUMC), Maastricht, The Netherlands; Philips Research Bangalore, India
2 Department of Radiation Oncology (MAASTRO), GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre (MUMC), Maastricht, The Netherlands
|Date of Web Publication||22-Oct-2020|
Department of Radiation Oncology (MAASTRO), GROW - School for Oncology and Developmental Biology, Maastricht University Medical Centre (MUMC), Maastricht, The Netherlands Philips Research Bangalore
Source of Support: None, Conflict of Interest: None
Objective: The aim of this study was to systematically review the literature to synthesize and investigate repeatability and reproducibility of radiomic features, considered in building prognostic and prediction modelling with respect to NSCLC.
Methods: The PubMed database was searched using combinations of the broad Haines and Ingui filters along with a set of text words specific to cancer, radiomics, reproducibility and repeatability. This systematic review was performed by two reviewers working entirely independently, and has been reported in compliance with PRISMA guidelines.
Results: Out of 624 unique records, 41 full text articles were subjected to review. The studies were primarily in NSCLC. The imaging modalities were CT, PET and cone-beam CT – no studies addressed MR. Only 7 studies addressed in detail every methodological aspect related to image acquisition, pre-processing and feature extraction. Only few studies have made either the image set or software, or both, openly accessible. Due to heterogeneity in statistical metrics, a meta-analysis of pooled data was not possible.
Conclusions: The repeatability and reproducibility of radiomic features are sensitive in varying degrees to processing details such as image acquisition settings, image reconstruction algorithm, image pre-processing and software used to extract radiomic features.
Advances in Knowledge: Intra-class and concordance correlations were the most widely used statistical metrics, but arbitrarily selected cut-offs were variable. First-order features were overall more reproducible than shape metrics and textural features. Entropy was consistently reported as one of the most stable first-order features. There was no emergent consensus regarding either shape metrics or textural features.
|How to cite this article:|
Patil R, Wee L, Dekker A. A systematic review of methodology in prognostic and prediction modelling with radiomics for non-small cell lung cancer - Part IA systematic review of methodology in prognostic and prediction modelling with radiomics for non-small cell lung cancer – Part I. J Precis Oncol 2020;1:29-35
|How to cite this URL:|
Patil R, Wee L, Dekker A. A systematic review of methodology in prognostic and prediction modelling with radiomics for non-small cell lung cancer - Part IA systematic review of methodology in prognostic and prediction modelling with radiomics for non-small cell lung cancer – Part I. J Precis Oncol [serial online] 2020 [cited 2021 Jun 19];1:29-35. Available from: https://www.jprecisiononcology.com/text.asp?2020/1/1/29/298284
| Introduction|| |
Lung cancer remains a major public health issue in developed countries, accounting for about 20% and 28% of cancer-related deaths in Europe and the United States of America, respectively (1). In 2012, the World Health Organization estimated in excess of 1.2 million new cases of lung cancer and 1 million deaths from lung cancer globally. More people die of lung cancer than of colon, breast and prostate cancer combined (2). Non-small cell lung (NSCLC) cancer accounts for 85% of all the lung cancers, making it the second most common type of cancer in both men and women. While a range of intervention options exist for NSCLC, a significant challenge remains for accurate risk stratification and optimal treatment selection.
Recent interest in personalized medicine is driven by an expectation of better outcomes (3) if unique characteristics of a person and their disease can be taken into account. There is a wide range of potential prognostic and/or predictive factors (4), among which are image-derived markers, i.e. “radiomics”. The aim is to identify factors that accurately indicate histology, intra-tumour heterogeneity and differential response to treatments. Radiomics-assisted personalized medicine requires computerized extraction of quantitative image metrics, i.e. “features”, from vast volumes of medical imaging examinations that can subsequently be linked to compare outcomes of different diseases/treatments. The hypothesis is that radiographic images contain more information about the tumour phenotype than human-generated semantic features derived by the unaided eye. Radiomics analysis may be readily integrated into automated multi-factorial decision support tools (4.a), that could assist physicians in selecting a superior treatment for a given individual.
Radiomics as disease characterization
A radiomics-guided approach has been investigated as a non-invasive form of tissue biopsy (4.b). Biopsies are the gold standard for determining histology and differentiation grade, but they are also associated with some risk of complications, such as pneumothorax (5). In lung cancer, fine needle biopsies take selective samples, thus results may vary due to finite sampling and tumour heterogeneity (6).
Small biopsies and radiomics analysis could complementarily inform tumour characterization and individual prognosis, which is then taken into account when selecting treatment. A radiomics-guided analysis has an advantage because it attempts to characterize the entire tumour in situ and can be repeated at multiple timepoints.
Radiomics as prognostic or predictive marker
Several key publications are credited with showing how radiomic features may contain additional information pertaining to treatment outcome and/or response monitoring (7, 8, 9). Complementary information could be obtained from radiological images and post-operative tissue samples to predict treatment outcome [Figure 1]. For (chemo)-radiotherapy treatment, radiomic features from follow-up imaging could be used to non-invasively monitor tumour response.
|Figure 1: Generic clinical workflow incorporating radiological examinations and biopsies, showing the potential applications of radiomics-based analysis for disease characterization, treatment selection and follow-up|
Click here to view
Distinction from previous review(s)
Generalizability and methodological questions pertaining to radiomics-based models are active research topics. Potential pitfalls have been identified in several other general reviews (10, 11, 12, 13). Specifically, methodological steps in congruence with TRIPOD (14) need to be taken in the reporting of radiomics modelling studies with many candidate features (sometimes well in excess of 100 features per outcome) to mitigate risks of over-fitting, false-positive associations and narrow clinical generalizability of models using external independent validation. A recent systematic review addressing studies of repeatability and reproducibility of radiomics features indicates that lack of standardization of feature definitions, clarity of reporting and open access (to images and software) presently slows down efforts to generalize and apply radiomics in clinical situations (15).
| Aim(S) of This Review|| |
In this review, we focus on updates in methodological robustness that have come to light since the previous reviews. Further, we wish to understand which are the prominent features that are considered by the studies to build predictive and prognostics models. In addition, we aim to analyze the approaches considered for feature selection, the software packages used for feature extraction and the methodology adapted to perform the validation while also identify the shortcomings and strengths associated with the studies.
| Overview of Radiomics Model Building and Model Validation|| |
A generic radiomics procedure for cancer outcome prediction comprises two principal steps; (i) feature extraction and (ii) signature computation. Feature extraction refers to an automated process of calculating quantitative image metrics from a radiological image annotated with a region of interest (ROI), typically the gross tumour volume (GTV). The ROI may be drawn via a manual, semi-automatic or automatic process, generally known as “segmentation”. The image and ROI are paired inputs into specialized computer software that calculates radiomic features.
A signature is a combination of one or more radiomic features that is statistically associated with the odds of a given clinical outcome. A prognostic/predictive signature may comprise either clinical observables or radiomic features, or a combination of both types. During model development, clinical risk factors (e.g. disease stage) can be combined with radiomics features in order to fit a statistical model to the observed outcomes [Figure 2]. In validation and prior to clinical use, the radiomic features and/or clinical risk factors are used as to predict the expected outcome using an already-fitted model [Figure 3] and compared to actual outcome.
In general, the data used to validate the model should be entirely independent from the data used to build the model, and not used anywhere during the model development process. This is known as external validation. If a subset of data has been used for interim validation of a model-under-development, this is known as cross-validation. Suitable external validation data must be different from the model-building data in at least one way, for instance, collected over a different time period, or at a different hospital/institution, or in a different clinical practice setting or in an intentionally different medical indication.
In radiomics studies of NSCLC, the most commonly encountered tomographic images are obtained from CT (Computed Tomography) or PET (Positron Emission Tomography), employing a wide variety of imaging protocols. There are vast numbers of permutations of image acquisition parameters, therefore repeatability and reproducibility of radiomic features needs to be considered when developing and validating models (15). To date, radiomic features are modality specific, though it is possible that a radiomic signature might comprise features derived from different imaging modalities. Standardization of image acquisition for the purpose of generalizing radiomics across multiple institutions remains an open question and may be hampered by institutional preferences. There are ongoing efforts by the National Cancer Institute Quantitative Imaging Network (NCI-QIN), the Quantitative Imaging Biomarkers Alliance (QIBA) and the Radiological Society of North America (RSNA) to arrive at some consensus for standards (16-18).
Defining regions of interest (ROIs) within a radiological examination is a crucial step in radiomic analysis. The interesting part of the image needs to be segregated from the rest, in order to isolate potentially useful information. Specifically in NSCLC, ROIs of relevance are the hyperintense tumours in the lung. These may be delineated as a single aggregated ROI with the index tumour or as individual ROIs, but it is important to clearly document the ROI from which the radiomic features are derived. Besides the tumour and nodes, it is also possible to consider radiomic features derived from adjacent areas in healthy tissue, as these may encode some information about blood flow, oedema, necrosis, cell density or other potentially useful risk factors (19).
Besides manual segmentation, there are various (semi-)automated segmentation algorithms that may be used to define a ROI, that typically fall into one of the following categories: (i) intensity-based, (ii) object shape-based or model-based, (iii) neighboring-anatomy constrained, (iv) region-growing based, or (v) artificial neural network (ANN) based. Each of these vary in degree of automation and algorithmic accuracy. Shape-based and ANN-based models have shown more consistently reproducible results, but it is an open question about what is an adequately accurate ROI for radiomics. It is rare to find a single tumour segmentation algorithm that works on all medical images (20, 21). Related to this is that inter-physician differences in manual ROI annotation is also a major source of disagreement (22). A study (23) suggests that machine-assisted segmentation (automatic initial segmentation, followed by physician review and manual corrections) provides both more consistent and more accurate ROIs of NSCLC than manual segmentation alone.
Each radiomic feature falls into one of two groups – Semantic Features or Agnostic Features. Semantic features generally encompass descriptive metrics of an ROI such as volume, location, sphericity, longest dimension, etc. Thus, semantic features are likely closest to what human radiologists report. Agnostic features relate to arbitrary metrics such as describing spatial variation in image intensity.
First-order agnostic features quantify only the numerical distribution of intensities, e.g. the mean. Second-order features quantify spatial variation (i.e. texture), and were first introduced in 1973 (24). Examples of second-order features include gray level run-length, gray level co-occurrence and other (2D or 3D) matrix measures as described in elsewhere(25, 26). Third and higher order features require the application of digital image filters immediately prior to feature extraction. The primary purpose of digital filtering is to amplify spatial patterns. For example, directional wavelets (akin to Fourier transformations in analog signal processing), resampled to an isotropic spacing, preferentially select intensity non-uniformity of either high or low modulation frequency. A Laplacian-of-Gaussian (LoG) filter may be used to emphasize coarse textural patterns. A hybrid class of features has been investigated that combines information from intensity, morphology and texture, such as threshold Minkowski functional analysis (27) and fractal analysis (28).
Radiomics features derived after resampling and filtering tend to depend on how the methods are implemented and this contributes to discrepancies between radiomics software packages.
An important procedure step is to select features during model development, and we had included this before model-building (Fig 2). Selection may be achieved by stepwise regression, recursive feature elimination or a regularized machine learning approach (such as ElasticNet). The reader is referred to a recent methodological review of machine-based classifiers for cancer outcomes modelling (29). A human expert may also hand-pick features that are known to be less sensitive to image acquisition settings and digital pre-processing. Finally, unsupervised learning methods (such as hierarchical clustering) may be used to reduce the number of features in the final model (30-32).
Model development and validation
Regarding model development and validation, two main approaches are adopted in radiomics-based modelling; a purely data-driven approach versus a hypothesis-based approach. The data-driven approach assumes no a priori knowledge. A vast number of radiomics features (of the order of 102 or greater) is calculated from each ROI. A smaller set of features contributing to the final model is generated during the model development.
A hypothesis-driven approach requires that a clinical domain expert initially selects a candidate set of features to be examined during model development. Such prior knowledge may come from independent studies of the association between these candidate features to either an outcome or a biological phenomenon of interest.
There is a justified concern that model parameter-tuning based on a large number of radiomics features on relatively few outcome events will lead to high risk of false positive association and over-fitting (33). Some mitigation measures such as multiple-fold cross-validation and repeated subsampling are generally accepted as best practice. Even so, it is broadly accepted that a lack of external validation leads to overestimation of model performance.
A pragmatic approach to radiomics model development therefore requires some combination of data and hypothesis driven approaches, together with robust feature selection and external validation.
| Methods|| |
We performed the systematic review and analysis until October 2018. The included articles met all of the eligibility criteria given below.
We accepted only peer-reviewed full-text reports published in journals that presented results of prognostic or predictive statistical models based on the use of one or more radiomics features. Reports giving purely qualitative results, reports published as letters to the editor, or in the form of abstracts (such as conference proceedings) were not included.
Only full-text reports in the English language were included in this review.
A manual electronic search was conducted in PubMed (MEDLINE citations had been previously merged into the PubMed repository). No search was made in grey literature sources for unpublished studies or conference proceedings.
A search of PubMed citations was performed using a broad Haynes (34) and Inqui (35) filters in combination with the modifications proposed by Geersing et al. (36) (each criteria combined using an 'OR' logical operator). A preliminary search in January 2017 for histological prediction using a radiomics approach was combined with the results of the above in October 2018. For the final search, additional filters for “non small cell lung cancer” (MeSH major topic), and text (abstract/title) strings “radiomics”, “quantitative imaging”, “features” and “texture/textural analysis” were applied.
We allowed studies specifically using radiomic features in a multivariable statistical model for either histology or treatment outcomes for NSCLC. Treatment of metastatic diseases from primary NSCLC was allowed, as were any interventions involving surgery, chemotherapy, radiotherapy and combinations thereof.
We restricted the date range of reviewed articles from January 2011 to October 2018, as clinical interventions for NSCLC and mathematical techniques had changed significantly since earlier investigations. Due to potential bias in very small sample sizes, we restricted our review to studies with more than 10 human subjects.
Electronic full text articles were downloaded using the university library subscription of author LW. A review-specific shared DropBox folder was set up to handle document collection, data extraction forms as well as disseminating the reviewer findings.
Two reviewers (RP and LW) worked jointly throughout all phases of the study selection process (abstract screening, eligibility and inclusion for full-text evaluation). The reviewers jointly compared the titles and the abstracts against the inclusion criteria. Discrepancies were resolved by discussion until unanimous agreement.
From each reviewed paper, we extracted information about sample size, screening/treatment type and imaging modality. For histological prediction, we considered whether the study involved histology classification at the human subject level, or nodule classification at the level of each examined nodule. For interventional procedures, we looked at the overall staging of the disease prior to treatment.
Among the intervention studies, we extracted which type of features were reported to be predictive. We also recorded what software was used to calculate radiomic features and what kind of feature selection (if any) had been used during model construction. We extracted the number of features used during model building, the number of events (for the outcomes of interest), the reported predictive performance of the model, and whether there had been an external validation of the model.
The population reported in the study are subjects diagnosed with non-small cell lung cancer. Animal subjects, biological samples outside of the human body, and non-clinical imaging studies were specifically excluded. Eligible results may have come from either retrospective or prospective studies, and may have been either observational or experimental in nature. Both single and multi-institution studies were allowed.
Outcomes for synthesis
We specifically assessed the following outcomes: survival after treatment, histology classification, pathological response and treatment failure (such as progression, recurrence or metastases).
| Results|| |
Radiomics for predicting histology or cancer stage
Our literature survey located the same two studies Dhara et al, Dilger et al (37,38) on NSCLC nodule classification, as found by Scrivener et al. in 2016 (11). Both studies used nodules as the primary unit of analysis. Further, we located three new studies of radiomics for either staging or nodule classification [Andersen et al, Patil et al and Wu et al] (3, 39, 40) after Parmar et al. (41). All six studies used CT as the imaging modality. Parmar et al (41) included a mixture of lung and head and neck cancers, Wu et al (40) included other lung cancers and Andersen et al (42) included cases with suspected but not confirmed lung cancer.
Dhara et al. (35) reported an AUC ranging from 0.841 - 0.951 for three different configurations used for binning the nodules based on different combinations of the rank of malignancy. The model was built using 891 unique nodules, however there was no explanation of how the subjects were selected for analysis, leading to a potential for selection bias. There was missing documentation about acquisition parameters and no external validation in an independent dataset, therefore we would expect that the reported AUCs would be over-optimistic.
Dilger et al (38) introduced features of the lung parenchyma and showed a modest improvement of nodule classification (AUC increased from 0.918 to 0.938). This model is at risk of over-fitting, since only 50 nodules were considered in their analysis and they have selected in total 58 features for the model building. There was no external validation of model performance. Andersen et al  used radiomics to differentiate malignant and non-malignant mediastinal nodes in 29 patients with NSCLC with a reported AUC of 0.834. To enhance the radiomic features of the lymph nodes, they applied a Laplacian of Gaussian filter within the ROI. The authors claimed that in-sample reproducibility was excellent, but did not demonstrate this in an external validation dataset with nominally similar image acquisition settings.
In contrast with the previous studies, Parmar et al (41). Performed a robust study using feature dimensionality reduction based on consensus clustering for a prognostic signature in 422 lung and 136 head/neck cancer patients, which were then externally validated in an additional 225 lung and 95 head/neck cases, respectively. For lung cancer staging and histology classification, they reported a final lung cancer radiomic signature with external AUCs of 0.64 and 0.64, respectively. These AUCs are much lower than the previously cited works, but may be a more realistic estimation of the radiomics model performance.
Some methodological deficiencies were also noted in new studies of histological classification. Patil et al. (39) demonstrated the extra value of textural radiomic features when classifying NSCLC histology, increasing the classification accuracy from 67% to 88% compared to shape and intensity features alone (39). The results were based on balancing the dataset using selective resampling of the minority events (i.e. SMOTE ) with a 10-fold cross validation approach. Again, no external dataset was used to independently test accuracy. Separately, the results of Wu et al (40) were consistent with Patil et al., in support of added value of radiomic features when classifying the histology of NSCLC, however Wu et al also didn't perform external validation of the algorithm.
Synthesis of radiomics performance in predicting histology or cancer stage
Radiomic features appear to have potential for discrimination of malignant nodules and mediastinal lymph node involvement in NSCLC. The reported metrics initially look appealing (AUCs >0.83), but requires external validation to prove that such performance is generalizable. Radiomic features appear to have moderate utility for classification of NSCLC histology. One methodologically robust study by Parmar et al with large sample size and careful feature selection suggests an AUC of about 0.64 (externally validated) for staging and histology is the current limit of performance with hand-engineered radiomics features. Two studies have been consistent in reporting that textural radiomic features could have added value beyond models that use only non-textural features.
| Discussion of Methodological Issues Encountered in Reviewed Studies|| |
Most of the models that are built using radiomics features for predicting the treatment outcome or for clinical staging use a cross validation approach. Although, the approach of cross validation might seem valid and correct from a machine learning perspective, without external validation with data from other institutes there is a high risk of models being over fit. It is a better approach to perform cross validation on local data to tune the model parameters and finally validate the model with external data to the test the model robustness before reporting the results. It is also observed that there is no uniform approach for robust feature selection to identify the most significant features. It will also be useful to understand the contribution to the final signature by each of features, which is possible, if their weights in terms of predictive power are better reported in the feature selection approach. It is also quite possible that different feature selection approaches will highlight different features as most significant; hence, it is imperative that ensemble approaches are adapted among the feature selection methods to arrive at the subset of features that are most significant. Further, events in some studies are not clearly defined or not stated, making it difficult to trust the data points on which the models are built. It is good practice to be precise and transparent about the endpoints chosen and the event rate.
| References|| |
Malvezzi M, Bertuccio P, Rosso T, et al.
European cancer mortality predictions for the year 2015: does lung cancer have the highest death rate in EU women?. Ann Oncol. 2015 Apr;26(4):779-86.
Hensing T, Chawla A, Batra R, et al.
A personalized treatment for lung cancer: molecular pathways, targeted therapies, and genomic characterization. Adv Exp Med Biol. 2014;799:85-117.
Mok TS. Personalized medicine in lung cancer: what we need to know.. Nat Rev Clin Oncol. 2011 Aug 23;8(11):661-8. 4.a Lambin P, van Stiphout RG, Starmans MH et.al Predicting outcomes in radiation oncology—multifactorial decision support systems. Nature Reviews Clinical Oncology.2013;10:27–40. 4.b Neri E, Re DM, Paiar F, et.al. Radiomics and liquid biopsy in oncology: the holons of systems medicine. Insights Imaging. 2018 Dec;9:915-24.
Tomiyama N1, Yasuhara Y, Nakajima Y, et al.
CT-guided needle biopsy of lung lesions: a survey of severe complication based on 9783 biopsies in Japan. K. Eur J Radiol. 2006 Jul;59:60-4.
Wu CC, Maher MM, Shepard JA. Complications of CT-guided percutaneous needle biopsy of the chest: prevention and management. AJR Am J Roentgenol. 2011 Jun;196:W678-82.
Aerts HJ, Velazquez ER, Leijenaar RT, et al.
Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach, Nat Commun 2014;5:4006.
Coroller TP, Grossmann P, Hou Y, et al.
“CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma.” Radiother Oncol 2015;114:345-50.
Fried DV, Mawlawi O, Zhang L, et al.
Stage III Non– Small Cell Lung Cancer: Prognostic Value of FDG PET Quantitative Imaging Features Combined with Clinical Prognostic Factors. Radiology 2016;278:214-22.
Kumar V, Gu Y, Basu S, et al.
Radiomics:vthe process and the challenges, Magn ResonvImaging 2012;30:1234–48.
Scrivener M, de Jong EEC, van Timmeren JE, et al.
Radiomics applied to lung cancer: a review. Translational Cancer Research 2016;5:398-409
Yip SS, Aerts HJ. Applications and limitations of radiomics. Phys Med Biol 2016;61: R150-166
Larue RT, Defraene G, De Ruysscher D, et al.
Quantitative radiomics studies for tissue characterization: a review of technology and methodological procedures. Br J Radiol 2017;90: 20160665.
Gary S. Collins, Johannes B. Reitsma, Douglas G. Altman, et al.
“Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement, Annals of internal medicine “, Ann Intern Med. 2015;162:55-63. DOI: 10.7326/M14-0697.
Alberto Traverso et.al, Repeatability and Reproducibility of Radiomic Features: A Systematic Review, International journal of Radiation Oncology, International of Radiation Oncology 2018;102:e4
Coroller TP, Grossmann P, Hou Y, et al
. “CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma.” Radiother Oncol 2015;114:345-50.
Clarke LP, Sriram RD, Schilling LB. “Imaging as a Biomarker: Standards for Change Measurements in Therapy workshop summary”. Acad Radiol 2008;15:501–530.
Shankar LK, Hoffman JM, Bacharach S, et al.
“Consensus recommendations for the use of 18F-FDG PET as an indicator of therapeutic response in patients in National Cancer Institute Trials”. J Nucl Med 2006;47:1059– 66.
Robert A. Gatenby, MD, corresponding author Olya Grove, and Robert J. Gillies. Quantitative Imaging in Cancer Evolution and Radiology. 2013;269:8–14.
Buckler AJ, Bresolin L, Dunnick NR, Sullivan DC. A collaborative enterprise for multi-stakeholder participation in the advancement of quantitative imaging. Radiology 2011;258:906–914.
Gatenby RA, Grove O, Gillies RJ. Quantitative imaging in cancer evolution and ecology. Radiology 2013;269(1):8–15.
Parmar C, Rios Velazquez E, Leijenaar, et al.
Robust Radiomics feature quantification using semiautomatic volumetric segmentation. PLoS One. 2014;9(7):e102107. doi: 10.1371/journal.pone.0102107.
C Parmar, ER Velazquez, R Leijenaar, M Jermoumi, S Carvalho, RH Mak Robust radiomics feature quantification using semiautomatic volumetric segmentation. PloS one 9;e102107.
Coroller TP, Grossmann P, Hou Y, et al.
“CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma.” Radiother Oncol 2015;114:345-50.
Clarke LP, Sriram RD, Schilling LB. “Imaging as a Biomarker: Standards for Change Measurements in Therapy workshop summary”. Acad Radiol 2008;15(4):501–530.
Shankar LK, Hoffman JM, Bacharach S, et al.
“Consensus recommendations for the use of 18F-FDG PET as an indicator of therapeutic response in patients in National Cancer Institute Trials”. J Nucl Med 2006;47(6):1059-1066.
Robert A. Gatenby, MD, corresponding author Olya Grove, and Robert J. Gillies,“Quantitative Imaging in Cancer Evolution and Radiology”. 2013 Oct; 269(1): 8–14.
Cusumano D et.al, Fractal-based radiomic approach to predict complete pathological response after chemo-radiotherapy in rectal cancer, Radiol Med. 2018 Apr; 123(4):286-295. doi: 10.1007/s11547-017-0838-3. Epub 2017 Dec 11
Deist TM, Machine learning algorithms for outcome prediction in (chemo)radiotherapy: An empirical comparison of classifiers, Med Phys. 2018;45(7):3449-59. doi: 10.1002/mp.12967. Epub 2018 Jun 13.
Aerts, H. et al.
Identification of residual metabolic-active areas within NSCLC tumours using a pre radiotherapy FDG PET CT scan: a prospective validation. Lung. Cancer. 2012;75:73-6.
Aerts, H. et al.
Identification of residual metabolic-active areas within individual NSCLC tumours using a pre-radiotherapy (18)Fluorodeoxyglucose–PET–CT scan. Radiother. Oncol. 2009;91:386-92.
Carrasco, J.L., King, T.S. & Chinchilli, V.M. “The concordance correlation coefficient for repeated measures estimated by variance components.” J Biopharm Stat. 2009;19:90-105.
Jake Lever, Martin Krzywinski & Naomi Altman, Model selection and overfitting, Nature Methods. 2016;13:703-4.
Haynes, R. B. et al.
Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey. BMJ 2005; 330:1179.
Ingui, B. J. & Rogers, M. A. Searching for clinical prediction rules in MEDLINE. J. Am. Med. Inform. Assoc. JAMIA. 2001;8:391–7.
Geersing, G.-J. et al.
Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PloS One, 2012;7:e32844.
Dhara AK, Mukhopadhyay S, Dutta A, et al.
“A Combination of Shape and Texture Features for Classification of Pulmonary Nodules in Lung CT Images”. J Digit Imaging 2016;29:466-75.
Dilger SK, Uthoff J, Judisch A, et al.
Improved pulmonary nodule classification utilizing quantitative lung parenchyma features. J Med Imaging (Bellingham) 2015;2:041004.
Ravindra Patil, Geetha Mahadevaiah, and Andre Dekker. “An Approach Toward Automatic Classification of Tumor Histopathology of Non–Small Cell Lung Cancer Based on Radiomic Features”. TOMOGRAPHY, December 2016;2: 374-7 DOI: 10.18383/j.tom.2016.00244.
Weimiao Wu, Chintan Parmar, Patrick Grossmann, et al.
Exploratory Study to Identify Radiomics Classifiers for Lung Cancer Histology. Front Oncol. 2016;30:6:71. doi: 10.3389/fonc.2016.00071.
Parmar C, Leijenaar RT, Grossmann P, et al.
Radiomic feature clusters and prognostic signatures specific for Lung and Head & Neck cancer. Sci Rep 2015;5:11044.
Andersen MB, Harders SW, Ganeshan B, et al.
CT texture analysis can help differentiate between malignant and benign lymph nodes in the mediastinum in patients suspected for lung cancer. Acta Radiol. 2016;57(6):669-76. doi: 10.1177/0284185115598808.
[Figure 1], [Figure 2], [Figure 3]