Systematic review and external validation of prediction models based on symptoms and biomarkers for identifying endoscopic activity in Crohn's disease.
Brand, Eelco C.; Elias, Sjoerd G.; Minderhoud, Itta M.et al.
2020 • In Clinical Gastroenterology and Hepatology
[en] BACKGROUND & AIMS: Endoscopic healing, an important target of treatment for Crohn's Disease (CD), requires ileocolonoscopy, which is costly and burdensome. We investigated whether published non-invasive models (based on symptoms and biomarkers) to evaluate CD activity have sufficient accuracy to replace ileocolonoscopy. METHODS: We performed a systematic review of published non-invasive diagnostic models to evaluate CD activity that used endoscopic features of activity (endoscopic activity) or healing as the reference standard. We externally validated these models for the outcome endoscopic activity (CD endoscopic index of severity scores of 3 or more) using data from the TAILORIX study (346 ileocolonoscopies in 155 patients) and the Utrecht Activity Index (UAI) study (93 ileocolonoscopies in 82 patients). We calculated area under the receiver operating characteristic curves (AUCROCs) for the models using data from these studies, and compared the performance of these models against measurements of fecal calprotectin (FC) and C-reactive protein (CRP). RESULTS: We screened 5303 articles and identified 27 models (from 21 studies) for our analysis. Seven models could be externally validated; in the TAILORIX dataset, these models identified patients with endoscopic activity with AUROC values ranging from 0.61 (95% CI, 0.51-0.70) to 0.81 (95% CI, 0.76-0.86). In this dataset, the AUROC value for FC concentration was 0.79 (95% CI, 0.74-0.85) and the AUROC value for CRP level was 0.72 (95% CI, 0.66-0.77). The AUROC values for the validation in the UAI-dataset were similar. In the TAILORIX and/or UAI-dataset, 4 of the 7 models, as well as the FC and CRP assays, were able to identify patients with endoscopic activity with positive predictive values of 90% or more. Two of the 7 models (but not the FC or CRP values) identified patients without endoscopic activity with a negative predictive value (NPV) of 90% or more, leading to correct prediction of endoscopic healing in 3.2% to 11.3% of all patients. For example, applying the Herranz-Bachiller model (1 of 7 models) at a NPV of 92.1% and positive predictive value of 91.9% correctly identified 35.7% of all patients in whom ileocolonoscopy could be avoided for expected endoscopic activity or healing but incorrectly identified 3.2% of all patients. Most ileocolonoscopies (66.5% in TAILORIX and 72.6% in the UAI of all ileocolonoscopies) could correctly be avoided based on concentrations of FC of 100 mug/g or less and 250 mug/g or higher. However, using this range of FC concentrations to identify patients that don't require ileocolonoscopy caused 18.7% of all patients in the TAILORIX cohort and 19.8% of all patients in the UAI cohort to incorrectly being predicted to have endoscopic activity or healing.. CONCLUSIONS: In a systematic review and external validation of non-invasive models to identify patients with endoscopic activity of CD, we found only 2 of 7 models evaluated to have NPVs of 90% or more, however, only leading to correct predicted EH in a small proportion of patients. Ileocolonoscopy must therefore be used to evaluate CD mucosal disease activity and healing.
Disciplines :
Gastroenterology & hepatology
Author, co-author :
Brand, Eelco C.
Elias, Sjoerd G.
Minderhoud, Itta M.
van der Veen, Julius J.
Baert, Filip J.
Laharie, David
Bossuyt, Peter
Bouhnik, Yoram
Buisson, Anthony
Lambrecht, Guy
Louis, Edouard ; Université de Liège - ULiège > Département des sciences cliniques > Hépato-gastroentérologie
Systematic review and external validation of prediction models based on symptoms and biomarkers for identifying endoscopic activity in Crohn's disease.
Neurath, M.F., Travis, S.P.L., Mucosal healing in inflammatory bowel diseases: a systematic review. Gut 61 (2012), 1619–1635.
Pineton de Chambrun, G., Blanc, P., Peyrin-Biroulet, L., Current evidence supporting mucosal healing and deep remission as important treatment goals for inflammatory bowel disease. Expert Rev Gastroenterol Hepatol 10 (2016), 915–927.
Best, W.R., Becktel, J.M., Singleton, J.W., et al. Development of a Crohn's disease activity index. National Cooperative Crohn's Disease Study. Gastroenterology 70 (1976), 439–444.
Harvey, R.F., Bradshaw, J.M., A simple index of Crohn's-disease activity. Lancet, 1, 1980, 514.
Falvey, J.D., Hoskin, T., Meijer, B., et al. Disease activity assessment in IBD: clinical indices and biomarkers fail to predict endoscopic remission. Inflamm Bowel Dis 21 (2015), 824–831.
Peyrin-Biroulet, L., Reinisch, W., Colombel, J.-F., et al. Clinical disease activity, C-reactive protein normalisation and mucosal healing in Crohn's disease in the SONIC trial. Gut 63 (2014), 88–95.
Mosli, M.H., Zou, G., Garg, S.K., et al. C-reactive protein, fecal calprotectin, and stool lactoferrin for detection of endoscopic activity in symptomatic inflammatory bowel disease patients: a systematic review and meta-analysis. Am J Gastroenterol 110 (2015), 802–819 quiz 820.
Bleeker, S.E., Moll, H.A., Steyerberg, E.W., et al. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol 56 (2003), 826–832.
Moons, K.G.M., de Groot, J.A.H., Bouwmeester, W., et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med, 11, 2014, e1001744.
Khanna, R., Nelson, S.A., Feagan, B.G., et al. Endoscopic scoring indices for evaluation of disease activity in Crohn's disease. Cochrane Database Syst Rev, 8, 2016, CD010642.
Geersing, G.-J., Bouwmeester, W., Zuithoff, P., et al. Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PLoS One, 7, 2012, e32844.
Ingui, B.J., Rogers, M.A., Searching for clinical prediction rules in MEDLINE. J Am Med Inform Assoc 8 (2001), 391–397.
Mary, J.Y., Modigliani, R., Development and validation of an endoscopic index of the severity for Crohn's disease: a prospective multicentre study. Groupe d'Etudes Therapeutiques des Affections Inflammatoires du Tube Digestif (GETAID). Gut 30 (1989), 983–989.
Daperno, M., D'Haens, G., Van Assche, G., et al. Development and validation of a new, simplified endoscopic activity score for Crohn's disease: the SES-CD. Gastrointest Endosc 60 (2004), 505–512.
Rutgeerts, P., Geboes, K., Vantrappen, G., et al. Predictability of the postoperative course of Crohn's disease. Gastroenterology 99 (1990), 956–963.
Moons, K.G.M., Wolff, R.F., Riley, R.D., et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med 170 (2019), W1–W33.
D'Haens, G., Vermeire, S., Lambrecht, G., et al. Increasing infliximab dose based on symptoms, biomarkers, and serum drug concentrations does not increase clinical, endoscopic, or corticosteroid-free remission in patients with active luminal Crohn's disease. Gastroenterology 154 (2018), 1343–1351.e1.
Minderhoud, I.M., Steyerberg, E.W., van Bodegraven, A.A., et al. Predicting endoscopic disease activity in Crohn's disease: a new and validated noninvasive disease activity index (The Utrecht Activity Index). Inflamm Bowel Dis 21 (2015), 2453–2459.
Hebuterne, X., Lemann, M., Bouhnik, Y., et al. Endoscopic improvement of mucosal lesions in patients with moderate to severe ileocolonic Crohn's disease following treatment with certolizumab pegol. Gut 62 (2013), 201–208.
Vuitton, L., Marteau, P., Sandborn, W.J., et al. IOIBD technical review on endoscopic indices for Crohn's disease clinical trials. Gut 65 (2016), 1447–1455.
Khanna, R., Zou, G., D'Haens, G., et al. Reliability among central readers in the evaluation of endoscopic findings from patients with Crohn's disease. Gut 65 (2016), 1119–1125.
Lott, A., Reiter, J.P., Wilson confidence intervals for binomial proportions with multiple imputation for missing data. Am Stat May, 2018, 1–7, 10.1080/00031305.2018.1473796.
R Core Team. R: a language and environment for statistical computing. 2018, R Foundation for Statistical Computing, Vienna, Austria Available from: https://www.R-project.org.
Von Hippel, P.T., How to impute interactions, squares, and other transformed variables. Sociol Methodol 39 (2009), 265–291.
Van Buuren, S., Groothuis-Oudshoorn, K., MICE: multivariate imputation by chained equations in R. J Stat Softw 45 (2011), 1–67.
Rubin, D.B., Multiple imputation for nonresponse in surveys. 1987, John Wiley and Sons, New York.
Beigel, F., Deml, M., Schnitzler, F., et al. Rate and predictors of mucosal healing in patients with inflammatory bowel disease treated with anti-TNF-alpha antibodies. PLoS One, 9, 2014, e99293.
Bodelier, A.G.L., Jonkers, D., van den Heuvel, T., et al. High percentage of IBD patients with indefinite fecal calprotectin levels: additional value of a combination score. Dig Dis Sci 62 (2017), 465–472.
Cellier, C., Sahmoud, T., Froguel, E., et al. Correlations between clinical activity, endoscopic severity, and biological parameters in colonic or ileocolonic Crohn's disease. A prospective multicentre study of 121 cases. Gut 35 (1994), 231–235.
de Bruyn, M., Arijs, I., De Hertogh, G., et al. Serum neutrophil gelatinase B-associated lipocalin and matrix metalloproteinase-9 complex as a surrogate marker for mucosal healing in patients with Crohn's disease. J Crohns Colitis 9 (2015), 1079–1087.
Faubion, W.A.J., Fletcher, J.G., O'Byrne, S., et al. EMerging BiomARKers in Inflammatory Bowel Disease (EMBARK) study identifies fecal calprotectin, serum MMP9, and serum IL-22 as a novel combination of biomarkers for Crohn's disease activity: role of cross-sectional imaging. Am J Gastroenterol 108 (2013), 1891–1900.
Garcia-Planella, E., Manosa, M., Cabre, E., et al. Fecal calprotectin levels are closely correlated with the absence of relevant mucosal lesions in postoperative Crohn's disease. Inflamm Bowel Dis 22 (2016), 2879–2885.
Herranz Bachiller, M.T., Barrio Andres, J., Fernandez Salazar, L., et al. The utility of faecal calprotectin to predict post-operative recurrence in Crohn's disease. Scand J Gastroenterol 51 (2016), 720–726.
Jones, J., Loftus, E.V.J., Panaccione, R., et al. Relationships between disease activity and serum and fecal biomarkers in patients with Crohn's disease. Clin Gastroenterol Hepatol 6 (2008), 1218–1224.
Klimczak, K., Lykowska-Szuber, L., Eder, P., et al. The diagnostic usefulness of fecal lactoferrin in the assessment of Crohn's disease activity. Eur J Intern Med 26 (2015), 623–627.
Langhorst, J., Elsenbruch, S., Koelzer, J., et al. Noninvasive markers in the assessment of intestinal inflammation in inflammatory bowel diseases: performance of fecal lactoferrin, calprotectin, and PMN-elastase, CRP, and clinical indices. Am J Gastroenterol 103 (2008), 162–169.
Lobaton, T., Lopez-Garcia, A., Rodriguez-Moranta, F., et al. A new rapid test for fecal calprotectin predicts endoscopic remission and postoperative recurrence in Crohn's disease. J Crohns Colitis 7 (2013), e641–e651.
Ma, C., Lumb, R., Walker, E.V., et al. Noninvasive fecal immunochemical testing and fecal calprotectin predict mucosal healing in inflammatory bowel disease: a prospective cohort study. Inflamm Bowel Dis 23 (2017), 1643–1649.
Meuwis, M.-A., Vernier-Massouille, G., Grimaud, J.C., et al. Serum calprotectin as a biomarker for Crohn's disease. J Crohns Colitis 7 (2013), e678–e683.
Nakarai, A., Kato, J., Hiraoka, S., et al. Slight increases in the disease activity index and platelet count imply the presence of active intestinal lesions in C-reactive protein-negative Crohn's disease patients. Intern Med 53 (2014), 1905–1911.
Nancey, S., Boschetti, G., Moussata, D., et al. Neopterin is a novel reliable fecal marker as accurate as calprotectin for predicting endoscopic disease activity in patients with inflammatory bowel diseases. Inflamm Bowel Dis 19 (2013), 1043–1052.
Viscido, A., Corrao, G., Taddei, G., et al. “Crohn's disease activity index” is inaccurate to detect the post-operative recurrence in Crohn's disease. A GISC study. Ital J Gastroenterol Hepatol 31 (1999), 274–279.
Walters, T.D., Steinhart, A.H., Bernstein, C.N., et al. Validating Crohn's disease activity indices for use in assessing postoperative recurrence. Inflamm Bowel Dis 17 (2011), 1547–1556.
Yarur, A.J., Quintero, M.A., Jain, A., et al. Serum amyloid A as a surrogate marker for mucosal and histologic inflammation in patients with Crohn's disease. Inflamm Bowel Dis 23 (2017), 158–164.
Zittan, E., Kabakchiev, B., Kelly, O.B., et al. Development of the Harvey-Bradshaw index-pro (HBI-PRO) score to assess endoscopic disease activity in Crohn's disease. J Crohns Colitis 11 (2017), 543–548.
Rizopoulos, D., Lesaffre, E., Introduction to the special issue on joint modelling techniques. Stat Methods Med Res 23 (2014), 3–10.
Nicolaie, M.A., van Houwelingen, J.C., de Witte, T.M., et al. Dynamic prediction by landmarking in competing risks. Stat Med 32 (2013), 2031–2047.
Kurtz, D.M., Esfahani, M.S., Scherer, F., et al. Dynamic risk profiling using serial tumor biomarkers for personalized outcome prediction. Cell 178 (2019), 699–713.e19.
D'Haens, G., Kelly, O., Battat, R., et al. Development and validation of a test to monitor endoscopic activity in patients with Crohn's disease based on serum levels of proteins. Gastroenterology 158 (2020), 515–526.
Arshad, H.M.S., Bhagatwala, J., Cheema, A., et al. The utility OF Monitr® test as a clinical-decision making tool in Crohn's disease. Gastroenterology, 156, 2019 S-850.
Hamilton, A.L., De Cruz, P., Wright, E.K., et al. Utility of a simple blood test for mucosal healing monitoring is accuracte in post-operative Crohn's disease. Gastroenterology, 156, 2019 S-843.
Collins, G.S., Reitsma, J.B., Altman, D.G., et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 162 (2015), 55–63.
Janssen, K.J.M., Vergouwe, Y., Donders, A.R.T., et al. Dealing with missing predictor values when applying clinical prediction models. Clin Chem 55 (2009), 994–1001.
Bossuyt, P., Louis, E., Mary, J.-Y., et al. Defining endoscopic remission in ileocolonic Crohn's disease: let's start from scratch. J Crohns Colitis 12 (2018), 1245–1248.