A big data analytics framework for forecasting rare customer complaints: A use case of predicting MA members' complaints to CMS

Abstract

Centers for Medicare & Medicaid Services (CMS) publishes Medicare Part C Star Ratings each year to measure the quality of care of Medicare Advantage (MA) contracts. One of the key measures is Complaints about the Health Plan, which is captured in Complaints Tracking Module (CTM). Complaints resulted in CTM are rare events: for MA contracts with 2–5 star ratings, number of complaints for every 1,000 members range from .10 to 1.84 over last 5 years. Reducing number of complaints is extremely important to MA plans as they impact CMS reimbursements to MA plans. Forecasting and reducing complaints is an extremely technically challenging task, and involves ethics considerations in patients' rights and privacy. In this research, we constructed a big data analytics framework for forecasting rare customer complaints. First, we built a big data ingestion pipelines on a Hadoop platform: a) Ingest MA plan's customer complaints data from CTM from past 3 years. b) Ingest health plan's call center data for MA members from past 3 years, including both structured data and unstructured text script for the calls. c) Ingest MA members' medical claims, including members' demographics and enrollment history. d) Ingest MA members' pharmacy claims. e) Integrate and unified data from above sources, and enrich the data with additional engineered features into a big wide table, one row per member for analysis and modeling. Second, we designed a unique decision tree based Large Ensemble with Over-Sampling (LEOS) algorithm, which mimics random forest but with extreme oversampling of target class to increase bias, and leverages the parallel computing of Hadoop clusters by generating thousands of fixed size training data sets, and for each such dataset training a decision trees with similar fixed tree structure, and ensemble them. Third, we validated our framework and LEOS learning algorithm with real data, and also discussed ethics issues we encountered in handling data and applying findings from research.