III: SMALL: Moving Beyond Knowledge to Action: Evaluating and Improving the Utility of Causal Inference

One of the key recent advances in machine learning is the ability to learn causal structures from observational data. Unlike correlations, causes let us robustly predict the future and identify which variables to intervene on to potentially change it. As a result, many computational methods have been introduced to better discover causes from the large datasets that are increasingly becoming available. However, algorithms for finding causes are mainly evaluated on how accurately they can recover ground truth. This assumes that the most complete and accurate causal model will be the most useful one, but this assumption has not been tested and people often struggle to make sense of complex information. Causal models can also be used to better understand the effects of actions, which could further improve decisions. While current methods identify the effects of turning a variable on or off, this is not the right level of detail for an individual making choices such as a person with diabetes deciding what specific food to consume for breakfast. Further, the users of the output of causal inference are not those developing the methods, but rather people with varying levels of background knowledge and perceived expertise. This project focuses on reducing the gap between machine learning and human decision-making by quantifying the utility of causal models, introducing new methods that make causal models more useful and usable, and leveraging the results to improve everyday decisions around diet and exercise.

This project aims to close the loop from data to knowledge to action, through better metrics for evaluating causal inference, and algorithms that make causal models more useful and personalized. This work will advance our ability to effectively use the output of machine learning, and encourage the development of methods that produce output with high utility. First, this project develops novel ways to automatically evaluate the utility of a set of inferred causes, which allow algorithms to be compared along this new dimension that provides more insight into real-world use. In particular, new metrics are developed that take into account model, user, and context features to allow causal models to be automatically scored on how useful they are for decision-making. Second, the developed metrics are used to guide development of more useful models that accurately predict the effects of interventions and incorporate mechanistic information. A key gap translating causal models to real-world use is the need to predict the result of interventions that may not directly map to variables (e.g. drinking orange juice is not the same as directly increasing glucose). The new methods developed can predict intervention effects using simulation, and map models to mechanistic information to enable further insight. Lastly, the project demonstrates that these enhanced causal models can improve real-life decisions. The project can help make the output of machine learning actionable, and may have applications in many important decision-making scenarios related to health, finance, and personal transportation. The research may more generally improve decision-making, and can be applied to areas as diverse as reducing distracted driving and understanding the impact of choices on energy usage.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.