Projects
Machine Learning vs Cognitive Modeling
When and which Cognitive Modeling Improves upon the Predictions of Pure Machine Learning Models? The example of decisions from experience
Recent research shows that machine-learning (ML) tools predict human behavior more accurately than prominent cognitive models across diverse settings. This project aims to identify the boundary conditions of this advantage by asking when cognitive modeling enhances predictive performance. It assumes that: (1) individuals rely on a diverse set of decision rules and that switching between rules is infrequent but consequential, and (2) ML methods excel at detecting the current rule, whereas cognitive models better approximate the set of rules people tend to use. Accordingly, we hypothesize that cognitive models are especially effective for predicting unfamiliar individuals in novel contexts and the long-run behavior of familiar individuals, but less effective for predicting the immediate next choice of a familiar individual. We will test these hypotheses through a three-track open choice-prediction competition, evaluating the added value of cognitive modeling by comparing top-performing submissions.
Publications
Erev, I., Plonsky, O., Roth, Y, Hasan, E. (2026) When and which Cognitive Modeling Improves upon the Predictions of Pure Machine Learning Models? The example of decisions from experience Under Review at Nature Human Behavior
Representational Modeling
Representational Inconsistency. Understanding when to re-evaluate decisions
Decision errors are often classified as noise or bias, yet little guidance exists for identifying the source of a specific decision error. We propose that this distinction can be inferred from the local structure of an observer’s decision space. Responses to similar stimuli reveal a latent decision space—a probabilistic mapping from representational structure to categorical decisions—against which individual decisions can be evaluated. An error is representationally inconsistent when the observed response is unlikely given the observer’s previous decisions on similar stimuli, reflecting a noise-like departure from the observer’s own decision space. In contrast, representationally consistent errors reflect stable, bias-like regularities within that structure. We formalize this framework using similarity-based models of categorization. Representational structure is operationalized using deep neural network embeddings, and an individual’s decision space is estimated by weighting prior decisions according to representational similarity. The framework predicts that representationally inconsistent errors should be more likely to be corrected through re-evaluation, whereas representationally consistent errors should tend to persist. We demonstrate these predictions in medical image classification tasks involving novice and expert decision makers. Building on this principle, we derive aggregation and selective re-sampling algorithms that reduce error by targeting locally inconsistent judgments and show how the framework can guide whether repeated evaluation or second opinions are more likely to improve decisions. These findings establish representational inconsistency as a computational marker of correctable error, integrating similarity-based theories of categorization with a broader account of judgment revision, decision aggregation, and the sources of human error.
Publications
Hasan, E., & Trueblood, J. S. (2022). Representational Smoothing to Improve Medical Image Decision Making. Proceedings of the Annual Meeting of the Cognitive Science Society . https://escholarship.org/uc/item/4p6878mm
Hasan, E., & Trueblood, J. (2026). Representational Inconsistency: Understanding When to Re-evaluate Decisions. Under Review at Psychological Review link
Natural and Artificial Intelligence Representations
Convergence of Representational Geometries in Natural and Artificial Intelligence in Medical Domains
How can we determine whether natural and artificial intelligences represent the world in similar ways, and what would such similarity imply for our understanding of human decision making? It may be impossible to establish whether the percept of the same stimulus, for example, the color red, is identical across two minds. But we can ask whether the relations among percepts are preserved: whether, for instance, both minds represent red as more similar to orange than to blue. This notion of second-order isomorphism provides a way to compare representations across fundamentally different systems and connects to a growing literature on representational alignment between humans and artificial systems. We use this idea to ask whether humans and artificial intelligence systems converge on similar representational geometries for specialized medical images. We estimate these representations using multidimensional scaling and align them using Procrustes transformations to interpretable, verbalizable dimensions. We find that natural and artificial systems often converge on similar representations
Publications
Hasan, E., Trueblood, J., Turner, B. & Nosofsky, R. (In Prep.). Convergence of Representational Geometries in Natural and Artificial Intelligence in Medical Domains
Wisdom of the Crowd for Medical Image Annotation
Boosting Wisdom of the Crowd for Medical Image Annotation Using Training Performance and Task Features.
A crucial bottleneck in medical artificial intelligence (AI) is high-quality labeled medical datasets. In this paper, we test a large variety of wisdom of the crowd algorithms to label medical images that were initially classified by individuals recruited through an app-based platform. Individuals classified skin lesions from the International Skin Lesion Challenge 2018 into 7 different categories. There was a large dispersion in the geographical location, experience, training, and performance of the recruited individuals. We tested several wisdom of the crowd algorithms of varying complexity from a simple unweighted average to more complex Bayesian models that account for individual patterns of errors. Using a switchboard analysis, we observe that the best-performing algorithms rely on selecting top performers, weighting decisions by training accuracy, and take into account the task environment. These algorithms far exceed expert performance. We conclude by discussing the implications of these approaches for the development of medical AI.
Publications
Hasan, E., Duhaime, E. P., & Trueblood, J. (2024). Boosting Wisdom of the Crowd for Medical Image Annotation Using Training Performance and Task Features. Cognitive Research Principles and Implications https://doi.org/10.1186/s41235-024-00558-6
Impact Of Presentation Format on Decision Making
A Registered Report on presentation factors that influence the attraction effect.
Context effects occur when the preference between two alternatives is affected by the presence of an extra alternative. These effects are some of the most well studied phenomena in multi-alternative, multi-attribute decision making. Recent research in this area has revealed an intriguing pattern of results. On the one hand, these effects are robust and ubiquitous. That is, they have been demonstrated in many domains and different choice settings. On the other hand, they are fragile and they disappear or even reverse under different conditions. This pattern of results has spurred debate and speculation about the cognitive mechanisms that drive these choices. The attraction effect, where the preference for an option increases in the presence of a dominated decoy, has generated the most controversy. In this registered report, we systematically vary factors that are known to be associated with the attraction effect to build a solid foundation of empirical results to aid future theory development. We find a robust attraction effect across the different conditions. The strength of this effect is modulated by the display order (e.g., decoy top, target middle, competitor bottom) and mode (numeric vs. graphical) but not display layout (by-attribute vs. by-alternative).
Publications
Hasan, E., Liu, Y., Owens, N., Trueblood J. S. (2025) A Registered Report on presentation factors that influence the attraction effect. Judgment and Decision Making https://doi.org/10.1017/jdm.2024.27
Cognitive Distortions on Social Media
One shot intervention reduces online engagement with distorted content.
Depression is one of the leading causes of disability worldwide. Individuals with depression often experience unrealistic and overly negative thoughts, i.e.~cognitive distortions, that cause maladaptive behaviors and feelings. Now that a majority of the US population uses social media platforms, concerns have been raised that they may serve as a vector for the spread of distorted ideas and thinking amid a global mental health epidemic. Here, we study how individuals (N=838) interact with distorted content on social media platforms using a simulated environment similar to Twitter (now X). We find that individuals with higher depression symptoms tend to prefer distorted content more than those with fewer symptoms. However, a simple one-shot intervention can teach individuals to recognize and drastically reduce interactions with distorted content across the entire depression scale. This suggests that distorted thinking on social media may disproportionally affect individuals with depression, but a simple awareness training can mitigate this effect. Our findings have important implications for understanding the role of social media in propagating distorted thinking and potential paths to reduce the societal cost of mental health disorders.
Publications
Hasan, E., Epping, G. P., Lorenzo-Luaces, L., Bollen, J., & Trueblood, J. S. (2025, In-Press). One shot intervention reduces online engagement with distorted content. PNAS NExys https://doi.org/10.31234/osf.io/47cqw
Salience and Decision Making
The Role of Salience in Multialternative Multiattribute Choice.
Attention plays a central role in multi-alternative multiattribute decision-making but the cognitive mechanisms for it are elusive (Yang & Krajbich, 2023; Molter, Thomas, Huet- tel, Heekeren, & Mohr, 2022; Trueblood, 2022). In this project, we explored the role of bottom-up attention by manipulating the salience of different options in a multi-alternative, multi-attribute choice display. Behaviorally, we observed that salience interacts with choice, where the salient option is selected more often, especially in quick decisions. Using computational modeling, we tested two different hypotheses for how salience impacts decision-making for different individuals. We tested (i) if salience created an initial bias in the decision-making process, and (ii) if salience impacted the comparisons that are made during the decision-making process. We find that there are large individual differences in the mechanism through which salience impacts choice. For many individuals, there was no impact of salience. However, for a sizable minority, salience created an initial boost in selecting the salient option. We do not find strong evidence for the impact of salience in the comparison process. In exploratory analyses, we observe that the impact of salience in decision-making is correlated with thinking styles. Our results indicate that salience-driven attention might impact decision-making in different ways for individuals.
Publications
Hasan, E., & Trueblood, J. S. (2024). The Role of Salience in Multialternative Multiattribute Choice. Proceedings of the Annual Meeting of the Cognitive Science Society https://escholarship.org/uc/item/5jq8n5w9
Confidence in Wisdom of the Crowds
Harnessing the wisdom of the confident crowd in medical image decision-making.
Improving the accuracy of medical image interpretation is critical to improving the diagnosis of many diseases. Using both novices (undergraduates) and experts (medical professionals), we investigated methods for improving the accuracy of a single decision maker and a group of decision makers by aggregating repeated decisions in different ways. Participants made classification decisions (cancerous versus non-cancerous) and confidence judgments on a series of cell images, viewing and classifying each image twice. We first examined whether it is possible to improve individual-level performance by using the maximum confidence slating algorithm (Koriat, 2012b), which leverages metacognitive ability by using the most confident response for an image as the ‘final response’. We find maximum confidence slating improves individual classification accuracy for both novices and experts. Building on these results, we show that aggregation algorithms based on confidence weighting scale to larger groups of participants, dramatically improving diagnostic accuracy, with the performance of groups of novices reaching that of individual experts. In sum, we find that repeated decision making and confidence weighting can be a valuable way to improve accuracy in medical image decision-making and that these techniques can be used in conjunction with each other.
Publications
Hasan, E., Eichbaum, Q., Seegmiller, A. C., Stratton, C., & Trueblood, J. S. (2023). Harnessing the wisdom of the confident crowd in medical image decision-making. Decision https://doi.org/10.1037/dec0000210
Metacognition and Similarity to Denoise.
Improving Medical Image Decision‐Making by Leveraging Metacognitive Processes and Representational Similarity.
Improving the accuracy of medical image interpretation can improve the diagnosis of numerous diseases. We compared different approaches to aggregating repeated decisions about medical images to improve the accuracy of a single decision maker. We tested our algorithms on data from both novices (undergraduates) and experts (medical professionals). Participants viewed images of white blood cells and made decisions about whether the cells were cancerous or not. Each image was shown twice to the participants and their corresponding confidence judgments were collected. The maximum confidence slating (MCS) algorithm leverages metacognitive abilities to consider the more confident response in the pair of responses as the more accurate “final response” (Koriat, 2012), and it has previously been shown to improve accuracy on our task for both novices and experts (Hasan et al., 2021). We compared MCS to similarity-based aggregation (SBA) algorithms where the responses made by the same participant on similar images are pooled together to generate the “final response.” We determined similarity by using two different neural networks where one of the networks had been trained on white blood cells and the other had not. We show that SBA improves performance for novices even when the neural network had no specific training on white blood cell images. Using an informative representation (i.e., network trained on white blood cells) allowed one to aggregate over more neighbors and further boosted the performance of novices. However, SBA failed to improve the performance for experts even with the informative representation. This difference in efficacy of the SBA suggests different decision mechanisms for novices and experts.
Publications
Hasan, E., Eichbaum, Q., Seegmiller, A. C., Stratton, C., & Trueblood, J. S. (2022). Improving Medical Image Decision‐Making by Leveraging Metacognitive Processes and Representational Similarity. Topics in Cognitive Science. https://doi.org/10.1111/tops.12588