Projects

Novice Medical Representations using Deep Neural Networks

Modeling Novice Psychological Representations in Medical Domains using Deep Neural Networks

Medical professionals undergo extensive training before categorizing medical images. This training potentially equips them with concepts that alter their psychological representations. How do novices who lack domain expertise perceptually represent medical images? This project compares representations obtained from artificial neural networks to untrained human representations. We focus on the two neural network representations for each image: (i) domain-general machine representations, which have not received training on medical images, and (ii) domain-specific representations trained in specific medical image classification tasks. Using each neural network representation, we calculate the similarity between two images as the Euclidean distance between the internal layers of the network. We probe human representations by eliciting similarity judgments on carefully curated pairs of images. We find that the representations of untrained novices are more like domain-specific representations. This implies that novices use features that discriminate between medical categories with limited exposure to medical images without any explicit knowledge of the medical categories.
Publications
Hasan, E., Duhaime, E. P., & Trueblood, J. (In Prep.). Modeling Novice Psychological Representations in Medical Domains using Deep Neural Networks

Wisdom of the Crowd for Medical Image Annotation

Boosting Wisdom of the Crowd for Medical Image Annotation Using Training Performance and Task Features.

A crucial bottleneck in medical artificial intelligence (AI) is high-quality labeled medical datasets. In this paper, we test a large variety of wisdom of the crowd algorithms to label medical images that were initially classified by individuals recruited through an app-based platform. Individuals classified skin lesions from the International Skin Lesion Challenge 2018 into 7 different categories. There was a large dispersion in the geographical location, experience, training, and performance of the recruited individuals. We tested several wisdom of the crowd algorithms of varying complexity from a simple unweighted average to more complex Bayesian models that account for individual patterns of errors. Using a switchboard analysis, we observe that the best-performing algorithms rely on selecting top performers, weighting decisions by training accuracy, and take into account the task environment. These algorithms far exceed expert performance. We conclude by discussing the implications of these approaches for the development of medical AI.
Publications
Hasan, E., Duhaime, E. P., & Trueblood, J. (2024). Boosting Wisdom of the Crowd for Medical Image Annotation Using Training Performance and Task Features. Cognitive Research Principles and Implications https://doi.org/10.1186/s41235-024-00558-6

Impact Of Presentation Format on Decision Making

A Registered Report on presentation factors that influence the attraction effect.

Context effects occur when the preference between two alternatives is affected by the presence of an extra alternative. These effects are some of the most well studied phenomena in multi-alternative, multi-attribute decision making. Recent research in this area has revealed an intriguing pattern of results. On the one hand, these effects are robust and ubiquitous. That is, they have been demonstrated in many domains and different choice settings. On the other hand, they are fragile and they disappear or even reverse under different conditions. This pattern of results has spurred debate and speculation about the cognitive mechanisms that drive these choices. The attraction effect, where the preference for an option increases in the presence of a dominated decoy, has generated the most controversy. In this registered report, we systematically vary factors that are known to be associated with the attraction effect to build a solid foundation of empirical results to aid future theory development. We find a robust attraction effect across the different conditions. The strength of this effect is modulated by the display order (e.g., decoy top, target middle, competitor bottom) and mode (numeric vs. graphical) but not display layout (by-attribute vs. by-alternative).
Publications
Hasan, E., Liu, Y., Owens, N., Trueblood J. S. (In-Press) A Registered Report on presentation factors that influence the attraction effect. Judgment and Decision Making https://doi.org/10.31234/osf.io/rfcgp

Cognitive Distortions on Social Media

One shot intervention reduces online engagement with distorted content.

Depression is one of the leading causes of disability worldwide. Individuals with depression often experience unrealistic and overly negative thoughts, i.e.~cognitive distortions, that cause maladaptive behaviors and feelings. Now that a majority of the US population uses social media platforms, concerns have been raised that they may serve as a vector for the spread of distorted ideas and thinking amid a global mental health epidemic. Here, we study how individuals (N=838) interact with distorted content on social media platforms using a simulated environment similar to Twitter (now X). We find that individuals with higher depression symptoms tend to prefer distorted content more than those with fewer symptoms. However, a simple one-shot intervention can teach individuals to recognize and drastically reduce interactions with distorted content across the entire depression scale. This suggests that distorted thinking on social media may disproportionally affect individuals with depression, but a simple awareness training can mitigate this effect. Our findings have important implications for understanding the role of social media in propagating distorted thinking and potential paths to reduce the societal cost of mental health disorders.
Publications
Hasan, E., Epping, G. P., Lorenzo-Luaces, L., Bollen, J., & Trueblood, J. S. (2024, Under Review). One shot intervention reduces online engagement with distorted content. https://doi.org/10.31234/osf.io/47cqw

Salience and Decision Making

The Role of Salience in Multialternative Multiattribute Choice.

Attention plays a central role in multi-alternative multiattribute decision-making but the cognitive mechanisms for it are elusive (Yang & Krajbich, 2023; Molter, Thomas, Huet- tel, Heekeren, & Mohr, 2022; Trueblood, 2022). In this project, we explored the role of bottom-up attention by manipulating the salience of different options in a multi-alternative, multi-attribute choice display. Behaviorally, we observed that salience interacts with choice, where the salient option is selected more often, especially in quick decisions. Using computational modeling, we tested two different hypotheses for how salience impacts decision-making for different individuals. We tested (i) if salience created an initial bias in the decision-making process, and (ii) if salience impacted the comparisons that are made during the decision-making process. We find that there are large individual differences in the mechanism through which salience impacts choice. For many individuals, there was no impact of salience. However, for a sizable minority, salience created an initial boost in selecting the salient option. We do not find strong evidence for the impact of salience in the comparison process. In exploratory analyses, we observe that the impact of salience in decision-making is correlated with thinking styles. Our results indicate that salience-driven attention might impact decision-making in different ways for individuals.
Publications
Hasan, E., & Trueblood, J. S. (2024). The Role of Salience in Multialternative Multiattribute Choice. Proceedings of the Annual Meeting of the Cognitive Science Society https://escholarship.org/uc/item/5jq8n5w9

Confidence in Wisdom of the Crowds

Harnessing the wisdom of the confident crowd in medical image decision-making.

Improving the accuracy of medical image interpretation is critical to improving the diagnosis of many diseases. Using both novices (undergraduates) and experts (medical professionals), we investigated methods for improving the accuracy of a single decision maker and a group of decision makers by aggregating repeated decisions in different ways. Participants made classification decisions (cancerous versus non-cancerous) and confidence judgments on a series of cell images, viewing and classifying each image twice. We first examined whether it is possible to improve individual-level performance by using the maximum confidence slating algorithm (Koriat, 2012b), which leverages metacognitive ability by using the most confident response for an image as the ‘final response’. We find maximum confidence slating improves individual classification accuracy for both novices and experts. Building on these results, we show that aggregation algorithms based on confidence weighting scale to larger groups of participants, dramatically improving diagnostic accuracy, with the performance of groups of novices reaching that of individual experts. In sum, we find that repeated decision making and confidence weighting can be a valuable way to improve accuracy in medical image decision-making and that these techniques can be used in conjunction with each other.
Publications
Hasan, E., Eichbaum, Q., Seegmiller, A. C., Stratton, C., & Trueblood, J. S. (2023). Harnessing the wisdom of the confident crowd in medical image decision-making. Decision https://doi.org/10.1037/dec0000210

Metacognition and Similarity to Denoise.

Improving Medical Image Decision‐Making by Leveraging Metacognitive Processes and Representational Similarity.

Improving the accuracy of medical image interpretation can improve the diagnosis of numerous diseases. We compared different approaches to aggregating repeated decisions about medical images to improve the accuracy of a single decision maker. We tested our algorithms on data from both novices (undergraduates) and experts (medical professionals). Participants viewed images of white blood cells and made decisions about whether the cells were cancerous or not. Each image was shown twice to the participants and their corresponding confidence judgments were collected. The maximum confidence slating (MCS) algorithm leverages metacognitive abilities to consider the more confident response in the pair of responses as the more accurate “final response” (Koriat, 2012), and it has previously been shown to improve accuracy on our task for both novices and experts (Hasan et al., 2021). We compared MCS to similarity-based aggregation (SBA) algorithms where the responses made by the same participant on similar images are pooled together to generate the “final response.” We determined similarity by using two different neural networks where one of the networks had been trained on white blood cells and the other had not. We show that SBA improves performance for novices even when the neural network had no specific training on white blood cell images. Using an informative representation (i.e., network trained on white blood cells) allowed one to aggregate over more neighbors and further boosted the performance of novices. However, SBA failed to improve the performance for experts even with the informative representation. This difference in efficacy of the SBA suggests different decision mechanisms for novices and experts.
Publications
Hasan, E., Eichbaum, Q., Seegmiller, A. C., Stratton, C., & Trueblood, J. S. (2022). Improving Medical Image Decision‐Making by Leveraging Metacognitive Processes and Representational Similarity. Topics in Cognitive Science. https://doi.org/10.1111/tops.12588

Representational Smoothing

Representational Smoothing to Improve Medical Image Decision Making.

We demonstrate how medical-image classification decisions can be denoised by aggregating decisions on similar images. In our algorithm, the final decision on a target image is cancerous if a percentage t of the k most similar images are cancerous, else it is not cancerous. Similarity between images is calculated as the distance between representations from an artificial neural network. We vary k and t for novice and expert participants using data from Trueblood et al. (2018) and Trueblood et al. (2021). We show that increasing k improves performance for novices, with their performance approaching that of experts. We also show that the algorithm is biased towards identifying cancerous cells, which is reflected in the representational space. The percentage t allows greater control over sensitivity and specificity and can be used to debias decisions. This algorithm is less effective for experts, partially explained by them giving similar responses on similar images.
Publications
Hasan, E., & Trueblood, J. S. (2022). Representational Smoothing to Improve Medical Image Decision Making. Proceedings of the Annual Meeting of the Cognitive Science Society . https://escholarship.org/uc/item/4p6878mm