Intentional State Machines: An Information Theoretic Approach to Conscious Purpose Martin Hilbert (DE Computational Social Science, Communication, GG Computer Science, University of California, Davis, CA ) C2
The notion of purpose and intentionality is intrinsically connected to the consciousness. This goes beyond more abstract discussions of free will, agency and representation, but is common daily practice for meditators, psychonauts, and mindfulness practitioners. The role of intentionality becomes ever more important in an economy and society increasingly influenced by artificial intelligence (AI), which, currently, is mainly dedicated to making predictions based on the past (Agrawal et al., 2018). This leads to a dynamic which perpetuates also undesired past patterns (Hilbert, 2014), including political intolerance and discriminatory algorithmic biases (Hajian et al., 2016; Hilbert et al., 2018; O'Neil, 2017; Pariser, 2011; Pasquale, 2015). AI leaders like Microsoft, agree that this poses a serious problem: "Ultimately the question is not only what computers can do. It's what computers should do?" (Shaw, 2019). At the same time, the concept of intentionality is remarkably missing from any formal and mathematical approach to consciousness. In this study, we propose a framework to conceptualize this notion formally by combining a successful approach from information theory, namely the information bottleneck approach (Tishby et al., 2000) with a fruitful expansion of information with theoretical computer science, namely computational mechanics (Crutchfield, 2012, 2017; Crutchfield & Young, 1989). Computational mechanics provides an optimized model for past-future predictions. The so-called epsilon-machine of a dynamic, which can be obtained from a series of machine learning techniques, represents the smallest size, optimally predictive, unifilar (deterministic) hidden Markov model of the dynamic (Shalizi & Crutchfield, 2001). The result represents the optimal way to get from the past to the future, which is why they are also known as "predictive state machines" (Hilbert & Darmon, 2019). The information bottleneck method finds the best tradeoff between accuracy and complexity when compressing a random variable (Slonim, 2002; Tishby et al., 2000). It is based on Shannon's original rate distortion theory (Shannon, 1948), which uses a distortion function that measures how well some kind of goal state, is predicted from a compressed representation of some other variable. It optimizes between relevant and irrelevant distinctions, and keeps only those that are relevant for a specific task. In this study, we use the information bottleneck method to capture the notion of intention and computational mechanics to derive the analytical representation that most effectively converts the past dynamic into the intended dynamic. The resulting "intentional state machines" represents the smallest size, optimally predictive, unifilar hidden Markov model that transitions from a given past, to a desired future. In contrast to the traditional information bottleneck method, our result is not a rather intangible mutual information (a joint distribution, which cannot stand by itself per se, (Crutchfield et al., 2009)), but a tangible representation (aka 'machine') that allows theorists to analytically reason about transitioning from a current past pattern to the intended pattern. This method has a myriad of applications in the field of consciousness that focus on transitioning from past patterns to intended new ones (be it cognitive, psychological, social, or technological).