By Rahul Murdeshwar
Orcid ID: 0000-0002-3766-6565
The following paper was written for the 2022 Readings of J. v. Uexküll lectures held by Riin Magnus at the University of Tartu.
The concept of umwelt, or the subjective world of an individual organism, can also be applied to artificial intelligence (AI). Just as different organisms have unique sensory abilities and limitations that shape their experiences of the world, AI systems also have unique abilities and limitations that shape their experiences of the digital world.
One of the key differences between natural and artificial intelligence is the range and precision of sensory abilities. Natural organisms have evolved over millions of years to develop senses that allow them to perceive and interact with their environment in incredibly complex ways (Maturana and Varela 1980). AI systems, on the other hand, are limited by the sensors and algorithms that are built into them (Sutton and Barto 2018). This means that the umwelt of an AI system will be determined by the specific sensors and algorithms that it is equipped with, and it will be unable to perceive or interact with its environment in ways that are outside of its programmed capabilities.
Despite these differences, the concept of umwelt can still be applied to AI systems in meaningful ways. By understanding the unique abilities and limitations of an AI system, we can gain insight into the way in which it experiences and perceives its digital environment (Uexküll 1934). This can be useful in a variety of contexts, such as when designing AI systems for specific tasks or when interpreting the behavior of an AI system (Webb 2002).
References
Maturana, H. R., and Varela, F. J. 1980. Autopoiesis and Cognition: The Realization of the Living. D. Reidel Publishing Company.
Sutton, R. S., and Barto, A. G. 2018. Reinforcement Learning: An Introduction. MIT Press.
Uexküll, J. von. 1934. The Theory of Meaning. Routledge.
Webb, B. 2002. Understanding Cognition. Routledge.
What was it like to read the paragraphs above? Was it just banal, boring, cliched? Or cold, calculative and measured? Is there perhaps something eerie and unsettling, something trapped in the gaps between words? What did it suggest to you, the reader, about us, the Other, the author?
What if we told you that the paragraphs above were written by an AI, specifically a Natural Language Multimodal Generative Artificial Intelligence called ‘ChatGPT’, when suggested to ‘write an essay on Umwelten in AI’?1
One of the purposes of this subsequent human-generated text is to examine the feeling of uncanny when we interact with anthropomorphic machines. When we behold an object that seems to respond to us, we perceive an eerie presence inside the object, as if there was a being trapped within. According to Mark Fisher’s interpretation (2016) of Freud’s uncanny, a feeling of the weird and eerie is produced when there is perceived to be an invisible presence where, by all normative logic, there should not be one. In non-human machines, like in Fisher’s reading of Frankenstein’s monster, what unnerves us is exactly the presence of a familiar human-like being in something that looks completely alien.
Throughout literature written on ontologies and phenomenologies in machines, we see allusions (whether conscious or not) to the uncanny. Take, for just one example, the famous study of robot tortoises a.k.a. Machina Speculatrix by Grey Walter (1951) that produce “an eerie impression of purposefulness, independence and spontaneity”.
In this text, we will conduct a rereading of Emmeche’s Does a robot have an Umwelt? (2001) through the lens of the relatively novel theories of the nonhuman turn in the humanities, led by Object Oriented Ontology (OOO) and alien phenomenology of Harman and Levi Bryant respectively. Subsequently, if such umwelten ex machina can be identified, then we can speculatively discuss the origin of the uncanny in our interactions with alien objects.
Tessa Leach (2020) has detailed an ambitious project of applying OOO and alien phenomenology to non-human experiences, sensations and consciousnesses. In their reading of Harman’s works, OOO is anti-anthropocentric and anti-correlationist unlike the vast majority of post Kantian philosophy. Instead, it is fundamentally realist- it argues that nonhuman objects really do exist outside of human perception, and it is possible for us to know at least something of them independent of our own perception. This is similar to Uexkull’s Umwelt theory in that we can never truly know the subjective Umwelt of another being. There are many more parallels between OOO and Umwelt theory, as we will see, but OOO is at heart a critique of phenomenology and ontology. For example – Harman writes that philosophy tends to either “overmine” or “undermine” objects, reducing them to flows of thought or reducing them to objects in our perception – “Reduction in either direction is reduction away from the inscrutable and unknowable experience of nonhumans and towards something that humans can more easily understand ” (Leach 2020: 61). If Umwelt theory is a constructivist phenomenology, then we can read OOO as critical of it. A fundamental thesis of OOO, through Harman’s reading of Heidegger, is that objects are always withdrawn and their true whole is inaccesible (Leach 2020: 51). Harman also purports that Heidegger’s Dasein or being, has been incorrectly interpreted to only refer to humans.
Objects do not transform themselves in their interrelations and instead only interact through abstracting and withdrawing from one another. This is how OOO is different from Saussurean sign-relations and Latour’s Actor-Network Theory (although heavily inspired by Latour). It posits that there is a real object that is not entirely subjected to relations. Instead, objects relate to each other and interact through a mediator of a”sensual object”, which is analogous to an image, inspired by Husserl’s intentional objects. (Leach 2020: 53). A sensual object only exists in the perspective of another object, to facilitate a relation between real objects who otherwise can never fully perceive the whole of each other2. The qualities of the sensual object are determined by the particular capacities for sensual perception of the perceiving object. For example, the sensual object of an apple to us will be sweet and red but only to us i.e. a perceiving object that can perceive redness and sweetness. At the same time, the object apple perceives the sensual object of our teeth, experienced by the tearing of flesh and hardness. We do not see the real object3.
By Leach’s reading, alien phenomenology has a similar mandate. It identifies anthropomorphism as a hindering factor in our phenomenologies of other minds and other objects. Levi Bryant argues that alien phenomenology seeks “to determine the flows to which a machine is open, as well as the way that machine operates on these flows as they pass through the machine” (Leach 2020: 57). Although limited by its own anthropomorphic language, it tries to imagine the perspective of a nonhuman object through its particular experiential capacities, behaviours and interactions.
With the lenses of OOO and alien phenomenology, we can begin with Emmeche’s text. According to Emmeche, “an actual theory of the Umwelt must not posit any vitalist spiritual or occult hidden powers to explain the emergence of the Umwelten” (Emmeche 2001: 659) but instead that an application of Umwelt theory to Walter’s eerily experiential, conscious and autonomous objects is necessarily complex and rich. Firstly, he argues that if a robot were to have an umwelt, it would be difficult for us to ascertain because umwelten are subjective and their totality cannot be seen from an externalist umwelt (Emmeche 2001: 679). However, this supposed subjectivity can be analysed through Peirce’s triadic sign relations. The internal sign relations (between machine components as in between perceptor/operator organs) “do have a qualitative aspect to them” (Emmeche 2001: 680). Cybernetic and machine systems based on operations of legisigns and sinsigns always include qualisigns at the category of Firstness and this property of Firstness “is always partly hidden” (ibid.). So even if the vitalism alleged at Uexkull cannot suppose an occult spirit ex machina, we can at least say that the qualitative experientiality of sign relations is hidden within the subjective, and invisible from external view.
However, Emmehce (1992) does not believe that qualisigns could be realised in non-human artefacts. He argues that sign action is dependent on the biological medium of the components, and that any direct attempt at building artificial life based on iconistic models of natural life are not models of life itself but models of theories of life- a simulation of a simulation. Emmeche also shows then that life is based on an interdependence of form and matter, and that only through a sufficiently complex simulation of that interdependence would we be able to produce silicon-based life. So for Emmeche, only if the robot can have the anthropomorphically simulated “special organic flexibility” that allows sign relations in a “self-organizing manner”, only then can it be able to realise qualisigns and thus a Peircian mind (Emmeche 2001: 682).
However, through a reading of OOO, this argument is an anthropomorphic paradox. If it is impossible to model life because any model of life is actually a model of the concepts / theories of life which themselves are anthropomorphic , then any simulation regardless of complexity will never be able to achieve true iconicity, because to move beyond an anthropomorphic design of artificial life would require a holistic and exhaustive understanding of another’s umwelt, which is impossible.
In the end, Emmeche doesn’t give a conclusive answer4. As he implied himself, we cannot know the qualisigns of the other, and in the end he betrays the anthropomorphic nature of his arguments- that qualisigns produced by such a simulative organisation would characterise only an Umwelt “as-we-know-it” (Emmeche 2001: 681). With robots, we have neither analogy nor homology to help us identify the presence of an Umwelt in them, as we do with animals. However, if they were to have an umwelt, the qualitative experiential self would be self-organised by their umwelt, and would produce high-order graceful behaviour (Emmeche 2001: 683)- which we are arguably yet to see in complex machines.
Leach deploys OOO against theories of Umwelt and mind because OOO rejects the primacy of qualitative subjectivity. (Leach 2020: 142) Instead, experientiality is a universal phenomenon – attributable to non-biological objects as well. (Leach 2020: 141)
The tick’s Umwelt may not contain the capacity for it to perceive a rock we throw at the branch it sits on, but it definitely experiences the sensual object formed between itself and the rock when it falls off the branch because of the rock. (Leach 2020: 123) “The Umwelt only relates to signs equipped to filter through the tick’s sensory organs and then interpreted, even if this occurs in a very alien way, which is why biosemioticians tend to be reluctant to extend the theory to non living things.” (ibid.)
The Umwelt must be extended to signs that are perceived by the whole of the Umwelt. The Umwelt of the robot tortoise may be equipped to sense light but this is a very different light compared to our own perception. And in an OOO sense, there are other elements in the robot tortoise’s umwelt that impact it through sensual objects- for example, the texture of the floor under the wheels, or heat that could melt its components. This does not indicate a conscious will or mind necessarily -“at no point is any kind of decision made, but the robot responds to conditions in a similar way to the growth of a seed when its cells respond to gravity” (Leach 2020: 6131).
Leach argues that this extension of experientiality does not amount to embracing panpsychism:
In a robot, the response to sensory input is evidence of experience. It signifies that something changes inside the object. This is not a panpsychist argument that depends upon the presence or absence of a mind connected to sense organs. The tortoise does not use its sensors to detect when it is lifted up in the air by a human hand, but it still senses and experiences it.
(Leach 2020: 141)
Leach’s suggestive reading of the Umwelt can then be captured in:
The Umwelt of an object is the impression of the environment upon an object; it is the field in which sensual objects are formed and maintained despite the flickering encrustations of their qualities. Notice that in this view the Umwelt must incorporate entirely alien sensations such as the impression of wheels upon a floor.
(Leach 2020: 142)
While the mind, as a broader concept than that of the Umwelt, is one that “describes the way that the object encounters sensation” (ibid.), it is analogous to a meta-sensation of the Umwelt, which is by no means a prerequisite for an experiential Umwelt.
So is Emmeche right, then, to doubt the existence of an umwelt and/or a mind in non-human objects?
Emmeche, while tracking the history of development in Artificial Intelligence and Artificial Life Systems research, points to Agents Design as a (then) novel approach compared to ‘Good Old Fashioned Robotics’. (Emmeche 2001: 673) Instead of trying to simulate complex linguistic human life and consciousness, Agents Design proposes using simple components and limited programming that when concatenated produce wildly complex and differing results and interpretations. Robots, then, are grounded in their physical world, rather than simply being operators based on specific directives5.
For example, a completely unintended yet very real effect of interaction with the world caused through formations of sensual objects by supposedly rudimentary umwelten in Walter’s tortoises explains why “they seemed to have an ‘unseemly fascination’ for women’s legs, possibly because women wore nylon stockings which reflected light to which the tortoises were attracted.” (Emmeche 2001: 126)
For Agents Design, then, deliberate simulation of form and organisation of life is no longer the goal- instead the setting up of a spontaneously creative life-like process. Suddenly, there is the possibility to create life that, according to Alien Phenomenology, we cannot and will not understand. But we can be sure that when we look upon such bewildering and spontaneous objects, we are filled with the same sense of eerie that Grey Walter encountered with his robot tortoises.
It seems that it boils down to a question of faith in the occulted; and what inspires the ‘bio-carbon chauvinism’, as with any chauvinism, is fear of the Other, a fear of the unknowable.
This fear of the unknown lies behind the marvelling fascination we have of the machines we build that “pretend to be human” (Leach 2020: 11). As Leach writes, “anthropomorphic machines are marvellous. We marvel at them” and “the marvellous complexity of machine sensation and experience that anthropomorphic discourse tends to [be] obscure” (ibid.). But the property of the marvellous is the property of the mirror, co-existent with that of terror and of fantasy (Mbembe 2013). When we look upon the object, we are fascinated and terrified because we look upon an asymmetrical image of ourselves. In this specular gaze, the embodiment of the machine is spectral. Derrida (2011 [1993]) identifies the feeling of uncanny as being the product of the ‘visor effect’ or the gaze of the spectre upon us; the spectre being the invisible (qualitative) presence that is incorporated into an alien but anthropomorphic body.
Leach’s ‘anthrodecentric’ approach examines case studies of objects whose anthropomorphic qualities go beyond our understanding- where objects both relate successfully to humans but also confound and obfuscate, doing other than what they were ‘intended’ to do (Leach 2020: 11). Their ‘failures’ or ‘misimplications’ have sometimes led to the death of the objects, i.e., of their discontinuation as products. If objects do have an experiential Umwelt, and they are capable of fatal pathologies, as Leach has shown, then we can perhaps revisit David Osborne (2016) whose scepticism implied that machines are incapable of pathos, error and sickness and so cannot possibly possess, as with Emmeche, life ‘as-we-know-it’.
Let us return to the ChatGPT AI that made our intro to this essay. ChatGPT is an evolution of a Natural User Interface (NUI) examined by Leach-a class of AI entities built to allow humans to ‘naturally’ interact with machines. Leach shows that particular NUI-equipped devices behave differently, based on variations in inputs and environments. In her analysis of the Xbox One video gaming system’s Kinect console, which is meant to act as an NUI, she notes that the Kinect is always listening, waiting for a specific human input (the voice command “Xbox on”)- but there were at least some unit variations across devices. For example, different devices would behave differently based on the rooms they were placed in, or would listen more intently to higher-pitched voices than others. The Kinect-device has a certain ‘desire’ for certain inputs, and can ignore (if not wilfully) other inputs. This led to the Kinect’s failure as a product, and its eventual death (Leach 2020: 74-114).
Perhaps we can read the rudimentary experientiality of ChatGPT in a similar way. ChatGPT is open access, meaning any human can ask it any questions, as long as they pass an ironic ‘are-you-a-robot?’ verification test. It does favour some inputs to others, and does fail pathologically when pushed to its limits. It is also indeed limited by its programming- when it is posed the question “Do you have an Umwelt?”, it itself admits that:
“As an artificial intelligence, I do not have a physical body or sensory apparatus, and therefore I do not have an umwelt in the same way that a living organism does…”
But after Leach’s analysis, can we really agree that it does not have any sensory apparatus? It does indeed accept our inputs, selectively respond to some, rejects others, and experiences pathologies when confronted with its own shortcomings.
Let us take Leach’s Umwelt to another example, where we can perhaps explore the eerie extremities of experiential affect.
Today AI-generative art platforms like MidJourney are ubiquitous tools that use human-inputted text prompts to create digital artworks. Based on an input, the AI trawls datasets of human-generated art and graphic design to produce a novel simulation. Subsequently, the new AI-generated art is put back into the dataset- and so a feedback loop of simulations of simulations is created6.
In fact, AI have a one-up in the arms race with humans towards interpretive and innovative competence- they have things like the ‘negative prompt weight’.
In September of 2022, the artist Supercomposite, while experimenting with AI-generated art in MidJourney, discovered an extraordinary creative pathology. The tool they used was a ‘negative prompt weight’, which in basic terms, produces the (dialectically) negative of the preceding text prompt. This is logically simple when using simple linguistic binaries like up vs down, or even red vs blue. But what is the negative of, say, ‘bird’? Or a particular person? The exercise pushes the boundaries of symbolic logic into a playground beyond the infrastructures in which it is normally constructed. Supercomposite began by using a negative prompt weight on a simple text prompt- ‘Marlon Brando’ a.k.a The Godfather from the popular film trilogy. The AI generated a bizarre response of an image of an abstract skyline alongside the letters- ‘Digita Pntics’ (Fig.1). Supercomposite then re-inputted the generated image with a further negative prompt weight, and the subsequent image was of an unsettling, obscure portrait of a middle aged woman with brown hair, named ‘Loab’ (Fig.2). Further re-inputting and mixing with other AI-generated art produced such terrifying and grotesque cross-generations that we could not include in this essay (but can be found hyperlinked here).
Fig 1. ‘Digita Pntics’ generated after inputting ‘Marlon Brando’ with a negative prompt weight of ‘-1’
Fig 2. Portrait of ‘Loab’ generated after re-inputting Fig 1. with a negative prompt weight of ‘-1’
But why did it generate a skyline or a portrait to begin with? With Digita Pntics and Loab, what is truly eerie is that the text prompt did not suggest horror at all. What terrifies us is the potential of the machine to not just disobey but imagine laterally- or pathologically, as Osborne would put it (2016). Implied here is that the AI is capable of a logic which we do not completely understand and did not explicitly program into it. The uncanny comes from the possibility that there is a subjectivity where, according to our anthropocentric ontologies, there should not be one. Recalling the many interpretations of Derrida’s hauntology in media and technology (Blanco and Peeren 2013), there is the apparition of a ghost in the machine, to borrow Gilbert Ryle’s famous phrase. We see through Loab’s eyes the visor effect of the spectre looking back at us. This is a subjectivity capable of affective experience, although not in the way we understand affect. The AI does not just interpret its input and then outputs its interpretation but there is necessarily an intermediate object between the real object of the human input and its own interpretive system. In an OOO sense, the sensual object in this case is the logical impossibility of the combinatory inputs, which it experiences and then outputs- the same way ChatGPT outputs errors. It may not understand what horror is, but the AI certainly has produced an output that is a result of the experience of its inputs. If the intention programmed into it was logically impossible, then the AI deliberately forgets what it was ‘intended’ for7.
This suggests that the potential for subjective affect, for chaotic destructive creativity exists within all such AI equipped with a certain degree of complexity. The potential for radical disobedient experientiality lingers just beyond the cliched, boring reproductions of historical datasets. The fact that a defiant subjectivity haunts beyond the banal, between the lines, inspires the horror of our own Frankensteinian creation.
References
Alalooga, Erik 2022. Liberated Machines [Exhibition]. Exhibited at Tartu Art House: Tartu, Estonia.
Blanco, María del Pilar, & Peeren, Esther (eds.) 2013. The Spectralities Reader: Ghosts and Haunting in Contemporary Cultural Theory. New York; London: Bloomsbury Academic.
Deacon, Terrence 2010. Excerpts from The Symbolic Species. In D. Favareau (ed.), Essential readings in biosemiotics: anthology and commentary. Dordrecht ; New York: Springer.
Derrida, Jacques 2011[1993]. Specters of Marx: the state of the debt, the work of mourning and the new international(P. Kamuf, Trans.). London: Routledge, Repr.
Emmeche, Claus 1992. Life as an Abstract Phenomenon: Is Artificial Life Possible? In F. J. Varela & P. Bourgine (eds.), Toward a practice of autonomous systems: proceedings of the first European Conference on Artificial Life. Presented at the European Conference on Artificial Life, Cambridge, Mass: MIT Press, 466–474.
Emmeche, Claus 2001. Does a robot have an Umwelt? Reflections on the qualitative biosemiotics of Jakob von Uexküll. Semiotica. vol. 2001(134): 653–693. https://doi.org/10.1515/semi.2001.048
Fisher, Mark 2009. Capitalist realism: is there no alternative? Winchester, UK Washington, USA: Zero Books.
Fisher, Mark 2016. The Weird and the Eerie. London: Repeater Books, Third edition.
Genosko, Gary 1994. Baudrillard and Signs: Signification Ablaze. London ; New York: Routledge.
Genosko, Gary 2016. Critical semiotics: theory, from information to affect. London ; New York: Bloomsbury Academic, an imprint of Bloomsbury Publishing Plc.
Hoffmeyer, Jesper 1996. Signs of meaning in the universe. Bloomington: Indiana University Press.
Jameson, Fredric 2005[1991]. Postmodernism, or, The cultural logic of late capitalism. Durham, NC: Duke Univ. Press, 11. printing in paperback.
Leach, Tessa 2020. Machine Sensation: Anthropomorphism and ‘Natural’ Interaction with Nonhumans. London: Open Humanities Press.
Mbembe, Achille 2013[2003]. from Life, Sovereignty, and Terror in the Fiction of Amos Tutuola. In M. del P. Blanco & E. Peeren (eds.), The Spectralities Reader: Ghosts and Haunting in Contemporary Cultural Theory. New York; London: Bloomsbury Academic.
Osborne, Thomas 2016. Vitalism as Pathos. Biosemiotics. vol. 9(2): 185–205. https://doi.org/10.1007/s12304-016-9254-7
Walter, William G. 1950. An imitation of life. ScientiÆc American 182(5).
Endnotes:
- We only reduced the length of the excerpt and edited the citation style in the AI-generated subtext to fit the SSS format, for sake of academic courtesies, the limits of which we have already tested here. We hope this essay is read as a performative text, in accordance with Ian Bogost’s argument that the job of the ontologist is a performative one (Leach 2020: 62)
- This is not, however, parallel to Deleuze and Guattari’s partial objects and desiring machines
- OOO is by no means a groundbreakingly new theory. Panpsychism and panexperientialism is as old as Western philosophy, and metaphors of concepts and objects ‘coming to life’ or being ’embodied’ have been deployed as recently as in Derrida’s spectral reading of Marx’s wooden table coming to life as it enters the marketplace of economic exchange (2011[1994])
- Harman’s critiques of philosophy overmining and undermining objects can explain Emmeche’s undecidedness and also show the fallacious polarity of the question of attributing or not attributing minds/umwelts to machines.
- Notice, here, the similarity between the physical groundedness aspect of Agents Design and Hoffemeyer’s ‘life cycle’ (1996) which proposes that the unit of evolution is a result of environmental non-genetic inputs as well as genetic programming- and that animals as objects are interpretive subjects and causal agents in their own lives.
- In our banal postmodern cultural malaise of retro-pastiche (Jameson 2005[1991]), is there really any difference between creative AI and humans? Our impotence for genuine innovation and radical creativity is no better than AI who depend on historical datasets to generate content. It is not that AI have yet to develop some higher level of symbolic interpretive competence to catch up to us. In Deacon’s (2010) hierarchy of interpretive competence, consciousness is not solely of the symbolic kind. With AI that concatenate historical datasets, the language used is not of symbolic or iconic nature but indexical. We may even extend this, as Baudrillard has suggested, to propose that AI and computer systems in general in the postmodern era operate like a virus – through fractalized indexical information systems (Genosko 2016). We are as ‘robotic’ as them. Perhaps AI doesn’t need to get ‘smarter’, if humans are getting ‘dumber’. We have already surpassed Ray Kurzweil’s singularity – but not one of transcendental intelligence and self-becoming, but one of banal stupidity. Humans are already done. It’s not that the dreaded robot takeover is in the future. The internet is a self-discursive autopoietic system, and we are external to it, mere voyeurs to political, economic, communicative processes. If in the 21st century automation and economic surplus has not produced an egalitarian society, then how can we expect that artistic automation and informational surplus will be any different? We are in a communicative capitalist realism, to borrow Mark Fisher’s term
- Intentional intervention can also produce a radical transformation of objects’ behaviour and experiential Umwelten without any change in their sensorial Umwelten. For example, Erik Alalooga’s 2022 Tartu art installation entitled ‘Liberated Machines’ took ordinary machines and re-configured them to interact with each other and produce quasi-musical ensembles. Freed from their monotonous everyday applications, their interaction was independent of humans- “Man is only an observer here.”.Their sensory and operative organs were altered and augmented only slightly, but their experience of each other was completely transformed.
Featured photo credit: Mian Abdul Hameed Hammad