Enlarge (credit: SEAN GLADWELL)

Unraveling how consciousness arises out of particular configurations of organic matter is a quest that has absorbed scientists and philosophers for ages. Now, with AI systems behaving in strikingly conscious-looking ways, it is more important than ever to get a handle on who and what is capable of experiencing life on a conscious level. As Christof Koch writes in Then I Am Myself the World, "That you are intimately acquainted with the way life feels is a brute fact about the world that cries out for an explanation." His explanation—bounded by the limits of current research and framed through Koch’s preferred theory of consciousness—is what he eloquently attempts to deliver.

Koch, a physicist, neuroscientist, and former president of the Allen Institute for Brain Science, has spent his career hunting for the seat of consciousness, scouring the brain for physical footprints of subjective experience. It turns out that the posterior hot zone, a region in the back of the neocortex, is intricately connected to self-awareness and experiences of sound, sight, and touch. Dense networks of neocortical neurons in this area connect in a looped configuration; output signals feedback into input neurons, allowing the posterior hot zone to influence its own behavior. And herein, Koch claims, lies the key to consciousness.

In the hot zone

According to integrated information theory (IIT)—which Koch strongly favors over a multitude of contending theories of consciousness—the Rosetta Stone of subjective experience is the ability of a system to influence itself: to use its past state to affect its present state and its present state to influence its future state.

Read 12 remaining paragraphs | Comments

The emergence of artificial general intelligence (AGI)—systems that can perform any task a human can—could be the most important event in human history, one that radically affects all aspects of our collective lives. Yet AGI, which could emerge soon, remains an elusive and controversial concept. We lack a clear definition of what it is, we don’t know how we will detect it, and we don’t know how to deal with it if it finally emerges.

What we do know, however, is that today’s approaches to studying AGI are not nearly rigorous enough. Within industry, where many of today’s AI breakthroughs are happening, companies like OpenAI are actively striving to create AGI, but include research on AGI’s social dimensions and safety issues only as their corporate leaders see fit. While the academic community looks at AGI more broadly, seeking the characteristics of a new intelligent life-form, academic institutions don’t have the resources for a significant effort.

Thinking about AGI calls to mind another poorly understood and speculative phenomenon with the potential for transformative impacts on humankind. We believe that the SETI Institute’s efforts to detect advanced extraterrestrial intelligence demonstrate several valuable concepts that can be adapted for AGI research. Instead of taking a dogmatic or sensationalist stance, the SETI project takes a scientifically rigorous and pragmatic approach—putting the best possible mechanisms in place for the definition, detection, and interpretation of signs of possible alien intelligence.

The idea behind SETI goes back 60 years, to the beginning of the space age. In their 1959 Nature paper, the physicists Giuseppe Cocconi and Philip Morrison described the need to search for interstellar communication. Assuming the uncertainty of extraterrestrial civilizations’ existence and technological sophistication, they theorized about how an alien society would try to communicate and discussed how we should best “listen” for messages. Inspired by this position, we argue for a similar approach to studying AGI, in all its uncertainties.

AI researchers are still debating how probable it is that AGI will emerge and how to detect it. However, the challenges in defining AGI and the difficulties in measuring it are not a justification for ignoring it or for taking a “we’ll know when we see it” approach. On the contrary, these issues strengthen the need for an interdisciplinary approach to AGI detection, evaluation, and public education, including a science-based approach to the risks associated with AGI.

We need a SETI-like approach to AGI now

The last few years have shown a vast leap in AI capabilities. The large language models (LLMs) that power chatbots like ChatGPT, which can converse convincingly with humans, have renewed the discussion about AGI. For example, recent articles have stated that ChatGPT shows “sparks” of AGI, is capable of reasoning, and outperforms humans in many evaluations.

While these claims are intriguing and exciting, there are reasons to be skeptical. In fact, a large group of scientists argue that the current set of tools won’t bring us any closer to true AGI. But given the risks associated with AGI, if there is even a small likelihood of it occurring, we must make a serious effort to develop a standard definition of AGI, establish a SETI-like approach to detecting it, and devise ways to safely interact with it if it emerges.

Challenge 1: How to define AGI

The crucial first step is to define what exactly to look for. In SETI’s case, researchers decided to look for so-called narrow-band signals distinct from other radio signals present in the cosmic background. These signals are considered intentional and only produced by intelligent life.

In the case of AGI, matters are far more complicated. Today, there is no clear definition of “artificial general intelligence” (other terms, such as strong AI, human-level intelligence, and superintelligence are also widely used to describe similar concepts). The term is hard to define because it contains other imprecise and controversial terms. Although “intelligence” is defined in the Oxford Dictionary as “the ability to acquire and apply knowledge and skills,” there is still much debate on which skills are involved and how they can be measured. The term “general” is also ambiguous. Does an AGI need to be able to do everything a human can do? Is generality a quality we measure as a binary or continuous variable?

One of the first missions of a “SETI for AGI” construct must be to clearly define the terms “general” and “intelligence” so the research community can speak about them concretely and consistently. These definitions need to be grounded in the disciplines supporting the AGI concept, such as computer science, measurement science, neuroscience, psychology, mathematics, engineering, and philosophy. Once we have clear definitions of these terms, we’ll need to find ways to measure them.

There’s also the crucial question of whether a true AGI must include consciousness, personhood, and self-awareness. These terms also have multiple definitions, and the relationships between them and intelligence must be clarified. Although it’s generally thought that consciousness isn’t necessary for intelligence, it’s often intertwined with discussions of AGI because creating a self-aware machine would have many philosophical, societal, and legal implications. Would a new large language model that can answer an IQ test better than a human be as important to detect as a truly conscious machine?

Getty Images

Challenge 2: How to measure AGI

In the case of SETI, if a candidate narrow-band signal is detected, an expert group will verify that it is indeed an extraterrestrial source. They’ll use established criteria—for example, looking at the signal type and source and checking for repetition—and conduct all the assessments at multiple facilities for additional validation.

How to best measure computer intelligence has been a long-standing question in the field. In a famous 1950 paper, Alan Turing proposed the “imitation game,” now more widely known as the Turing Test, which assesses whether human interlocutors can distinguish if they are chatting with a human or a machine. Although the Turing Test has been useful for evaluations in the past, the rise of LLMs has made it clear that it’s not a complete enough test to measure intelligence. As Turing noted in his paper, the imitation game does an excellent job of testing if a computer can imitate the language-generation process, but the relationship between imitating language and thinking is still an open question. Other techniques will certainly be needed.

These appraisals must be directed at different dimensions of intelligence. Although measures of human intelligence are controversial, IQ tests can provide an initial baseline to assess one dimension. In addition, cognitive tests on topics such as creative problem-solving, rapid learning and adaptation, reasoning, goal-directed behavior, and self-awareness would be required to assess the general intelligence of a system.

These cognitive tests will be useful, but it’s important to remember that they were designed for humans and might contain certain assumptions about basic human capabilities that might not apply to computers, even those with AGI abilities. For example, depending on how it’s trained, a machine may score very high on an IQ test but remain unable to solve much simpler tasks. In addition, the AI may have other communication modalities and abilities that would not be measurable by our traditional tests.

There’s a clear need to design novel evaluations to measure AGI or its subdimensions accurately. This process would also require a diverse set of researchers from different fields who deeply understand AI, are familiar with the currently available tests, and have the competency, creativity, and foresight to design novel tests. These measurements will hopefully alert us when meaningful progress is made toward AGI.

Once we have developed a standard definition of AGI and developed methodologies to detect it, we must devise a way to address its emergence.

Challenge 3: How to deal with AGI

Once we have discovered this new form of intelligence, we must be prepared to answer questions such as: Is the newly discovered intelligence a new form of life? What kinds of rights does it have? What kinds of rights do we have regarding this intelligence? What are the potential safety concerns, and what is our approach to handling the AGI entity, containing it, and safeguarding ourselves from it?

Here, too, SETI provides inspiration. SETI has protocols for handling the evidence of a sign of extraterrestrial intelligence. SETI’s post-detection protocols emphasize validation, transparency, and cooperation with the United Nations, with the goal of maximizing the credibility of the process, minimizing sensationalism, and bringing structure to such a profound event.

As with extraterrestrial intelligence, we need protocols for safe and secure interactions with AGI. These AGI protocols would serve as the internationally recognized framework for validating emergent AGI properties, bringing transparency to the entire process, ensuring international cooperation, applying safety-related best practices, and handling any ethical, social, and philosophical concerns.

We readily acknowledge that the SETI analogy can only go so far. If AGI emerges, it will be a human-made phenomenon. We will likely gradually engineer AGI and see it slowly emerge, so detection might be a process that takes place over a period of years, if not decades. In contrast, the existence of extraterrestrial life is something that we have no control over, and contact could happen very suddenly.

The discovery of a true AGI would be the most profound development in the history of science, and its consequences would be also entirely unpredictable. To best prepare, we need a methodical, comprehensive, principled, and interdisciplinary approach to defining, detecting, and dealing with AGI. With SETI as an inspiration, we propose that the AGI research community establish a similar framework to ensure an unbiased, scientific, transparent, and collaborative approach to dealing with possibly the most important development in human history.

Enlarge / Daniel Dennett, a leading philosopher with provocative takes on consciousness, free will, and AI, has died at 82. (credit: Alonso Nichols/Tufts University)

World-renowned philosopher Daniel Dennett, who championed controversial takes on consciousness and free will among other mind-bending subjects, died today at the age of 82.

(Full disclosure: This loss is personal. Dennett was our friend and a colleague of my spouse, Sean Carroll. Sean and I have many fond memories of shared meals and stimulating conversations on an enormous range of topics with Dan over the years. He was a true original and will be greatly missed.)

Stunned reactions to Dennett's unexpected passing began proliferating on social media shortly after the news broke. "Wrenching news. He's been a great friend and incredible inspiration for me throughout my career," the Santa Fe Institute's Melanie Mitchell, author of Artificial Intelligence: A Guide for Thinking Humans, wrote on X. "I will miss him enormously."

Read 18 remaining paragraphs | Comments