1 Introduction: What Is AI?

1.1 Cognitive self-experiment: The nine-dot puzzle

Copy link

⊕Each chapter in this book will include one or more “cognitive self-experiments.” These are small experiments that can be done by any reader who is a human. (;

⊕The point of these experiments is to give you a chance to get direct, first-hand experience with interesting ideas that turn out to be fundamentally important in AI.

⊕A word of advice: You will learn a lot more by actively doing these experiments than by just passively reading through them. Nothing can replace the valuable learning that happens when your brain is actively wrestling with a problem or activity.

⊕Moreover, a lot of these experiments have “spoilers” at the end, and so if you just read through them without doing them, you may not ever be able to go back and recreate the experience.

We are going to kick off our adventures in intelligence by considering a very famous puzzle called the nine-dot puzzle.

First, draw nine dots on your piece of paper, as shown above.

Now, your task is to draw a series of straight lines that will connect all nine dots. The catch is that you can use no more than FOUR lines, and they must all be connected, i.e., you have to draw them WITHOUT picking up your pencil from the paper.

Ready? Go!

Give yourself at least 5 minutes to work on this.

…

…

…

…

…

⊕In American culture, there is no more iconic “thinking sound” than the music from the Jeopardy game show. (If you are like me, there is also, ironically, no sound more guaranteed to completely disrupt your thinking processes!)

(Jeopardy music plays in background)

…

…

…

…

…

If you have never seen it before, the nine-dot puzzle is quite difficult, and can seem downright impossible. Four straight lines? WITHOUT picking up your pencil? Inconceivable!

⊕Chronicle, E. P., Ormerod, T. C., & MacGregor, J. N. (2001). When insight just won’t come: The failure of visual cues in the nine-dot problem. The Quarterly Journal of Experimental Psychology: Section A, 54(3), 903–919.

If you find yourself struggling, don’t worry, you are in good company. In most research studies given to “naive” participants (i.e., people who have never seen this puzzle before), a whopping 0% of participants are able to solve it. (The difficulty of this puzzle is the reason that it is such a classic!)

⊕MacGregor, J. N., Ormerod, T. C., & Chronicle, E. P. (2001). Information processing and insight: A process model of performance on the nine-dot and related problems. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27(1), 176.

For example, in one study, not a single one of 27 undergraduate students was able to solve this puzzle, even after ten separate attempts.

Once you have found it or seen it, the solution to the nine-dot puzzle can seem blindingly obvious. But, getting there requires a kind of leap of intuition regarding where you believe your line segments can start and end.

⊕…which is why solving this puzzle is often wryly described as “thinking outside the box.”

In particular, most people attempting the puzzle for the first time will assume that each of their line segments must start and end on a dot. It takes significant creativity and cognitive flexibility to realize that line segments can stretch beyond the invisible “box” defined by the grid of dots.

The nine-dot puzzle is an example of what cognitive scientists call insight problem solving, as opposed to routine problem solving. For example, doing long division involves routine problem solving, because if you know the steps, then it’s just a matter of applying them (correctly) until you are done. Insight problem solving, on the other hand, requires you to fundamentally rethink some aspect of the problem in order to solve it.

“This is all well and good,” you might be thinking, “but what does the nine-dot puzzle have to do with AI?” We are getting there! But we first need to describe the problem using more precise terminology.

1.2 A more precise description of the nine-dot puzzle

Copy link

One way to describe the nine-dot puzzle is as a search problem:

Given all possible straight line segments going through some of the dots, find a contiguous set of four segments that connects all nine dots.

In your early attempts to solve the nine-dot puzzle, you might have tried out some different solutions like these:

In making these attempts, you were searching through a space of possible collections of line segments. Using this terminology, we can describe what makes the problem so difficult:

If possible line segments are restricted to those that start and end on a dot (call this Search Space A), then the correct answer is impossible to find, i.e., the correct answer is not contained within the space of possibilities being searched.

⊕A quick back-of-the-envelope calculation can tell us how many possibilities we are searching through, even using the (incorrectly) constrained Search Space A:
   —   Assume you start the first line segment at any one dot (out of 9); then your first segment ends at any other dot (out of 8 remaining); second segment ends at any other dot (out of 7); third segment ends at any other dot (out of 6); and finally the fourth segment ends at any other dot (out of 5).
   —   So, \(9 \times 8 \times 7 \times 6 \times 5 = 15,120\) possible sets of four connected line segments, give or take. This is a slight overestimate, as it includes line segments that might overlap or be continuations of the same line.
   —   However, this estimate should be sufficient to convince ourselves that this is indeed a sizable search space. (This kind of rough estimate is common in AI, as we shall see throughout this book!)

Even if the search space is defined in this limited (and, it turns out, incorrect) way, the search space is still quite large. In other words, there are many, many possible line segments that would fit under the terms of this incorrect definition of the problem, which is probably why people can spend such a long time trying all kinds of various (and inevitably incorrect) solutions, like the ones shown in the above figure.

We can also use this terminology to describe the magical leap of intuition that will allow finding the correct answer:

In order to find the correct solution, the space of possible line segments must include those that start/end on one or more dots and end/start either on another dot or anywhere in the empty space outside the square defined by the nine dots (call this Search Space B).

We can observe that Search Space B is much bigger than Search Space A. (In fact, A is a strict subset of B. Moreover, A is finite, whereas B is infinite!) Therefore, even knowing that you should be using Search Space B, you might have a lot of lollygagging around to do before finding the solution—and there are no guarantees that you even will be able to find the correct solution! In particular:

⊕It turns out these search properties—using an iterative, trial-and-error type of process; having to remember previous attempts; and trying to use some additional cleverness or insight to guide the search—are common to many AI techniques! We will come back to these many times throughout this book.

• your process of searching will probably still involve a good deal of trial-and-error;
• you have to remember your previous attempts so you don’t keep repeating the same mistakes over and over;
• there is still a need for some cleverness or insight on your part (or just blind luck!) to land on the correct solution.

All of these are characteristics of the process that you use to search within a given search space. We can call this your search algorithm, where “algorithm” is just our fancy computer science term for a sequence of steps that you take.

So, putting it all together, we might describe a person’s typical nine-dot puzzle experience as follows:

1. Assume Search Space A.
2. Spend some time fruitlessly searching through A.
3. Eventually, change to Search Space B.
4. Spend some time searching through B until the solution is found.

All four of these steps require intelligent effort, and these brief descriptions gloss over a lot of important details. For example, when/why would someone decide to finally give up on Search Space A? How might someone decide to try Search Space B (as opposed to other possible search spaces)? Etc.

More generally, we can say that finding the correct solution to the nine-dot puzzle requires landing on the right search space AND having an effective search algorithm.

• If you don’t have the right search space, it doesn’t matter how good your search algorithm is; you will never find the correct answer.
• If you don’t have a good search algorithm, it doesn’t matter whether you are in the right search space; you are not likely to find the correct answer.

It turns out that these basic principles about search spaces and search algorithms are EXACTLY what AI is all about!

⊕Interestingly, finding the right search space and the right search algorithm for a given problem is still mostly the province of AI researchers. In other words, it is mostly through human insight that we’ve managed to make AI techniques work for so many interesting problems. We are still quite a long way from having AI systems that can really, truly look at a problem from scratch, and come up with their own search spaces and search algorithms to solve it.

Virtually all of AI is about trying to solve complex problems using various kinds of search algorithms. However, like the nine-dot puzzle, it is more of an art than a science to figure out what the right search space is. Then, for a given search space, it takes additional work to find the right kind of search algorithm, including figuring out specifics of the three search properties described above: iterative trial-and-error, remembering what you’ve already tried, and incorporating some kind of extra cleverness or insight to help the search along.

We will come back to these ideas shortly! But first, we’ll take a slight detour to start building the definition of AI that we will use throughout this book.

1.3 What is artificial intelligence? Part 1

Copy link

Ah, the million dollar question, a.k.a., what are we all doing here on this AI learning adventure?

It is easy to get bogged down in philosophical discussions when talking about what AI is, or isn’t, or might someday be. However, an accurate and pragmatic definition begins with observing that AI is, first and foremost, a scientific discipline. You take courses in AI, there are AI textbooks and AI professors, and you can say, “I will use AI techniques to solve this particular problem.”

Thus, you can think of AI in the same way that you think about other disciplines like chemistry, physics, math, or mechanical engineering. AI is a particular field of human knowledge that has to do with studying certain kinds of problems and certain kinds of solutions to those problems. (What kinds of problems and solutions we will get to shortly!)

The chemistry analogy is a useful one, because you probably already have some pretty good intuitions about chemistry (assuming you went through a traditional educational pathway that included chemistry in high school or college). Here are some useful takeaways from this analogy:

• Chemists study both naturally occuring chemical reactions as well as artificial ones created in the lab. Similarly, AI people study both naturally occurring intelligences (e.g., people, animals) as well as artificial systems created in the lab.
• There are many subfields and offshoots of chemistry (e.g., environmental chemistry, materials science, biochem, etc.). Similarly, there are many subfields and offshoots of AI (e.g., computer vision, machine learning, robotics, etc.).
• Chemists have created new things that have been both very good and very bad for society, often at the same time (e.g., plastic). Similarly, AI people have created new things that have been both very good and very bad for society (e.g., online behavior tracking and prediction).

One source of confusion regarding definitions of AI is that people often use the term AI to refer to the scientific discipline as well as the objects of study, e.g., “She built an AI to solve the problem.” In chemistry, people usually use different terms for the discipline versus the objects of study, e.g., “He developed a chemical reaction,” or “She created a new chemical,” or “They discovered an interesting substance.”

And of course, increasing the confusion further are the values that people often attach onto the term AI, for instance implications that every AI system is somehow sentient or eerily similar to humans in some magical way.

AI is not magic, any more than chemistry is. (Although, just as in chemistry, we can do some very cool things with AI techniques and systems!) An “AI” is not somehow magically intelligent just because we call it that, any more than a combustion reaction in the lab is the same as a wildfire just because we call it a “fire.” (Although, just as we can learn valuable new things about wildfires by studying combustion reactions in the lab, certainly we can learn valuable new things about human intelligence by building and studying AI systems in the lab.)

Following this observation that AI is a scientific discipline like chemistry, for the rest of this book, we will use the term “AI” either as a noun to refer to the discipline, or as an adjective to refer to some object of study, e.g., AI technique, AI system, AI person, etc.

1.4 What is artificial intelligence? Part 2

Copy link

So, given that AI is a scientific discipline…what is AI the study of, exactly? In this book, we will use the following definition:

⊕Notice that we did NOT use the word “intelligence” anywhere in this definition! (There are a lot of circular definitions of AI that say something like, “AI is the study of computational systems that emulate human intelligence,” which is not very helpful when we are trying to pin things down more precisely.)

There are six key elements of this definition that bear further scrutiny, and we address them in reverse order.

1.5 AI as a scientific discipline

Copy link

There is a sixth term in our AI definition that we didn’t yet define, and that is the “scientific study” part. What does it mean to study something scientifically?

⊕The scientific study of science itself (!) falls under the fascinating discipline of philosophy of science.

One of the most amazing bits of luck I had in my education was having Nancy Nersessian, a world-renowned philosopher of science, working two doors down from my PhD advisor’s office in graduate school. Nancy’s research centers on understanding the cognitive processes by which scientists make new discoveries, which, if you think about it, is an incredible demonstration of our species’ creative powers. Her classes and writings completely changed the way I think about science and creativity.

This book is an excellent primer: Nersessian, N. J. (2008). Creating scientific concepts. MIT press.

Getting AI sytems to demonstrate this level of creative intelligence remains an aspirational goal, though there have been attempts! For more, see the chapter on Creativity.

You may be familiar with definitions of science that revolve around using the scientific method: coming up with hypotheses, making predictions, and then doing experiments to test those predictions. While this rather strict definition does describe some scientific doings, real-world science has always been, and continues to be, much more diverse in how it progresses. Real science involves storytelling, accidents, influences of contemporary politics and culture and economics and technology, collaboration, competition, thought experiments, drawings, field trips, mistakes, arguments, and all sorts of other messy and interesting human activities.

While all sciences share these kinds of activities, there are certain “flavors” of science that lend themselves more to certain systematic methods of study. Three of the most common flavors are:

1. Mathematics
2. Engineering
3. Empirical science

AI (and really computer science more broadly) is a very interesting sort of science because it combines aspects of three distinct types of scientific fields: mathematics, engineering, and

where does knowledge come from

math, engineering, empirical science

Newell and Simon.

etc.

1.6 Different kinds of AI <=> Different kinds of AI people

Copy link

While you might imagine that all AI people are number-crunching, code-writing geeks, there are actually lots of different kinds of contributions that are valuable in AI, and we really need lots of different kinds of people to contribute!

1.7 Rest of the chapter stuffs

Copy link

For now, suffice it to say that virtually every AI technique can be broken down in these terms. (For instance, when encountering any new AI technique, it can be a useful exercise to ask: “What are the representations?” and “Where is the search?” and “How hard is the search?”—i.e., questions that map onto these three elements.)

⊕For alternative definitions, see “What exactly IS AI?” in the excellent AI overview written by Selmer Bringsjord and Naveen Sundar Govindarajulu in the Stanford Encyclopedia of Philosophy (SEP).

⊕(I find SEP to be an indispensable resource for learning about big ideas in many areas of AI, cognitive science, math, and of course philosophy.)

Unlike other definitions of AI, this definition puts many kinds of intelligence on equal footing in terms of how we can study them, and also in terms of which ones fall under the purview of AI to study: human intelligence, machine intelligence, non-human animal intelligence. Hypothetical alien intelligence. Babies, adults, the elderly. Neurotypical people, neurodiverse people. Intelligence that is rational and/or irrational. Intelligence that is mathematical, physical, linguistic, visual, spatial, social, emotional, and/or cultural. AI for robots, AI for interactive systems…and so on.

I find it all equally fascinating, and we will consider examples of all of these kinds of intelligence throughout this book.

<being a cognitive scientist means having a subject to study all the time!>

<computational perspective. humans and other animals are an existence proof of what is possible.>