Our pond had been shrinking for weeks. Rain had been sparse for several months and the waterline had dropped nearly four feet, exposing a wide band of ground that normally sat underwater. For the deer that roam through the property, it meant new territory. Several feet of additional shoreline provides more room to wander and places to explore.

One night, a small group of deer came down near the pond. It was the same familiar group that wandered around the area every day. A couple of them were young, perhaps two or three years old.

We heard the commotion before we understood it. A loud splashing, something violent enough to make us run and grab a flashlight to look across the water. By the time the beam reached the opposing bank, everything appeared normal… almost. The flashlight caught the eye shine of an alligator, floating innocently in the darkness. Seeing it the next morning estimated its length to be roughly seven feet.

In the darkness we saw no deer, no struggle, nothing that told us exactly what had happened. By morning, one of the young deer was floating in the pond. The alligator had drowned it.

It would be easy to describe the scene in moral language; brutal, cruel, even tragic. But none of those words belonged to the alligator, nor the deer, or the environment in which it took place. The drought followed its course. The deer followed its instincts. The alligator followed its nature. The tragedy occurred at the point where those patterns intersected.

The alligator did not hate the deer. The deer did not make a moral mistake. The landscape had changed, which created opportunity and a lethal capability was already waiting at the edge of it.

By morning, the event looked straightforward. A deer had wandered too close to the water, and an alligator had taken it. Nature did what nature does. Simplicity is often something we add afterward.

In the dark, nothing had looked obviously wrong. The pond was familiar, but not quite the same. The waterline had dropped and new ground had appeared. The deer explored it. The alligator remained what it had always been. No one had changed character rather the environment they lived in had changed around them.

That is the part worth noticing.

Human beings are very good at carrying old assumptions into new conditions. We recognize the shoreline, so we assume we understand the pond. We recognize the actors, so we assume we understand the risks. Then something happens that seems sudden only because the changes leading to it were gradual. We like clean explanations.

• Predator and prey; good and bad; safe and dangerous.

We like to assign intent afterward. We label one thing dangerous, another foolish, another destructive, etc. However, systems rarely behave that neatly. Sometimes the more useful question is not who was good or bad. Sometimes the better question is: What has changed inside the system? And lately, one of our systems has been changing very quickly.

Artificial intelligence is giving us access to capabilities that didn't exist in ordinary life only a short time ago. And much like our receding pond shoreline, those capabilities are opening new ground faster than our instincts, rules, and assumptions can comfortably adjust.

Perhaps the emerging AI safety debate is not really about whether the alligator is evil or the deer is innocent. Rather it is about whether we are changing the shoreline faster than we can learn where danger now lives. Are we evaluating artificial intelligence with concepts inherited from the old landscape; tool, servant, threat, intelligence, intention, deception, responsibility; while the boundaries of those concepts are themselves moving?

Humans Change the Shoreline Too

Nature is not the only thing that changes the landscape around us, human beings do it constantly.

Consider the automobile.

Before cars existed, no one died in a car crash. There were no traffic lights, no driver’s licenses, no seat belts, no crash standards, no auto insurance, no interstate highways built around machines moving seventy miles an hour. There was also no such thing as driving twenty miles to work, crossing a state in a few hours, living far from where you worked, or loading a family into a vehicle and deciding on a whim to go somewhere that would once have required days of travel.

The automobile did not simply give humanity a better way to move. It changed the world people had to learn how to live in. It created new industries and erased others. It changed where cities grew. It changed commerce, leisure, warfare, emergency response, farming, travel, family life, even the distance people considered “close.”

And almost immediately, people began advancing it. They modified engines to make them faster so they could race. They adapted them for work, exploration, sport, luxury, transport, and war. The technology kept opening new ground. Some of that ground was useful while some of it was dangerous. Some of it was both. The important point is that society did not fully understand the consequences before the technology spread.

• The rules came later.
• The roads came later.
• The safety systems came later.
• The habits came later.

Human beings had created a new environment and then had to learn how to inhabit it. That pattern is not unusual. Technology changes more than what we can do, it changes the world we have to learn how to live in. There may be an assumption hiding inside the metaphor: it’s that we may be too quick to decide which creature represents humanity, the deer or the alligator...

A Different Kind of Machine

Computing changed the landscape again. At first, computers were tools of calculation. They processed numbers, followed instructions, and performed tasks humans had explicitly programmed them to perform. But almost as soon as computers became powerful enough to be useful, researchers began asking a different question:

• Could a machine do more than calculate?
• Could it recognize patterns?
• Could it solve problems?
• Could it imitate parts of human reasoning?

By the 1950s and 1960s, researchers were already experimenting with what would eventually be called artificial intelligence. Some of those early systems could play games, prove mathematical theorems, or carry on surprisingly convincing textual conversations. The promise was enormous while the capability was not.

The computers were slow and memory expensive. Data was limited and many of the early predictions about intelligent machines proved wildly optimistic, and the enthusiasm eventually cooled. The shoreline had appeared, but humanity did not yet have the machinery to explore very far beyond it.

Then the machinery changed. Computers became faster as storage became cheaper. Networks connected billions of people while vast quantities of information became digital. New methods allowed machines to learn patterns rather than simply follow instructions written line by line. Eventually, something changed in the relationship between humans and computers; we stopped only telling machines exactly what to do. We began building machines that could learn how to do things we had not explicitly described.

That is a very different kind of technological shoreline.

Recursive Improvement

There is nothing especially alien about the idea of recursive improvement. Human beings do it constantly. An athlete changes a training plan, watches the result, then adjusts the plan again. An engineer improves a design, tests it, learns from the failure, and builds the next version differently. A manufacturer studies a process, removes waste, measures the improvement, and uses what was learned to make the next improvement easier.

Even learning itself works this way. What we learn today changes how effectively we can learn tomorrow. In that sense, recursive improvement is familiar.

Artificial intelligence changes the scale.

A human being is limited by time, biology, attention, memory, access to information, and the number of experiments that can reasonably be performed. Software can operate under very different constraints. It can analyze large amounts of information, test alternatives rapidly, preserve what works, discard what does not, and use those results to improve the next iteration; very quickly.

That does not make the process mysterious. It makes it powerful, and power changes the environment.

When artificial intelligence eventually becomes capable of helping to, or improving the code, models, tools, or development processes that produce the next generation of artificial intelligence, the feedback loop becomes faster still. But even then, the system does not appear in a vacuum.

• Human beings provide the computers.
• Human beings decide what the system can access.
• Human beings decide what it is allowed to modify.
• Human beings decide whether an improvement remains an experiment or becomes something deployed into the world.

We establish objectives, create the incentives, define the boundaries, and determine how much authority we are willing to hand over. So perhaps recursive improvement is not best understood as the moment the machine somehow leaves humanity behind. Perhaps it is another shoreline we are creating, and the “unusual” part is not that improvement can build upon improvement. We have always done that. The “unusual” part is how quickly, broadly, and autonomously we may be able to automate the process.

And once again, the question becomes less about whether technology is behaving according to its capabilities and more about whether we understand the environment we are creating around those capabilities.

The pond is still there.

The deer still come down to drink. The alligator still moves through the water. The shoreline did not change the nature of either animal. It changed what became possible when they encountered one another. What changed was the space between them.

I suggest that may be the most useful way to think about artificial intelligence. Technology will continue to become more capable. Some of those capabilities will be useful while others create risks we have not yet learned how to manage. Some will almost certainly surprise us. But surprise is not the same thing as helplessness.

We are not standing outside this system watching it happen to us; we are building it. We decide how much capability to create, where to deploy it, what authority to give it, what limits to impose, and how quickly to move when the landscape begins changing beneath us.

The alligator did not choose the drought that exposed the shoreline. We are choosing much of this shoreline. Humans are building the machines, defining their environments, testing their limits and discovering what opportunities their capabilities create. Perhaps before deciding how frightened we should be of what has entered the new territory, we should ask which creature in the story behaves the most like us.

That does not mean every consequence can be predicted or controlled. Human beings have never had that luxury with transformative technology. But responsibility does not require perfect foresight, it requires attention. It requires recognizing when familiar ground is no longer familiar. It requires learning before certainty arrives.

Maybe the first test of artificial intelligence is not what it becomes, maybe it is what we become while learning how to live beside it. The question then is simply what type of human does a powerful new technology require us to become?