This is the first part of several essays on the principle of grounding. I feel that this principle helps me communicate and think better. I’ll try to keep each essay short
A few years ago I was doing my masters, with the goal of using language to command robots. I was very excited because the research was very interesting to me - I needed to understand how to write software for robots, how the robot works as a system, and various fascinating aspects of human-robot interaction and linguistics. It felt super futuristic to be at the meeting point of so many disciplines.
Specifically, I was trying to teach a robot to follow some simple
commands such as Pick up the cup. I was working with a robot which
had two hands, and a 3D scanning sensor for eyes, and was usually
facing a table with a bunch of standard household objects on top of
it. If the robot was facing a specific table with specific object
configuration, it is possible to program and hard-code the robot arm
to move towards the cup, grab the cup, and then lift it. However, as
soon as something tiny changes in the configuration, the position of
the cup the robot, or the command, then the hard-coded solution would
no longer work, instead making the robot wave its hands in air
clumsily. Programming the robot in a single situation may be fun, but
doing the same thing over and over will get boring and annoying quite
fast, and on top of it wouldn’t be effective.
My task was to make the robot succeed when the cup looks differently,
or is at a different location, or even if the command asked for a
different item. In each new environment, the robot needed to connect
the abstract symbol for cup, bowl, bottle from the command to the
sensory information from the environment, figure out the right motions
and execute them.
Luckily, I didn’t have to start from scratch. I stepped on the shoulders of giants, my professor and the senior graduate students, who had already invented algorithms for this meeting point of disciplines. They created techniques for connecting the symbolic command to the physical reality, albeit for a very different robot, in a different setting. My task was to port the algorithm to this new robot and make sure it works, helping justify the algorithm’s ability to adapt to new situations.
The algorithm connected the text of the command and the sensory
perceptions of the robot and its imaginary motions. This is
grounding, as I came to know it. Grounding is the connecting
between something completely abstract and symbolic, such as the
symbols for the action pick up and the object cup and on the other
hand the specific sets of 3D pixels and specific hand trajectories and
motor settings. The latter ones exist in the reality, they are very
exact - they are solid ground. The former are just the names of items
or motions and could resolve differently depending on the situation.
In this project, grounding is the same as name resolution. Finding the
best object which corresponds to the meaning of the cup, given the
whole context of the command gets to be named the cup. Finding the
best trajectory of the robot hand that corresponds to the action of
pick up, again gets to be named pick up.
This connection, the grounding, bridged the gap between the abstract, clean and imaginary part of the robot’s understanding, the name of each object and trajectory, and the concrete, noisy and complicated physical world. Without crossing this divide somehow, the robot can’t to do anything of physical importance correctly. These physical actions are what made the robot a ROBOT, instead of just a normal computer. The algorithm built a bridge for the robot between the abstract and the concrete, so that the robot can cross it every time it needs to do something of physical importance.
The specific algorithm may have been clever, but what I found to be the really wise part, was the step of connecting the abstract to the specific. Every time I was trying explain something and provided specific examples of my abstract ideas I noticed that not only I was able to communicate these ideas more clearly, I also understood my ideas better. Grounding became a tool for better communicating and a tool for clearer thinking.
The next several essays will dive more deeply into the various aspects and applications of this tool.