Messi, Ronaldo, Mr…Roboto? – AI in Sports

(*) Congratulations to the UT Austin Robot Soccer Team for winning worldwide championship in the 3D Simulation League with a second-place in the Standard Platform League (SPL) at the 2016 RoboCup Competition in Leipzig, Germany.

On a hot Central Texas afternoon in the middle of June , members of SparkCognition’s marketing and communications team were attending soccer practice at the University of Texas at Austin (UT). They weren’t running up and down the field, they hadn’t broken a sweat, they weren’t even suited up. They were there as spectators to watch one of UT’s best soccer teams prepare for the year’s most prestigious international competition, the RoboCup.

That’s right, the RoboCup.   It’s not your typical soccer tournament. Hosted in Leipzig, Germany this summer, the RoboCup is an “international scientific initiative with the goal to advance the state of the art of intelligent robots.” The RoboCup competitions provide a channel for the dissemination and validation of innovative concepts and approaches for autonomous robots under challenging conditions. When established in 1997, the original mission was to field a team of robots capable of winning against the human soccer World Cup champions by 2050. And while we may be a ways off from that reality, the University of Texas at Austin RoboCup team is competitively advancing this mission, led by one of the top coaches in the league.

Peter Stone’s office, and the Robotic Soccer Lab, is housed in the UT Computer Science Department’s Gates Dell Complex, where on any given day one will see autonomous robots roaming the hallways. Dr. Stone, or on this particular afternoon, Coach Stone, is the David Bruton, Jr. Centennial Professor of Computer Science at UT. Having received his Ph.D. in 1998 and his M.S. in 1995 from Carnegie Mellon University, both in Computer Science, Dr. Stone is a leading researcher and professor in the field of Artificial Intelligence. His focus areas include planning, machine learning, multiagent systems, robotics, and e-commerce. His long-term research goal is to create complete, robust, autonomous agents that can learn to interact with other intelligent agents in a wide range of complex and dynamic environments. As a lifelong soccer player with a passion for the sport – at one point in his career Stone almost made it to the major league – Coach Stone is pursuing his research goals by scoring goals on the soccer field, and he is making major progress.

Since 2002, Coach Stone has led the UT Robotics Team  to six RoboCup championships in the simulation challenge (computer software), and one in the standard platform challenge (robots). As a researcher, Dr. Stone works with his students year-round to advance his and their knowledge of the field, keeping abreast of the leading research, and publishing some of their own. As a champion competitor, Coach Stone leads a team of student researchers who share his interest, or more typically his passion, for the sport of soccer and the advancement of robotics.

When we caught up with Dr. Stone, he was leading a meeting of his RoboCup team, checking in on the state of their robots and the code which directs their play. This meeting was akin to a practice. Coach Stone began with a status check of where his team members were with their code, and then had them ‘drill’ their software. The practice resembled that of young children first learning to play soccer. Some robots made their way to the ball while others wandered the field. Some fell down and others stood with their ‘heads’ on a swivel, gauging where on the field they were in relation to the ball. Like parents of young children, the human team members followed the robots around the field, ensuring that they were on track and safeguarding them from hard falls. All the while, Coach Stone questioned his team (the humans) on approach and reasoning. He guided them with solutions to improve their programs and encouraged them to press forward with urgency – the competition is only a few weeks away.

In retrospect, the practice was quite fascinating.    The fact that these robots were behaving in a similar fashion as young children on a soccer field leaves one surmising that the next step is adolescence. The improvement over the last decade has been phenomenal, and if Moore’s Law holds water, we may see a robot beat a human at a mechanical sport sooner than anticipated.

As an athlete, I was surprised by how similar the RoboCup practice resembled a walk-through or film session. Perhaps this was the most interesting aspect of the practice – the human dimension. Dr. Peter Stone was, for the hour, Coach Stone. And as a coach on the field, he guided his team with stern direction, vision, and encouragement. He has led teams to championships before, and he will likely do so again.

After Coach Stone ended practice, we caught up with him for an interview.

What follows is a conversation with one of the country’s leading artificial intelligence and robotics researchers, and a top RoboCup coach.

Cognitive Times: Dr. Stone, what are your specific research areas and what are the goals of the lab that you run?

Peter Stone: Our lab is called the Learning Agents Research Group. The unifying long term goal is trying to create fully autonomous agents. It could be robots, it could be software agents that can exist autonomously in the real world for extended periods of time. That means dealing with lots of uncertainty with perception and with action. That’s the unifying theme. The sub areas of artificial intelligence that I work on are known as machine learning. In particular reinforcement learning which is learning about the effects of actions. Also multi agent systems – how do you get multiple autonomous entities, individuals, and programs to either cooperate or work against each other, or to just coexist such that they are all achieving their own goals and reasoning about each other’s goals. That’s what’s known as multi agent systems.

So reinforcement learning, multi agent systems, and then I use several test bed domains that involves robotics. We have the robot soccer team which involves learning, it involves multi agent systems, it involves robotics. We also have robots you may have seen wandering about the hallways; that’s our building wide intelligent projects. We have robots that can figure out where they are in the building and can interact with people. The long term goal of that project is to get to the point where people walk into the building and expect to see robots and interact with them to either do things that are entertaining or useful in some way. The robots should exhibit a clear intelligence or knowledge of everything that’s going on in the building. The long term goal of the robot soccer project, it’s a worldwide initiative called RoboCup, is by the year 2050 to have a team of human robots that can beat the best soccer team on a real soccer field.

Cognitive Times: And how far away are we from creating that goal, from creating the next Ronaldo?

Peter Stone: It turns out to that there is a lot of dimensions to that question. There is the physical hardware aspect of it. A robot that is as physically agile as an athletic person is a long ways off. There have been some examples of robots that show impressive degrees of agility, but then to combine that with sort of long running autonomy and the ability to make decisions, that is the software aspect of it, we are still a long ways off.

You sort of saw what is known as the standard platform league in RoboCup, where we all have the same hardware, that’s not by any stretch the state of the art of robots in the sense of agility. There are robots that can move faster, with more agility. Those are chosen with a particular price point in mind. It’s a nice, stable platform that allows us to push the limit of how fast they can move and also think about the strategic levels. There are enough teams from around the world who can have multiple of these robots so that we can have five versus five competitions.

“There have been some examples of robots that show impressive degrees of agility, but then to combine that with sort of long running autonomy and the ability to make decisions, that is the software aspect of it, we are still a long ways off.”

We also participate in the 3D simulation league which is where there are eleven robots on each team, but it’s all in software simulation. There we can even lift the next level up and think at more of the strategic multi agent level.  There are different leagues in RoboCup as well, there are some with wheeled robots that moves much more quickly than these. We have had people versus robot games against those robots every year since 2007 and it’s getting harder- but people are still much better than the robots.

Cognitive Times: Dr. Stone, you have a great record at the RoboCup, you seem to be very prevalent in the robotics league, you have a bookcase full of trophies, a lot of accolades – you are winning a lot. What is the secret sauce, what’s the magic here?

Peter Stone:I think the key is that over the years I have really deeply integrated my RoboCup team into PhD level research. There are some people who do RoboCup on the side, there are some of my students who you saw in the group meeting there who are doing RoboCup on the side, it’s not their main research project topic. But for many of my students there is actually something related to RoboCup that ends up playing a big role in their PhD thesis. It’s time when they are very motivated to come up with a new algorithm because not only is it going to help our team, but it is also pushing the frontier of research.  We have had a lot of research papers that have resulted from our contributions in RoboCup. I think that’s what makes it so that I can be doing this at a level of very deep involvement by top level graduate students – because it’s very closely tied to their research.

Cognitive Times: So, integration with the highest level of academy. I have read that your goal is to have complete robust autonomous agent that can learn and interact with other intelligent agents on a wide range of complex and dynamic tasks. We saw some of that in the soccer lab and we have seen robots roaming the halls here. You mentioned having guests eventually able to walk into the UT Computer Science building expecting to see a robot. I know that you also do some work with autonomous vehicles. What can we expect to see, or what is your hope, for the future of robotics?

Peter Stone: In terms of robots, we are trying to create the next generation of robots that people can interact with. This building is full of students who are trying to get the robots to be out and about among the students and doing things that students want to do. That is the project we are really focusing on, next generation robots and robotics.

There are other domains, I have a student who just finished work on autonomous bidding agents. These are agents that bid in market based systems where there are multiple other agents they need to cooperate or compete with in an economy. I have one student who is working at the intersection between machine learning and music. That’s another form of interaction with the machines. We’re asking if we can learn people’s preferences for sequences of songs, can we learn the impact that music has on people’s decision making?  There are a wide variety of domains that can serve as test beds for the study of robust autonomous agents. I basically try to give my students a lot of freedom in terms of their applications as long as there’s some tie to the main themes of the lab – reinforcement learning, multi agent systems, and robotics.

Cognitive Times: You have talked about reinforcement learning. What about reasoning? Do you think that reasoning plays a role here? Is that an approach that your students are employing?

Peter Stone: Yeah, we have a collaboration with Vladimir Lifschitz, who works in logic based reasoning, formulation of action languages. We had a paper last year at the main AI conference on integrating common sense reasoning with dialogue systems on a mobile robot. For example, if a robot heard a person at 9:00 am speaking to the robot and saying “please bring me toffee”, with common sense reasoning and the right knowledge representation on the line, the robot can reason that at 9:00 am it’s more likely that the person is asking for coffee rather than toffee. Just by setting the priors that at that time of day it is more likely for a person to ask for coffee than toffee, this sort of integration of reasoning and knowledge into the whole perception system can make a big difference.

Cognitive Times: It’s clear that over the span of the RoboCup there have been tremendous improvements in robotics and machine learning. What do you see as the key challenges today that you face, that your students face? Are the challenges more algorithmic or mechanic?

Peter Stone: They both go hand in hand. As the mechanical engineering improves, as the computational power improves, it gives us more capabilities and more software challenges to try to rise up and make better use of the hardware and technology. I focus more on the software side. So we are not making the mechanical engineering improvements in my lab but we are certainly trying to keep abreast of them and always using the best new hardware, because yeah, it provides different capabilities and different software challenges.

Cognitive Times: We talked about reinforcement learning and reasoning. In the practice arena, when the robot is active in the lab, one can see them thinking. They are turning their heads trying to find the ball, they are getting a sense of where they are on the field. Could you explain in high level terms how the robots’ decision-making process works when they are out there playing soccer?

Peter Stone: Yeah, I mean what you see is not them thinking; you are seeing them acting. You are seeing them behave. Really, you are projecting onto them the concept of thinking and what you would do if you were in their position. People are very quick to anthropomorphise robots or any kind of object.

“You are seeing them acting. You are seeing them behave. Really, you are projecting onto them the concept of thinking.”

Really what’s going on is that there are several different processes. There is a vision process at first, which is just sensing the world, seeing where is the ball, where is the line, where is the goal, where are the different sorts of recognizable objects – that’s the vision process. Then on top of that, consuming the output of vision is what we call localization – knowing where the robot is and where the other object, the ball, is on the field. The robot is keeping a probabilistic representation of where it is at any given time, where it thinks it is most likely to be, but then there are a lot of other places that it might be. And then on top of that, there is more of a behaviour-based system. You can think of it as sort of a finite state machine. If I am at the ball, then I should kick, if I am not at the ball I should walk towards the ball. If I don’t know where the ball is I should be looking for the ball. There is communication involved, the robots are sharing knowledge of where the ball is and what they are doing so that they are all not going to be doing the same thing at the same time.  Underneath that are the action modules, the actual low levels of where the robot sets its joints after it has decided to kick- how do you actually execute that kick, how do walk without falling, how do you get up after you have fallen over. So there is this pipeline of vision, localization, behaviour, and action.

Cognitive Times: How far along do you think we are from machines that actually do think? You are right, I mean I stand there and I look at those robots and say, “oh they are thinking. That robot is thinking about where it is on the field and were the ball is”, but I guess what you are saying is that it’s going through an action sequence?

Peter Stone: Well in some sense that’s what thinking it is. That’s what you are doing as well. There is a sequence of information processing that’s happening in your brain. So, it is fair to say that the robot is thinking. Just be careful to say it’s not doing it the way a person would do it, or the way an animal would do it. But, in the sense the thinking means taking in information through your sensors and deciding what actions to execute- then the robot is thinking.

Cognitive Times: How do you and your team and your colleagues evaluate the progress of AI development, specifically to robotics lab? What are the benchmarks, the goals that you all are setting, and how are you all looking at the development of the technology?

Peter Stone: Well that is one of the beauties of RoboCup, the robot soccer competition, it is sort of a built in benchmark. You can see how your team is doing compared to the other teams around the world. You can see how it’s doing especially in the simulation leagues. You can see how it’s doing compared to last year’s teams.

Those are not the only benchmarks. There have been many challenge problems that have been proposed throughout the years. DARPA has been running grand challenges and challenge events for robotics and for autonomous cars over the years. Then there are landmarks of capabilities like the computer that beat the world champion at the game of Go Twenty years ago in chess. There have been benchmarks against people. There was recently a demonstration of a machine playing poker. There are lots of games that we can test against people’s performances. Then there are capabilities like the first time a robot can unload a dishwasher without breaking the dishes and stack them up. When can the first robot fold laundry? But I wouldn’t necessarily call them benchmarks as much as milestones.

Cognitive Times: Now, you are a soccer player, and you are a violinist. There are music competitions that are often much more subjectively scored than a sport that would be defined by a set of rules. What do you define as the ultimate milestone? Is it getting to the point where a machine can beat a human in a rules-based competition, or would it be the point where a machine can create music that brings you to tears?

Peter Stone: Beating a human at something is not a challenge anymore. Machines have been able to beat people at arithmetic for a long time and there are many things that computers are better at than people. I don’t think the goal is for computers or robots to be better than people at everything. I don’t think that’s ever likely to happen. I think that it’s better to think of machines as a different species, with different capabilities and different weaknesses. It would be great if there would be a computer that could create a work of art that would be able to bring people to tears. That would be a very nice achievement and milestone. But for me there are a lot of concrete research challenges, things that programs can’t do yet that we think they ought to do and ought to be able to do. How can we think about the algorithms that are required to make that happen? One of the exciting things about AI is that it’s a moving target in some sense. Once the field achieves a goal, the world sort of stops thinking of that goal as being an “AI thing”. There’s always the next thing. Some people define the field of AI as the science of getting the computers to do what they can’t do yet. And there is always something they can’t do yet.

“Beating a human at something is not a challenge anymore. Machines have been able to beat people at arithmetic for a long time and there are many things that computers are better at than people.”

That means that what you were working on ten years ago may not be current anymore. We are not at risk of running out of things to do, so I don’t think there is really one challenge. I am inspired by the RoboCup goal of trying to obtain that objective measure of being better than a human at soccer, but that won’t be the mark of all intelligence by any stretch.

Cognitive Times: So when we talk about that, there is a view around augmented intelligence that, while AI is getting better, there is the idea that human and machine combos can out perform machines by themselves. Do you see this as a short term situation where some point down the road AI will be able to out perform humans in all capacities, or do you think that human advantage will always outlast machines?

Peter Stone: I think there is always going to be things that are unique to humans. I think that’s a long term thing Collaborative systems between robots and people and how machines can augment people is something that’s going to be a reality for quite some time.

Cognitive Times: Turning to the University of Texas, what are the some other exciting areas that your colleagues are working on, or areas that you see as really advancing to the state of the art in robotics and AI?

Peter Stone: AI is a very broad field, we are actually right now growing in a very exciting way in a broad area that we are calling “machine perception.” It includes computer vision and we have one the world’s experts in computer vision, Kristen Grauman. We just hired a new young faculty member in computer vision.

Another aspect of perception is natural language understanding and Ray Mooney is one the world’s experts in natural language processing. There are also people who have been around in the department for quite some time working on AI in very important areas. Our Chair, Dr. Bruce Porter, has worked in the areas of knowledge representation and reasoning, and he has made some really big contributions that are still ongoing, exciting areas of AI. Some people are sort of pointing towards machine learning as being a thing that can solve everything. I don’t think that’s true. We need to stay grounded in some of the logical reasoning that Vladimir Lifschitz does, and in the knowledge representation and reasoning that Bruce Porter does. There are other forms of machine learning. Risto Miikkulainen works in a type of machine learning known as evolutionary computation. Dana Ballard is sort of an interface between psychology and machine learning. Then there is Adam Klivans who works in machine learning theory. I could keep going for a while here. I haven’t even talked about more statistical machine learning. AI is a very broad area.

In robotics, there are faculty who are doing very interesting work, sort of related to reinforcement learning, in human robot collaborative systems. They have created a system where the person and the robot worked collaboratively to assemble IKEA furniture, having the robot understand from demonstrations the steps of the tasks that the person was trying to demonstrate. Understanding the impact or the effects of actions in the world is something that many of us are sort of touching up.

Cognitive Times: That’s a lot happening on one campus. You’re highly regarded academic, you know what’s going on, not only in the US, but also globally, and your alma mater is Carnegie Mellon, one of the top schools in the country. You have a good sense of the industry and academic achievement in the field.

There are a number of companies that are focused on machine learning and artificial intelligence here in Austin. As you know our CEO, Amir Husain, has publicly stated our goal of transforming Austin into a hub of innovation for AI. You have a lot of individuals and organizations working to make this goal a reality. Do you see that as an achievable goal here in Austin given the trajectory that we are on now?

Peter Stone: Yeah, absolutely. In fact, I have personally recently founded a company with two colleagues that’s focused on exactly the kind of research that I do – learning about the continual effects of actions – continual learning. I have also talked to a lot of people in this area and I think we have a lot of expertise here. There are a lot of PhD students who have graduated from here. There are a lot of masters and undergraduate students who have some experience in artificial intelligence through our classes. I think Austin is one of the best tech hubs. We’ve got a lot of people with entrepreneurial experience and a lot of really deep technical experience. So yes, I think Austin should aim to be the capital in this area.

Cognitive Times: I know you touched on this specifically for your research group, but taking a macro view of the industry, what do you see as the next big challenge for AI research?

Peter Stone: There is not just one, there are many. There are challenges in computer vision, there are challenges in natural language understanding, moving toward more dialogue based systems. The challenges that I am focused on do pertain to reinforcement learning or understanding the effects of actions, being able to learn the effects of actions, and being able to take the sequential decision- making from that towards real world problems.

“The picture of AI in the media and in the press has always been that it’s a series of either huge successes and huge failures.”

Really, this is actually very important. The picture of AI in the media and in the press has always been that it’s a series of either huge successes and huge failures. Right now we are in an upswing and so AI is getting lots of positive attention. Inevitably, if history rings true, many of the promises that are being made won’t be delivered upon, and then there will be a big crash of disappointed, unmet expectations. Then there will be a trough. Then people will say, “Oh no it’s better than that”. There’s a view that it is all happening in these waves, when really the reality of AI is that, like any other science, it’s been a gradual and steady process over the past fifty to sixty years. I think that’s going to continue to happen. There will be incremental progress that needs to be made in many different areas from robotics to natural language to knowledge representation. Gradually, as the technologies keep maturing, they can all come together and every once in a while there will be a confluence of data, computation, and techniques that get us to a new level on some benchmark. The public’s perception is often that these breakthroughs are sudden, but really they are the result of decades of gradual progress. That’s going to continue to happen from the long term perspective of people who have been working in the field for many years. I think that’s the way it has always been, and that’s the way it will continue to be.

Cognitive Times: Dr. Stone, thank you very much for having us, we really appreciate your time, and best of luck in the upcoming RoboCup.

If you’d like to learn more about how Artificial Intelligence is being applied in sports, check out AI 3.0: Are We Closer to Creating the Mind of Tom Brady?