For Tech United, developing advanced robotics is all part of an exciting game
Sydney, 7 July 2019. On the day that the Dutch national women’s football team, the Orange Lionesses, played their World Cup final against the USA, Tech United’s football robots line up against China to battle for the World Cup. The score is still 3 – 3 two minutes before the end of normal playing time. Suddenly there is a resounding “Nooo!” from the sideline: China has scored. Was a year of backbreaking hard work about to end in defeat?
Because of delays earlier in the game, the referee allows twenty seconds of extra playing time. Out of the blue, Lieke Motors darts into a gap in front of the goal. She fails to control the ball at first, but then she steadies it and scores! Back to a draw, and that means extra time. Anything is possible again…
Everything hinged on that one game
“That final was incredibly nerve-wracking, it took years off our lives”, says Wouter Kuijpers in retrospect. He is the team’s coach and a PhD researcher in mechanical engineering at Eindhoven University of Technology. “We spent a year working towards this moment as a team and even found ourselves programming into the early hours of the morning during the World Cup itself. But, ultimately, everything hinged on that one game.” You would have thought that the programmers would know exactly what to expect of their robots. Even so, they often follow the antics of Lieke Motors, Jackie Groenestroom and the four other metal ladies in the team with bated breath.
Autonomous robots
The football robots fielded by Tech United for the RoboCup world championship play completely autonomously: as soon as the referee sounds his whistle, they are on their own and the team can only watch from the sideline. And when a human team member has to venture onto the field - if a robot has taken a hard knock and is temporarily out of action, for example - a time penalty is imposed. So the real work takes place off the field and, other than software development, focuses mainly on the mechanical engineering aspects. Wouter: “That is an area where we particularly see opportunities for improvement.”
How the game drives innovation
And those improvements are a direct result of that bloodcurdling final in 2019. The Chinese robots were fast; Tech United's outfielders were hardly able to get past them. That will not happen to the team of students, PhD researchers and employees of TU/e again. So they are now working hard on a robot with eight wheels, rather than just three. A similar robot is also being tested in a number of hospitals. It will soon be used to move hospital beds around, a physically demanding task that is often too challenging for nurses. This is a good example of how the game drives innovation that can be applied in a variety of different sectors.
World modelling
Another example – this time in the field of software - is the base coordinate system, or world model (how the robots visualise their world), that has been developed to keep an eye on the opposing team and identify every single opportunity for ‘passing’ (building triangles) and scoring. This model is also being used in a project with commercial partners, including Vanderlande and Lely. The objective is to develop a robot that can navigate effortlessly through a space full of moving people, animals or other robots.
Tech United is one of the top favourites for the World Cup in Sydney. The football robots from Eindhoven have already reached the finals on eleven previous occasions and won the championship four times. But they get off to a bad start: the compass that the robots use to determine where they are turns out not to work in the spacious indoor stadium where the tournament is being played. With just two days to go until the first match, they feverishly look for an alternative. The first match – against an easy opponent – even has to be abandoned because the ladies have completely lost their way and stray outside the field all the time.
Good communication between the robots
“We found a rather mundane solution for that particular problem in the end”, says Wouter. “Simply by making sure that the robots faced the opposing team's goal at the beginning of the game. From that time on, they could keep track of their movements and turns. However that ‘simple’ solution led to drastic changes in the software. We only got everything working again properly after four matches, so we had a lot of catching up to do.” According to Wouter, the fact that they succeeded and ultimately won the final against China, beating them 6 – 4, is mainly due to “our ability to pass accurately and quickly and create space on the field.” That is mainly a question of good communication between the robots. How does that work? Wouter: “During demonstrations at schools, we say that the robots are all in the same WhatsApp group and continuously send each other messages saying: ‘I’m over here’. They also continuously share what they see with each other: the ball, the opponents, the other players on their own team, etc.”
Sport at the highest level
But it is not the robots who are competing at the highest level here. For a long and strenuous week, the team members spend twelve and sometimes even fourteen hours a day programming their systems. “That’s because of the differences between the field and the opponents”, says Wouter. “Our field at the TU/e is much smaller and there are no opponents. So something that works perfectly at the TU/e, suddenly stops working altogether during a major tournament like this. And then you have to make all kinds of changes. In addition, we constantly come up with new ideas during the competition. We save a copy of all the world models associated with a game and then review them: where did things go wrong and why, and how can we solve the problem? For example, we saw that our robots were often unable to pass because the Chinese robots are programmed to stay close to each other. So we asked ourselves whether we could pass between them. After a couple of evenings spent programming until late at night, we had a working solution. And then another idea comes into your head…”
Open source competition
A few months after the tournament, all the teams get together and share their data with each other. Sitting on your laurels is impossible: the organisation changes the rules of the game every year and makes the requirements even more challenging. This forces the teams to innovate continuously. The ultimate goal is incredibly ambitious: they want to develop autonomous robots that will be capable of running rings around the FIFA world champions in 2050.
Sydney, July 2019. A completely different game is being played at the other end of the RoboCup indoor stadium. This is where Tech United has entered Hero, an autonomous care robot, for a variety of domestic challenges. Ranging from ‘take out the garbage’ to ‘clear the groceries away’. The atmosphere is just as tense though. Everything is going smoothly but let’s not cheer too soon. Remember what happened in 2018!
Care robots Amigo, Sergio and Hero
“In that year, we participated with Amigo and Sergio, two care robots that we had built ourselves,” says Josja Geijsberts, team captain and a recently graduated mechanical engineer. “We entered them eight times in the Open Standard League, which is open to any type of robot. Things went really well until just before the final. That was when Amigo’s gripper broke off when opening the dishwasher. Zero points, no place in the final, no shiny trophy at the end of it. Even though our software was working perfectly.”
The team entered Hero for the second time in the Domestic Standard League in 2019. In this league, everybody has the same robot and the main focus is your software.
Lightening the burden on nurses and carers
Like many of her teammates, Josja came to Tech United via a bachelor degree project. “I started with the footballers, but transferred to the care robots after a while. There is more opportunity for creativity here. When working on the football robots, you have to adhere to all kinds of strict rules and write extremely efficient code; one millisecond can mean the difference between winning and losing. In the case of the care robots, robustness is the main priority. After all, the idea is that they can be used to lighten the burden on nurses and carers in the near future. We are already involved in several experiments in care institutions with Hero. My dream is that every department in these institutions will soon have a care robot and be able to deploy either a robot or a human carer, depending on the client’s preference. That will give human carers more time for personal contact.”
Robots need to be able to cope with unexpected situations
The tasks that the care robots have to perform at the World Championships may sound simple, but there’s a lot involved. Take ‘receiving visitors’ as an example. The robot needs to know where it is at all times (localisation) and has to be able to make its way to a destination without colliding with anything, or find an alternative route if it encounters an obstruction (navigation). To achieve this, it is equipped with a laser in its ‘foot’ and a 3D camera as its eye. In addition, it needs to be able to engage in conversation with the guests (by recognising what they are saying and giving a relevant answer), detect an empty place on the bench so that it can assign people a seat, bring each guest the right drink, and so on.
In this competition as well, as soon as the robot enters the ‘arena’, the team is no longer allowed to help in any way. Even though the smallest unexpected variation in circumstances can mean the end of the competition: an item of furniture that is slightly out of place, a door that is ajar, a noise in the indoor stadium. Josja: “That’s what makes these competitions so important. If you want to create the robustness required in an unstructured human environment, the robot needs to be able to handle unexpected variations effectively.”
If anything goes wrong, the team member responsible for that particular task often knows exactly what change needs to be made to the code (file x, line 37!). You are allowed to use a joker and repeat the task twice. That’s when the whole team holds its breath in suspense: will Hero do a better job this time? In Sydney, very little went wrong. However, with the 2018 debacle still fresh in their minds, the team never even considers the possibility of winning until just before the end.
I'm sitting on the bench and would like a can of Coke
The final is a task of their choice. Right up Tech United's street. “This was an opportunity for us to show off a number of ground-breaking developments that had not been put to the test during the fixed tasks”, says Josja. “For example, you can send a message to Hero from your phone: ‘I’m sitting on the bench and would like a can of Coke’, and then he will bring you a Coke. You can also send a message saying: ‘I’m looking for a girl wearing a red shirt, roughly 25 years old’. And then he will say: ‘I expect you are looking for Josja’. This is possible because we have developed personal recognition software that allows the robot to ‘stick’ labels to people, such as age, gender and the colour of your shirt.”
Other teams did not have many extra features to show off
These innovations and the glass of beer that Hero served to the judges earned Tech United's care robot team the world title. Josja: “We knew that if we could just get into the final, we had a good chance of winning because the other teams didn’t have much in the way of extras that they could show off.” In her opinion, the team's victory was above all a reward for all the effort the team has put into keeping it software generic during the past 10 years. “We have always kept in mind that you are likely to change your hardware. So everything that we developed during all those years for Sergio and Amigo can also be used for Hero – even though he operates in a very different way.”
Both teams are already looking forward to the next world championship in 2021. Robotics development is hugely rewarding. But returning home as the world champion is even better.
Last amended 11 November 2020.