Nanny McPi - Results and Videos

We had a great weekend watching many robots competing in the Pi Wars @Home 2021 live stream. 

Well done to Mike, Tim and Dave for managing to switch between live feeds, recorded videos and remote locations for live Pi Noon without too many hitches.  

And also well done for Brian at Micro Pi Noon HQ for keeping the robots going and connected and giving us a taste of live robot competition at last. 

So - when the results came in on Sunday afternoon - we sat with baited breath. We thought we had done well and were hoping for a top three position - but were blown away as the challenge results started coming through.

So here are Nanny's results and videos for the Sunday competition in the intermediate category: 

Feed The Fish - 2nd

• 1st M.E.T. Martian Emergency Team
• 2nd Nanny McPi
• 3rd Forest Fighters

Whilst we only managed to score 10 hits, doing this autonomously gave us more points and put us in second place. 

It doesn't seem fair in some way seeing the effort put in trying to get 15 golf balls to go into the bottom of the fish bowl by remote control, but having said that we did spend many hours and rebuilds trying to create our fish food cannon. Before the filming we would have been happy to have scored 6 or 7 shots so were delighted to score 10. 

Watching the M.E.T. team score a perfect 15 with their ingenious cannon and reload mechanism - 2nd place was by far the best we could hope for. 

Up The Garden Path - 3rd

• 1st - Red Brick
• 2nd - Forest Fighters 
• 3rd - Nanny McPi 

We were happy with our run but guessed it wouldn't be the quickest. The Forest Fighters run was shown on the day and was very fast. The RedBrick robot run wasn't shown, but I had followed his progress on twitter and knew it would be very quick - so wasn't surprised  to see him win this challenge. 

Tidy The Toys - 1st

• 1st Nanny McPi
• 2nd CNM HackerSpace
• 3rd RedBrick

We were delighted to come first. I'm guessing by the other results that we were one of the few (if not only) competitors to attempt this autonomously in the intermediate category. 

This challenge was really the main one that appealed to me when applying for Pi Wars 2021. So to win it was great. However, there were some great automated solutions in the Beginners and Young Persons categories that would have easily beaten us, so perhaps we were a little fortunate. 

Obstacle Course - 3rd 

• 1st Jkpg Makers 
• 2nd Dumont Cybernetics
• 3rd Nanny McPi 

This one was a surprise. The Jkpg maker team from Sweden managed to do their run fully automated, so this gave them a nice haul of extra points.  

However for most teams - especially family teams, I guess, where young drivers are itching to show off their skills - remote control was the way to go. 

Having watched the RedBrick obstacle course on the live stream, I'm surprised that wasn't in the top 3 though. Perhaps I'm being self critical but I thought it was funnier, more challenging and more dangerous than ours and deserved to finish higher.

Personally I think the obstacle course should be left as a remote control driving challenge going forward. You need at least one event where roboteers have a chance to show off their driving skills and won't automatically be beaten by some clever programming.  Hopefully at the in person events this will be the case. 

Artistic and Technical Merit - 4th (joint) / 8th (joint) 

How you're supposed to rank such different robots by artistic and technical merit I don't know - I'm glad we didn't have to judge this one. 

We were happy with these results - we spent a bit of effort fitting a lovely mane of hair on Nanny, and making her eyes move and light up so were happy these were appreciated in the artistic merit, although underneath it was just a few bits of wood screwed together. 

Another surprise to be quite high up in the Technical Merit table - getting the same result as the M.E.T. team with their walking robot. Perhaps the image detection software I described in the video was taken into account as we had taken quite a minimal approach to the hardware. 

Blogging - 1st 

I don't need to post a link to the blog - as you're in it! 

I really enjoy the blogging aspect of the competition and was delighted to come first overall. I started the blog back in November 2020 and tried to post fairly regularly as we made progress. I'm a little guilty of only posting successful outcomes though - if something doesn't work I don't blog about it until we've solved the problem. 

I think this is an often neglected challenge - in fact it's shown as 'optional' in the list of challenges - but it has the same points weighting as all the other challenges - so can really have a big effect on the overall positions. 

In fact this post was only supposed to be a quick link to the results and videos, but I seem to have got carried away. 

Overall Placings - Intermediate - 1st

So - with all these good results we were delighted to find out we had won the intermediate category outright. 

Lots of cheering and high fives ensued - not sure what the neighbours thought? It's also taken some of the pain away from another result that came in much later on the same day that we don't like to talk about. 

Perhaps the nature of this year's event was skewed towards competitors who were able to complete the challenges autonomously. Because we were able to repeatedly run the challenges until they worked, the automated teams would always beat those using remote control.  

In live events the autonomous option would be much higher risk as it may not work in different conditions on the day and you have limited runs, so the remote controlled teams would be better placed to compete.

A Brilliant Competition

We had a great time building and programming Nanny McPi and hope she provided some entertainment on the live stream. We would once again like to thank all the organisers, judges sponsors and anyone who helped to make Pi Wars 2021 a reality. 

Hopefully we can all meet up in person next year. 

Plan vs Reality Part 3 - The Challenges

Tidy Up The Toys

We thought this would be the hardest challenge to write the software for and we weren't wrong. 

As I mentioned in the original blog I had already developed some computer vision software in Python to detect coloured blocks, although the performance was pretty lousy. 

However, we continued to persevere with this, and managed to do a successful trial run of "Tidy The Toys" using this in December last year. The delay to the competition gave me time to revisit this and rewrite it as a 'C' function that could be called from Python. I wrote about this in a previous blog entry. This fixed all the performance issues.

I also wrote some computer vision software to detect the 'horizon' where the arena wall meets the floor. By keeping this horizontal we were able to make sure the robot always travelled in a straight line, perpendicular to the walls. Using this technique together with the distance sensor, we could move to the ideal points in the arena to line up the blocks. 

We had already constructed a prototype block grabber using a Lego compatible servo motor to control the gripper arms and a stepper motor to lift the whole thing to a specific height. 

The servos are great for connecting to Lego - but the spindle doesn't line up with the Lego Technics holes, so trying to fit cogs is not ideal. For the final design we used 2 of these servos and connected the gripper arms directly to the servo spindles. This made them a little weak, but was a much simpler design. A bit of self adhesive insulation foam stuck to the inside of the gripper arms helped lift the blocks. 

We were planning to use a smaller stepper motor than the heavy NEMA 17 stepper we built the prototype with, but because it worked very well we stuck with the big one - the wheel motors seemed to cope with the weight OK. 

Nanny McPi with the forklift attachment fitted. 

Feed The Fish

Looking back at my original blog entry we were very brief on the plan because we didn't have a clue how to do it. In fact the only part of the plan that became reality here is that we did use Nerf Rival balls as the fish food.

In the end we built a rubber band powered fish food cannon out of plywood to propel all 5 Nerf balls into towards the bowl at the same time. An electro magnet was used to release the cannon. We used the same horizon and colour recognition software to travel to the right position and find the fish bowl. 

I wrote about this in more detail in a fairly recent blog entry so won't go over it again again but here's a picture. 

Nanny McPi with the fish food cannon fitted.

Up The Garden Path

We had two choices for this - either use the camera to follow the line with voice control at the junctions - or just use voice control to guide the robot around the course. 

Happily we managed to go with the 1st option for maximum points. I just needed to write some line recognition software for the camera, but that was supprisingly straightforward once the image was converted to black and white. The robot doesn't try to navigate the junctions - but it can detect them and will stop and wait for voice commands to continue. Here we used the eyes to follow the course of the line and also turn red and cross-eyed when it lost the line or detected a junction. We hadn't planned to actually make much use of the eyes in the challenges themselves - just to change colour to indicate what autonomous mode is selected - so this was quite entertaining, and perhaps we should have used that more. 

As Svetlana is doing a part time machine learning degree course we had hoped to write and train our own models for voice recognition. However the course, together with her full time job, takes enough of her time, so in the end we used the Vosk voice recognition module after watching Neil Stevenson's video at the Pi Wars 2021 Conference. 

And we did use a bluetooth headset for the audio - this worked very well. 

However because we were marking out the course directly onto the kitchen floor, we didn't do a full course test until just before we filmed the challenge, so that meant some last minute tweaking to get it working - especially going round the long curves where we always lost the line due to our camera angle so had to just keep turning in the direction we last detected it. 

Conclusion

So all in all, where we did have a plan, we pretty much followed it with just a few tweaks, the biggest change probably being in the arena construction. 

And where we didn't have a plan originally - mainly with the fish food cannon - we did manage to get a design that actually worked. 

So we're pretty happy with how the robot and challenge builds went.

It's such a privilege to be accepted into the Pi Wars competition and we hope we've done justice to our original application. 

Many thanks to Mike, Tim and David for giving us this great opportunity in a difficult year. 

  

Plan vs Reality Part 2 - Building the Courses

The Arena

The original plan was to build a foldable solid arena from wood painted matte black with a white hardwood floor. 

However, the final arena was built from plastic "Foamex" sheets cut to size from an online company: http://cutplasticsheeting.co.uk. This actually worked out a similar price to the wood we would need and it came in matte black, so there was no painting involved. We stuck it together with tape and Velcro to form the arena. 

Rather than build a hardwood floor, we stuck to using the kitchen floor and marked out the challenge courses with tape before filming.

This worked well - although I think it would have been better to build something more solid.  

The arena marked out for the "Up The Garden Path" challenge. 

Tidy The Toys

We bought some 5cm wooden cubes from eBay and covered them in coloured tape. I thought this would be better than paint - but was wrong - the colours were very patchy where the tape stretched or overlapped. However the biggest problem was the weight - they were too heavy for our gripper to pick up consistently without slipping. 

Tape Covered Solid Wooden Cubes.

So we decided to make some from ply wood. These were much lighter and the templates I had printed out for cutting the blocks were the right colour for detecting.

ply-wood cut cubes. 

However - as we were testing in the kitchen, occasionally there would be small drops of water on the floor that caused the ink to run and they soon looked a mess. So we took some thin coloured cardboard sheets and used it to cover the blocks. These looked much better and the solid colours worked well for our colour recognition. 

Cardboard covered cubes with colour detection.

These blocks were the ones we used for the final filming of the challenge.

Feed The Fish

The original blog was very brief over our plans for building the fish bowl. I mentioned that I was looking for any cardboard boxes that were the right size for building it. 

Well, as luck would have it, I received a pizza oven for my Birthday at the end of November and, like a kid who's more excited about the box than the toy inside, I knew the box would be just right for building the fish bowl. The pizza oven was pretty good too - pizzas + robots - is there anything better? 

The plain cardboard fish bowl was used for testing, but before we filmed the challenge we covered it in white paper so the boys could decorate it. They painted sand, water and kelp using watercolour paints and stuck on some real shells. They also drew some Pokemon fish that they stuck on. Because our camera could only see the bottom half of the box, we made a 'no golf balls' sign to go over the bottom opening, and our camera targeted the yellow 'sand' at the bottom of that when aiming. 

 

The Decorated Fish Bowl

Obstacle Course

Back in November last year when I wrote the original blog, we didn't really have any ideas for the obstacle course. 

We have a small lawn in our garden with a path running all the way round, so we thought that would be perfect for setting up the course, but didn't really have any ideas for the obstacles. 

As the months progressed, we came up with a few ideas that fit with the DIY theme and noted them down. 

These included: 

• "Pit of Nails" - a tray of nails and screws that the robot had to drive over.
• "Builder Tea Slalom" - mugs of very strong tea set up as a slalom obstacle. 
• "Wonky Shelves" - a series of pivoting ramps. 
• "Cement Mixer" - the robot has to enter a bucket and drive inside to move over obstacles. 
• "Sledgehammer Pendulum" - the robot has to drive through a swinging hammer. 

The bucket idea didn't really work - the robot couldn't drive in due to the curved sides, and if we did get it in, as soon as it hit any obstacle, the robot would drive up the inside of the bucket and fall upside down. We did keep a homage to this idea in the final run using a flexible plastic garden 'trug'. Because this is flexible the robot was able to drive in, but instead of trying to move it over obstacles, we just used the driving inside technique to get it out of the way of the path. 

For the 'Pit Of Nails' we built a tray with an offset entrance and exit - but instead of just nails and screws we also put in some tools for the robot to drive over. 

The "Terrible Tray of Tools"

The 'Wonky Shelves' idea turned into a 3 part flip ramp with the middle ramp higher than the outer two. This was the biggest thing to build in the obstacle course. 

The "Wonky Shelves"

The other ideas we implemented as planned - and added some off road driving and some paving slabs. 

We had always planned to do this challenge using remote control only our 11 year old could show off his driving skills. 

We realised late on that if we used voice commands, we could gain some extra points, but, with time running out to film the challenge, we kept to remote control only. 

 

Plan vs Reality Part 1 - The Robot

Now the videos are submitted and we await the results, I thought it would be a good idea to revisit the 3 part blog posts I made back in November 2020 where I talked about the plan for the robot, course and challenges.

The Robot 

The robot went pretty much to the original design. She was built from 3mm plywood cut using a scroll saw. The moving eyes worked well. We were trying to look a little more like the Nanny McPhee character, but it was difficult to source the right style of hair and hat, so in the end we went for a more stereotypical 'granny' look with grey hair and a pair of glasses (the glasses came with the wig!). We're very happy with the result.

The sonar sensor and camera are fitted in the nose as planned - these worked well - although we needed to bend the nose down for the "Up The Garden Path" challenge so the camera could see the line properly. It was also a little low for detecting the upper box in the "Feed The Fish Challenge" so we needed to make a sign for the lower box that the camera could detect. 

The sonar sensor worked very well - we did have one issue where, after working well for a long time, we then found when we were about to film, the distance sensing wasn't very accurate. After trying to fix it through software changes, in the end we just swapped out the sensor and it was all good. So not sure what happened there. 

The robot is controlled by a Raspberry Pi 4 and a Red Robotics Redboard+ motor controller - which, again, was always the plan. We were going to use a 2GB Pi but had issues with bluetooth when trying to connect the headset for voice commands. We swapped the SD card into another 4GB Pi we were using for something else, and it worked perfectly so there must have been something wrong with the bluetooth on the 2GB Pi. So that's why we're using the 4GB version even though we don't really need the extra memory. 

We did stick with these 12V motors: https://www.technobotsonline.com/918d100112-1-re-280-1-4mm-metal-geared-motor-100-1.html

If I'm honest they're a little disappointing. Running at 12V they're a similar speed to the 6V Pololu ones we used in Sputnik (running at around 8V), but have less torque. This means that when stopping they have a habit of allowing the robot to run on, rather than stopping dead. Also they didn't run very well at slow speeds. This meant it was harder to turn the robot by slowing down one side, as the faster side tended to just force it to keep going straight.

I think maybe we were a little spoilt by the excellent Pololu motors we had used previously, and I think it's probably worth investing the extra money on better motors. We spent a lot more time trying to get the robot turning slightly left and right for the automated tasks, and I think this would have been saved had we had better motors.

Having said that, they were good value for money and we were able to get running with them straight away (unlike Sputnik where we went through 4 different sets of motors trying to find ones with the torque and speed we wanted).

So all in all, the robot pretty much went to plan, and matches what we said we were going to build in the Pi Wars application.

Flying Fish Food

We've filmed our final arena challenge - Feed The Fish. This was always the one that troubled us the most as we really weren't sure our fish food cannon could do the job. 

Nanny McPi's Fish Cannon Ready to Fire

This picture shows the final design viewed from the top. It's constructed from 3mm plywood (as per the rest of the robot) glued and bolted together to form a rubber band powered cannon. The plunger that holds the Nerf Balls fish food is pulled back against the rubber bands and held in place with an electro magnet. When the electro magnet is disabled, the cannon fires. 

Up The Garden Path

So - we've managed to film the 'Up The Garden Path' challenge. It took a couple of days to get a good run which, considering we've never tested this challenge in full before, was quite something. 

The Challenges of Filming The Challenges

We've pretty much completed the robot build - and the code is mostly written, so now onto filming the challenges for submission to the competition. 

This is the easy part - or so we thought. Hmmm..... 

Challenge 1 - Framing the arena

The arena design went through many iterations. Initially it was going to be a wooden construction, complete with an MDF or plywood floor that we could put up anywhere.