Plan vs Reality Part 3 - The Challenges

Tidy Up The Toys

We thought this would be the hardest challenge to write the software for and we weren't wrong. 

As I mentioned in the original blog I had already developed some computer vision software in Python to detect coloured blocks, although the performance was pretty lousy. 

However, we continued to persevere with this, and managed to do a successful trial run of "Tidy The Toys" using this in December last year. The delay to the competition gave me time to revisit this and rewrite it as a 'C' function that could be called from Python. I wrote about this in a previous blog entry. This fixed all the performance issues.

I also wrote some computer vision software to detect the 'horizon' where the arena wall meets the floor. By keeping this horizontal we were able to make sure the robot always travelled in a straight line, perpendicular to the walls. Using this technique together with the distance sensor, we could move to the ideal points in the arena to line up the blocks. 

We had already constructed a prototype block grabber using a Lego compatible servo motor to control the gripper arms and a stepper motor to lift the whole thing to a specific height. 

The servos are great for connecting to Lego - but the spindle doesn't line up with the Lego Technics holes, so trying to fit cogs is not ideal. For the final design we used 2 of these servos and connected the gripper arms directly to the servo spindles. This made them a little weak, but was a much simpler design. A bit of self adhesive insulation foam stuck to the inside of the gripper arms helped lift the blocks. 

We were planning to use a smaller stepper motor than the heavy NEMA 17 stepper we built the prototype with, but because it worked very well we stuck with the big one - the wheel motors seemed to cope with the weight OK. 

Nanny McPi with the forklift attachment fitted. 

Feed The Fish

Looking back at my original blog entry we were very brief on the plan because we didn't have a clue how to do it. In fact the only part of the plan that became reality here is that we did use Nerf Rival balls as the fish food.

In the end we built a rubber band powered fish food cannon out of plywood to propel all 5 Nerf balls into towards the bowl at the same time. An electro magnet was used to release the cannon. We used the same horizon and colour recognition software to travel to the right position and find the fish bowl. 

I wrote about this in more detail in a fairly recent blog entry so won't go over it again again but here's a picture. 

Nanny McPi with the fish food cannon fitted.

Up The Garden Path

We had two choices for this - either use the camera to follow the line with voice control at the junctions - or just use voice control to guide the robot around the course. 

Happily we managed to go with the 1st option for maximum points. I just needed to write some line recognition software for the camera, but that was supprisingly straightforward once the image was converted to black and white. The robot doesn't try to navigate the junctions - but it can detect them and will stop and wait for voice commands to continue. Here we used the eyes to follow the course of the line and also turn red and cross-eyed when it lost the line or detected a junction. We hadn't planned to actually make much use of the eyes in the challenges themselves - just to change colour to indicate what autonomous mode is selected - so this was quite entertaining, and perhaps we should have used that more. 

As Svetlana is doing a part time machine learning degree course we had hoped to write and train our own models for voice recognition. However the course, together with her full time job, takes enough of her time, so in the end we used the Vosk voice recognition module after watching Neil Stevenson's video at the Pi Wars 2021 Conference. 

And we did use a bluetooth headset for the audio - this worked very well. 

However because we were marking out the course directly onto the kitchen floor, we didn't do a full course test until just before we filmed the challenge, so that meant some last minute tweaking to get it working - especially going round the long curves where we always lost the line due to our camera angle so had to just keep turning in the direction we last detected it. 

Conclusion

So all in all, where we did have a plan, we pretty much followed it with just a few tweaks, the biggest change probably being in the arena construction. 

And where we didn't have a plan originally - mainly with the fish food cannon - we did manage to get a design that actually worked. 

So we're pretty happy with how the robot and challenge builds went.

It's such a privilege to be accepted into the Pi Wars competition and we hope we've done justice to our original application. 

Many thanks to Mike, Tim and David for giving us this great opportunity in a difficult year. 

  

Plan vs Reality Part 2 - Building the Courses

The Arena

The original plan was to build a foldable solid arena from wood painted matte black with a white hardwood floor. 

However, the final arena was built from plastic "Foamex" sheets cut to size from an online company: http://cutplasticsheeting.co.uk. This actually worked out a similar price to the wood we would need and it came in matte black, so there was no painting involved. We stuck it together with tape and Velcro to form the arena. 

Rather than build a hardwood floor, we stuck to using the kitchen floor and marked out the challenge courses with tape before filming.

This worked well - although I think it would have been better to build something more solid.  

The arena marked out for the "Up The Garden Path" challenge. 

Tidy The Toys

We bought some 5cm wooden cubes from eBay and covered them in coloured tape. I thought this would be better than paint - but was wrong - the colours were very patchy where the tape stretched or overlapped. However the biggest problem was the weight - they were too heavy for our gripper to pick up consistently without slipping. 

Tape Covered Solid Wooden Cubes.

So we decided to make some from ply wood. These were much lighter and the templates I had printed out for cutting the blocks were the right colour for detecting.

ply-wood cut cubes. 

However - as we were testing in the kitchen, occasionally there would be small drops of water on the floor that caused the ink to run and they soon looked a mess. So we took some thin coloured cardboard sheets and used it to cover the blocks. These looked much better and the solid colours worked well for our colour recognition. 

Cardboard covered cubes with colour detection.

These blocks were the ones we used for the final filming of the challenge.

Feed The Fish

The original blog was very brief over our plans for building the fish bowl. I mentioned that I was looking for any cardboard boxes that were the right size for building it. 

Well, as luck would have it, I received a pizza oven for my Birthday at the end of November and, like a kid who's more excited about the box than the toy inside, I knew the box would be just right for building the fish bowl. The pizza oven was pretty good too - pizzas + robots - is there anything better? 

The plain cardboard fish bowl was used for testing, but before we filmed the challenge we covered it in white paper so the boys could decorate it. They painted sand, water and kelp using watercolour paints and stuck on some real shells. They also drew some Pokemon fish that they stuck on. Because our camera could only see the bottom half of the box, we made a 'no golf balls' sign to go over the bottom opening, and our camera targeted the yellow 'sand' at the bottom of that when aiming. 

 

The Decorated Fish Bowl

Obstacle Course

Back in November last year when I wrote the original blog, we didn't really have any ideas for the obstacle course. 

We have a small lawn in our garden with a path running all the way round, so we thought that would be perfect for setting up the course, but didn't really have any ideas for the obstacles. 

As the months progressed, we came up with a few ideas that fit with the DIY theme and noted them down. 

These included: 

• "Pit of Nails" - a tray of nails and screws that the robot had to drive over.
• "Builder Tea Slalom" - mugs of very strong tea set up as a slalom obstacle. 
• "Wonky Shelves" - a series of pivoting ramps. 
• "Cement Mixer" - the robot has to enter a bucket and drive inside to move over obstacles. 
• "Sledgehammer Pendulum" - the robot has to drive through a swinging hammer. 

The bucket idea didn't really work - the robot couldn't drive in due to the curved sides, and if we did get it in, as soon as it hit any obstacle, the robot would drive up the inside of the bucket and fall upside down. We did keep a homage to this idea in the final run using a flexible plastic garden 'trug'. Because this is flexible the robot was able to drive in, but instead of trying to move it over obstacles, we just used the driving inside technique to get it out of the way of the path. 

For the 'Pit Of Nails' we built a tray with an offset entrance and exit - but instead of just nails and screws we also put in some tools for the robot to drive over. 

The "Terrible Tray of Tools"

The 'Wonky Shelves' idea turned into a 3 part flip ramp with the middle ramp higher than the outer two. This was the biggest thing to build in the obstacle course. 

The "Wonky Shelves"

The other ideas we implemented as planned - and added some off road driving and some paving slabs. 

We had always planned to do this challenge using remote control only our 11 year old could show off his driving skills. 

We realised late on that if we used voice commands, we could gain some extra points, but, with time running out to film the challenge, we kept to remote control only. 

 

Plan vs Reality Part 1 - The Robot

Now the videos are submitted and we await the results, I thought it would be a good idea to revisit the 3 part blog posts I made back in November 2020 where I talked about the plan for the robot, course and challenges.

The Robot 

The robot went pretty much to the original design. She was built from 3mm plywood cut using a scroll saw. The moving eyes worked well. We were trying to look a little more like the Nanny McPhee character, but it was difficult to source the right style of hair and hat, so in the end we went for a more stereotypical 'granny' look with grey hair and a pair of glasses (the glasses came with the wig!). We're very happy with the result.

The sonar sensor and camera are fitted in the nose as planned - these worked well - although we needed to bend the nose down for the "Up The Garden Path" challenge so the camera could see the line properly. It was also a little low for detecting the upper box in the "Feed The Fish Challenge" so we needed to make a sign for the lower box that the camera could detect. 

The sonar sensor worked very well - we did have one issue where, after working well for a long time, we then found when we were about to film, the distance sensing wasn't very accurate. After trying to fix it through software changes, in the end we just swapped out the sensor and it was all good. So not sure what happened there. 

The robot is controlled by a Raspberry Pi 4 and a Red Robotics Redboard+ motor controller - which, again, was always the plan. We were going to use a 2GB Pi but had issues with bluetooth when trying to connect the headset for voice commands. We swapped the SD card into another 4GB Pi we were using for something else, and it worked perfectly so there must have been something wrong with the bluetooth on the 2GB Pi. So that's why we're using the 4GB version even though we don't really need the extra memory. 

We did stick with these 12V motors: https://www.technobotsonline.com/918d100112-1-re-280-1-4mm-metal-geared-motor-100-1.html

If I'm honest they're a little disappointing. Running at 12V they're a similar speed to the 6V Pololu ones we used in Sputnik (running at around 8V), but have less torque. This means that when stopping they have a habit of allowing the robot to run on, rather than stopping dead. Also they didn't run very well at slow speeds. This meant it was harder to turn the robot by slowing down one side, as the faster side tended to just force it to keep going straight.

I think maybe we were a little spoilt by the excellent Pololu motors we had used previously, and I think it's probably worth investing the extra money on better motors. We spent a lot more time trying to get the robot turning slightly left and right for the automated tasks, and I think this would have been saved had we had better motors.

Having said that, they were good value for money and we were able to get running with them straight away (unlike Sputnik where we went through 4 different sets of motors trying to find ones with the torque and speed we wanted).

So all in all, the robot pretty much went to plan, and matches what we said we were going to build in the Pi Wars application.

Flying Fish Food

We've filmed our final arena challenge - Feed The Fish. This was always the one that troubled us the most as we really weren't sure our fish food cannon could do the job. 

Nanny McPi's Fish Cannon Ready to Fire

This picture shows the final design viewed from the top. It's constructed from 3mm plywood (as per the rest of the robot) glued and bolted together to form a rubber band powered cannon. The plunger that holds the Nerf Balls fish food is pulled back against the rubber bands and held in place with an electro magnet. When the electro magnet is disabled, the cannon fires. 

Up The Garden Path

So - we've managed to film the 'Up The Garden Path' challenge. It took a couple of days to get a good run which, considering we've never tested this challenge in full before, was quite something. 

The Challenges of Filming The Challenges

We've pretty much completed the robot build - and the code is mostly written, so now onto filming the challenges for submission to the competition. 

This is the easy part - or so we thought. Hmmm..... 

Challenge 1 - Framing the arena

The arena design went through many iterations. Initially it was going to be a wooden construction, complete with an MDF or plywood floor that we could put up anywhere. 

With Great Power ...

I've worked on UNIX based systems since 1989, and I've always had a healthy respect for accessing the system as super user. 

I've seen many times where mistakes made as superuser can have devastating consequences. 

So when we added LEDs to the robot, I was a little concerned to find that the neopixel library required root access in order to run.