Plan vs Reality Part 3 - The Challenges

Tidy Up The Toys

We thought this would be the hardest challenge to write the software for and we weren't wrong. 

As I mentioned in the original blog I had already developed some computer vision software in Python to detect coloured blocks, although the performance was pretty lousy. 

However, we continued to persevere with this, and managed to do a successful trial run of "Tidy The Toys" using this in December last year. The delay to the competition gave me time to revisit this and rewrite it as a 'C' function that could be called from Python. I wrote about this in a previous blog entry. This fixed all the performance issues.

I also wrote some computer vision software to detect the 'horizon' where the arena wall meets the floor. By keeping this horizontal we were able to make sure the robot always travelled in a straight line, perpendicular to the walls. Using this technique together with the distance sensor, we could move to the ideal points in the arena to line up the blocks. 

We had already constructed a prototype block grabber using a Lego compatible servo motor to control the gripper arms and a stepper motor to lift the whole thing to a specific height. 

The servos are great for connecting to Lego - but the spindle doesn't line up with the Lego Technics holes, so trying to fit cogs is not ideal. For the final design we used 2 of these servos and connected the gripper arms directly to the servo spindles. This made them a little weak, but was a much simpler design. A bit of self adhesive insulation foam stuck to the inside of the gripper arms helped lift the blocks. 

We were planning to use a smaller stepper motor than the heavy NEMA 17 stepper we built the prototype with, but because it worked very well we stuck with the big one - the wheel motors seemed to cope with the weight OK. 

Nanny McPi with the forklift attachment fitted. 

Feed The Fish

Looking back at my original blog entry we were very brief on the plan because we didn't have a clue how to do it. In fact the only part of the plan that became reality here is that we did use Nerf Rival balls as the fish food.

In the end we built a rubber band powered fish food cannon out of plywood to propel all 5 Nerf balls into towards the bowl at the same time. An electro magnet was used to release the cannon. We used the same horizon and colour recognition software to travel to the right position and find the fish bowl. 

I wrote about this in more detail in a fairly recent blog entry so won't go over it again again but here's a picture. 

Nanny McPi with the fish food cannon fitted.

Up The Garden Path

We had two choices for this - either use the camera to follow the line with voice control at the junctions - or just use voice control to guide the robot around the course. 

Happily we managed to go with the 1st option for maximum points. I just needed to write some line recognition software for the camera, but that was supprisingly straightforward once the image was converted to black and white. The robot doesn't try to navigate the junctions - but it can detect them and will stop and wait for voice commands to continue. Here we used the eyes to follow the course of the line and also turn red and cross-eyed when it lost the line or detected a junction. We hadn't planned to actually make much use of the eyes in the challenges themselves - just to change colour to indicate what autonomous mode is selected - so this was quite entertaining, and perhaps we should have used that more. 

As Svetlana is doing a part time machine learning degree course we had hoped to write and train our own models for voice recognition. However the course, together with her full time job, takes enough of her time, so in the end we used the Vosk voice recognition module after watching Neil Stevenson's video at the Pi Wars 2021 Conference. 

And we did use a bluetooth headset for the audio - this worked very well. 

However because we were marking out the course directly onto the kitchen floor, we didn't do a full course test until just before we filmed the challenge, so that meant some last minute tweaking to get it working - especially going round the long curves where we always lost the line due to our camera angle so had to just keep turning in the direction we last detected it. 

Conclusion

So all in all, where we did have a plan, we pretty much followed it with just a few tweaks, the biggest change probably being in the arena construction. 

And where we didn't have a plan originally - mainly with the fish food cannon - we did manage to get a design that actually worked. 

So we're pretty happy with how the robot and challenge builds went.

It's such a privilege to be accepted into the Pi Wars competition and we hope we've done justice to our original application. 

Many thanks to Mike, Tim and David for giving us this great opportunity in a difficult year.