Robotics & Engineering - Week of 12/01/25
This week during my time in shop, I began a new task on the Ford Think… AI.
Jonas has been focused on the LiDAR aspect of the autonomous navigation system, and I have been focused on the camera system
The idea of doing a combination of both LiDAR and camera’s as inputs to our final AI system we think has significant advantages over simply using cameras as the inputs, like Tesla does on their vehicles.
Not only are computer vision systems that can do real-time object detection and range finding extremeley expensive and complex, but they are also difficult to interface and utilize in a closed-loop system (e.g. our autonomous car)
With combined sensor fusion, we are able to create a more effective, safe, and likely easier way for the vehicle to autonomously navigate and avoid threats in real-time.
To begin with a super simple base, I decided to train my own YOLO (You Only Look Once) model (YOLO model “weights” really), a standardized computer vision program. The idea is to use the camera feeds to identify things like people, crosswalks, bicycles, and signs (stop, slow, speed limit, etc.) This can all be accomplished with a well trained and tuned YOLO model.
On Monday, I got the main dashboard fully installed and wired up. It almost works great!
On Tuesday, I did some fine tuning (voltage reference was completely wrong on the dashboard) and ultimately got the voltage readings to be within +/- 0.05 volts, which is fine for me. The only issue is that the reading doesn’t seem to change even under load, which contradicts what the multimeter says.
Originally I thought this was because of the fact that it averages several readings, but after doing some investigating, the code takes samples over the course of a few hundred milliseconds, which should have no significant effect on the reading. Nonetheless, I removed it to no avail. This is still an ongoing problem I will have to look into down the road. For now, we are stuck with the multimeter.
On Wednesday, Jonas and I decided to begin working on the AI. But it is hard to train when you have nothing to train on! We needed real photos of the environment the golf cart would be autonomously navigating. So we set out with a USB webcam duct taped to the roof, plugged into a laptop and drove around and ultimately collected over 1000 photos of various positions and scenarios the golf cart would likely be put into.
Here’s just a sneak peek at a few of the samples we took.
This sample contains photos of people, signs, and vehicles.
The photos are of terrible quality. This is by no means our final setup nor our final trained model for this. I really just wanted to make a proof of concept and get an idea to see how precise we can be with our messy YOLO setup. It will give us a bar to see what we need to improve upon.
In order to train the model, we needed to annotate the photos manually by dragging boxes over the regions containing the thing we want to identify.
We basically have to do the work for it, and then it learns by example, and gets really good at identifying just those things.
The parameters fed into the model to train are as follows, and are represented in the following image:
So, I spent the rest of my Wednesday annotating a few hundred frames, and on Thursday, Jonas and I both worked on annotation for our time in shop.
Thursday, I also trained our first model, but this is where issues began to show up.
When Jonas and I were both annotating, we had different indexes set for our identifications. For example, “car” on my annotations were all index 0, while for Jonas, they were index 1. This throws the model off, as it associates cars with both index 1 and 0, which realistically means it is training on index 0 (cars) and whatever index 1 is (likely stop sign, people, etc). This leads to a mistrained model that won’t perform well at all.
The results of the brief training can be seen in visualizations below:
Next week will focus on testing this model, and (more realistically) have a lot of retraining going on!
0
Views