[ad_1]
Starship is building a fleet of robots to deliver packages locally on demand. In order to successfully achieve this goal, robots must be safe, polite and fast. But how to achieve this goal with lower computing resources and expensive sensors (such as LIDAR)? This is an engineering reality problem you need to solve, unless you live in a world where customers are willing to pay $100 for delivery.
First, the robot first perceives the world through radar, a large number of cameras and ultrasound.
However, the challenge is that most of this knowledge is low-level and non-semantic. For example, a robot may feel an object ten meters away, but it is difficult to make safe driving decisions without knowing the type of the object.
Machine learning through neural networks is very useful in transforming this unstructured low-level data into high-level information.
Starship robots mostly drive on sidewalks and crossing roads when needed. Compared with self-driving cars, this brings a series of different challenges. Traffic on car roads is more structured and predictable. Cars drive along the lane and do not change direction often, while humans often stop suddenly, meandering, can be accompanied by a dog on a leash, and do not use turn signals to indicate their intentions.
In order to understand the surrounding environment in real time, the core component of the robot is the object detection module-a program that inputs an image and returns a list of object boxes.
All this is fine, but how do you write such a program?
The image is a large three-dimensional array of countless numbers representing the intensity of the pixels. These values change significantly when the image is taken at night instead of during the day; when the color, proportion, or position of the subject changes, or when the subject itself is cut off or blocked.
For some complex problems, teaching is more natural than programming.
In robotic software, we have a set of trainable units, mainly neural networks, where the code is written by the model itself. The program is represented by a set of weights.
At first, these numbers are initialized randomly, and the output of the program is also random. The engineers showed examples of models they wanted to predict and asked the network to get better the next time it saw similar inputs. By iteratively changing the weights, the optimization algorithm searches for programs that can predict the bounding box more and more accurately.
However, you need to think deeply about the examples used to train the model.
- When the model detects a car in a window reflection, should it be punished or rewarded?
- What will it do when it detects a picture of a person from the poster?
- Should a trailer full of cars be annotated as an entity, or should each car be annotated separately?
These are all examples of what happens when building an object detection module in our robot.
When teaching machines, big data alone is not enough. The data collected must be rich and diverse. For example, using only uniformly sampled images and then annotating them will show many pedestrians and cars, but the model will lack examples of motorcycles or skaters to reliably detect these categories.
The team needs to dig out difficult examples and rare situations, otherwise the model will not work. Starship operates in several different countries, and different weather conditions enrich this set of examples. Many people were surprised when the Starship delivery robot was operating in a snowstorm ‘Emma’ in England, However, the airport and schools are still closed.
At the same time, annotating data requires time and resources. Ideally, it is best to use less data to train and enhance the model. This is where construction works come into play. We encode prior knowledge into the architecture and optimization process to reduce the search space to programs that are more likely to appear in the real world.
In some computer vision applications, such as pixel-by-pixel segmentation, it is useful for the model to know whether the robot is on the sidewalk or at a road intersection. To provide hints, we encode global image-level clues into the neural network architecture; then the model decides whether to use it without learning from scratch.
After data and architecture engineering, the model may work well. However, the deep learning model requires a lot of computing power, which is a big challenge for the team because we cannot use the most powerful graphics card on the battery-powered low-cost delivery robot.
Starship wants our delivery costs to be low, which means our hardware must be cheap. This is why Starship does not use LIDAR (a detection system that works on the principle of radar, but uses light from lasers), which will make it easier to understand the world-but we don’t want our customers to pay more than They need to be delivered.
The most advanced object detection system published in academic papers runs about 5 frames per second [MaskRCNN], And the real-time object detection paper did not report a significant rate exceeding 100 FPS [Light-Head R-CNN, tiny-YOLO, tiny-DSOD]More importantly, these numbers are reported on a picture; however, we need a 360-degree understanding (equivalent to processing about 5 single images).
To provide a perspective, the Starship model runs at more than 2000 FPS when measured on a consumer-grade GPU, and processes a complete 360-degree panoramic image in one forward pass. This is equivalent to 10,000 FPS when processing 5 single images with a batch size of 1.
Neural networks are better than humans on many vision problems, although they may still contain errors.For example, a bounding box may be too wide, the confidence level is too low, or an object may be Hallucinations In a place that is actually empty.
Fixing these errors is challenging.
Neural networks are considered a black box that is difficult to analyze and understand. However, in order to improve the model, engineers need to understand the failure cases and delve into the details of what the model learns.
The model is represented by a set of weights and can visualize what each specific neuron is trying to detect. For example, the first layer of the Starship network is activated in standard modes, such as horizontal and vertical edges. The next layer detects more complex textures, while the higher layer detects car parts and complete objects.
Technical debt has another meaning in machine learning models. Engineers continue to improve the architecture, optimize processes and data sets. As a result, the model becomes more accurate. However, changing the detection model to a better model does not necessarily guarantee the success of the robot’s overall behavior.
There are dozens of components that use the output of the object detection model, and each component needs a different accuracy and recall level set based on the existing model. However, the new model may function in different ways. For example, the output probability distribution may be biased towards larger or wider values. Even if the average performance is better, it may be worse for a specific group like a large car. To avoid these obstacles, the team calibrated the probabilities and checked the regression of multiple hierarchical data sets.
Compared with monitoring standard software, monitoring trainable software components brings a series of different challenges. Little attention is paid to inference time or memory usage, as they are mostly constant.
However, the transfer of data sets has become a major problem-the data distribution used to train the model is different from the data distribution of the currently deployed model.
For example, suddenly there may be an electric scooter driving on the sidewalk. If the model does not consider this class, it will be difficult for the model to classify it correctly. The information from the object detection module will be inconsistent with other sensory information, resulting in the need for the help of a human operator, thereby slowing down the delivery speed.
Neural networks enable Starship robots to be safe at road intersections by avoiding obstacles such as cars, and to be safe on sidewalks by knowing all the different directions that humans and other obstacles can choose.
Starship robots achieve this goal by using cheap hardware, which brings many engineering challenges, but makes robot delivery a reality today. Starship’s robots deliver real deliveries in multiple cities around the world 7 days a week. It’s worth seeing how our technology continues to bring more convenience to people’s lives.
[ad_2]
Source link
