Sensor Simulation

October 3, 2022

We are working on the verification of sensor technology utilizing simulation. In recent years, the utilization of various external sensors such as cameras, RADAR, GPS, and LiDAR has been increasing. Various sensors are being utilized on board, but it is expensive to install and verify sensors on operating ships, and it is difficult to verify sensor technology in advance of new buildings. By using a simulation environment, it is possible to verify the function, installation position, number of sensors and so on, without using operating ships, thereby reducing costs and shortening the verification cycle.

As one of our research examples, let us explain about the berth recognition and distance measurement by 3D simulation of LiDAR and a camera. In this case, the simulation environment was built using ROS and Gazebo, an open-source software. ROS is like an operating system for robots, which allows communication and data linkage between multiple software such as sensors, control, and recognition. Gazebo is a physics engine that can be combined with ROS to simulate interference and recognition between the robot model and other objects.

On the ROS and Gazebo, we built a simulation environment with a simple berth and ship model, which has a simple pitching added with a sine wave and a sensor model of LiDAR and a camera. The sensor model is realized using a standard simulator model and packages released by third parties, each with a view angle, data acquisition cycle, and effective field of view set to match the real product. These models are used to verify the LiDAR and camera models in the simulator.

*https://bitbucket.org/DataspeedInc/velodyne_simulator/src/master/

Sensor model of 3D LiDAR

The application of the LiDAR was verified by measuring the distance using a point cloud and measuring it against the berth. The point cloud acquired from LiDAR was clustered and the horizontal distance to the point cloud in front of the sensor was measured, confirming that the distance measurement was performed as designed.

Distance measurement by 3D LiDAR

In addition, a simple image recognition was performed as a verification of the camera application. The object placed on the berth was recognized from the ship model, and detection was confirmed by the green frame line indicating the recognition area in the following figure.

Image recognition by the camera

These results show that simple verification of sensor models such as a camera and LiDAR can be performed on the simulator. By further building up the models and recognition system, it is expected to be more efficient to perform close to reality verification repeatedly at lower cost, and to verify the refined models at actual sites.

In the future, we plan to add complex motion and realistic motion models to the ship model in the above simulation environment, and further verification will include increasing recognition accuracy through sensor fusion and adding other sensors such as GPS.

(Writer: Keisuke Tanihara)