Lidarview with Livox HAP and integrated IMU

LidarView, Lidar, and SLAM
1

Greetings,
I recently acquired a Livox HAP LIDAR and I’m trying out Lidarview with it. I’ve encountered a few things that I don’t know if I’m doing wrong.
The version of the Livox HAP I have is the version with IMU, Lidarview seems to recognize it, but it doesn’t seem to record it or use it to do SLAM.
I’ve tried recording with wireshark and filtering the file to keep only the messages from the point cloud port and the IMU port. When loading it into Lidarview it also seems to recognize the IMU, but it still doesn’t seem to use it during SLAM. Also, in this case, when playing the file, in the information panel with “Raw information” selected, they don’t change, as if I wasn’t reading them.
Am I leaving something out of configuration correctly?

2

Hi @Alvaro,

Just to confirm, you can see the point cloud with LidarView and your recording?
The SLAM doesn’t support yet to to take IMU data directly from LidarView, you have to save it first in a .csv and then reloaded it in the SLAM (more info here).

Would be possible for you to share the livox .pcap file? (wireshard recording)

Best regards,
TImothée

3

Thank you very much for the answer.

The point cloud is displayed correctly, and SLAM can be done.

image
image1415×511 149 KB

I have tried to follow the steps indicated in the link you indicate. I export the IMU to csv, I change the name of the columns to those expected by Lidarview (acc_x/acc_y/acc_z, w_x/w_y/w_z and time), I create the calibration_external_sensor.mat file with the matrix data indicated by the manufacturer, but when I apply the changes the software closes.

I have performed all the tests with the Windows download of version 5.0 from your page

4

Hi @Alvaro,

Have you been able to sort out the issue since then ?
As we haven’t yet tested out on our side the IMU usage in SLAM with HAP sensor in LidarView, it would be great if you could share (privately if needed) some data.

Thanks and have a great day,

Gatien

5

Sorry, I’ve been away a couple of weeks. I’ve shared with Timothée Couble a file with IMU data. I’m sharing with you right now.
Feel free to ask for something specific if needed.

6

Hi Gatien,

I hope you’re doing well.

I wanted to follow up to check if you received the data I shared regarding the IMU usage in SLAM with the HAP sensor. Could you confirm if everything came through correctly? Also, I’d be interested to know if there has been any progress on your side or if there’s anything else I can provide to assist.

Thanks again for your support, and I look forward to hearing from you.

Best regards,
Álvaro

7

Hi @Alvaro,

We did receive correctly the data, thank you.
We do reproduce that crash issue, which happens in GTSAM (the dependency we use in the SLAM for IMU handling) when 2 consecutive timestamps are received with the same value in the IMU data, which happens here.
This is now tracked within this issue : https://gitlab.kitware.com/keu-computervision/slam/-/issues/71

On your data more specifically, it seems the acceleration values are scaled in a certain way (norm is close to 1, but not strictly equal, when we would expect an average norm of 9.81 if it contains gravity, and much smaller if not).
Do you know how it should be interpreted ?

Best,
Gatien

8

Greetings,
I find the Livox documentation quite confusing in some cases, but from what I have seen from other developments, it seems that the HAP IMU returns values ​​normalized to G. If Lidarview expects the acceleration in m/s^2 I will have to modify everything by multiplying by 9.8.

Is there a plan to update the HAP parser to read the values ​​directly from the .pcap file and save the steps of exporting and reordering to generate the csv?

I have also done tests with GNSS data to try to correct the SLAM trajectory, but I better open another thread about that to keep the answers organized

9

Hi,

LidarView does indeed expect the acceleration values to be in m/s^2.
Though, we could imagine implementing different variants to autocompute that directly in the filter.
The 3 variants I’ve seen so far ( but if anyone knows extra variants, it would be great to let us know here ):

  • Gravity compensated, and only the remaining ego acceleration present in the data
  • Raw data including the gravity. this requires the user to input the gravity direction in the SLAM filter
  • Raw data including the gravity, but normalized to g. Not directly handled as of now

A choice between variants could be added, and the data correctly interpreted in the filter to avoid this.

Implementing a direct parser from the sensor data to the SLAM filter is definitely possible.
Though, it currently had another limitation, which is data IMU data handling is not fast enough to be computed in real time in the SLAM algorithm ( which would be acceptable for replaying data, but not for streaming).
Runtime improvements are possible to reach that.

Note: If these developments are of interest of your project, and you would be willing to help funding it, feel free to reach out or to me directly ( gatien.ferret@kitware.com )

Best,
Gatien