mapping in python

[pascalisfort]

Hello, I have installed the latest ouster-sdk (0.9.0). I would like to use the mapping (SLAM) functions and OSF file conversion directly in Python. This way, I can customize these functions according to my needs.

Do you have any examples of Python scripts to help me?

Thank you
Pascal

• Ouster Sensor: OS-0
• Ouster Firmware Version: 2.3.0
• Programming Language: Python 3.9
• Operating System: Windows
• Machine Architecture: 64

[kairenw]

Hello!

How exciting! Anything you're looking to do in particular?

Barring any details, it might be useful for you to look at the code that powers the ouster-cli mapping features -- it's leveraged atop the kiss-icp library.

OSF is currently in a soft launch state and we are looking to add more documentation for it in the coming half a year.

[pascalisfort]

Yes we are a group of research in forest ecology who are using TLS. we are interested to build a point cloud (slam) with the ouster scanner and addind all channel (reflectivity, ambient, etc.) on a registered point cloud.

Thank you
Pascal

[yhao10]

Hello Pascal,

To generate a slam processed OSF file, you can execute the ouster-cli mapping slam command using either a live sensor feed or a recorded pcap file. I would recommend using a recorded pcap file for testing purposes. Here's an example command you can use:
ouster-cli mapping slam sample.pcap viz
The current OSF -> point cloud converter has a limitation: it can only store a single channel in the point cloud file. You can select a specific channel by using the -f or --field option. The available choices are SIGNAL, NEAR_IR, or REFLECTIVITY.

We intentionally do not save data from all channel fields to avoid doubling the file size, as an oversize file could lead to potential application crashes.

Is it acceptable for you to use the command provided above to test all available channels? Once you identify the most suitable channel, you can choose one of them as the desired output.

[pascalisfort]

Hello, Thank you for the advice you provided me. I used the command 'ouster-cli mapping slam sample.pcap viz', and it works wonderfully. However, I would like to go much further in using your various scripts from the Ouster SDK.
So, I took the file sdkx/mapping/mapping.py, removed all the click decorators(@click.argument), and named this file mapping_pascal.py. I wrote a simple code to call the 'run_slam' function.

import mapping_pascal as  mappeur
pcap ='C:\\Users\\Administrateur\\Documents\\ouster\\OS-0-128_v3.0.1_1024x10_20230216_172749-000.pcap'
json = 'C:\\Users\\Administrateur\\Documents\\ouster\\OS-0-128_v3.0.1_1024x10_20230216_172749.json'
mappeur.run_slam(source=pcap, viz='', meta=json, slam_name='kiss_slam', lidar_port= 7502, imu_port= 7503, output='out.osf')

After some processing time, I receive this error:
Traceback (most recent call last): File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\ouster\client\core.py", line 418, in __iter__ packet = next(it) File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\ouster\pcap\pcap.py", line 129, in __iter__ if not _pcap.next_packet_info(self._handle, packet_info): RuntimeError: ftell error: errno 22

I've checked and added a counter within the 'slam_scans_generator' function, and when it reaches scan 1013, that's when I receive the error. If I stop the loop at scan 1012, the OSF file is created correctly and can easily be converted to a PLY file.

def slam_scans_generator(scan_source, slam, osf_writer):
    aa= 0
    for scan in scan_source:
        if aa == 1012:
           break
        scan_slam = slam.update(scan)
        scan_ts = client.first_valid_column_ts(scan_slam)
        osf_writer.write_scan(scan_ts, scan_slam)
        yield scan_slam
        aa =aa +1
        print(aa)

    osf_writer.close()

Do you have any idea what might be causing this error? And how to resolve it?
Thank You
Pascal

[yhao10]

We can valid the source pcap file first
Could you run ouster-cli --traceback source OS-0-128_v3.0.1_1024x10_20230216_172749-000.pcap info and ouster-cli --traceback source OS-0-128_v3.0.1_1024x10_20230216_172749-000.pcap viz ?

[pascalisfort]

Hello, indeed I tested the file using the command line
ouster-cli --traceback source OS-0-128_v3.0.1_1024x10_20230216_172749-000.pcap info, and it stops at 84% with this error.

(ouster) C:\Users\Administrateur\Documents\ouster>ouster-cli --traceback source OS-0-128_v3.0.1_1024x10_20230216_172749-000.pcap info
Reading pcap:  [##############################------]   84%  00:00:00
ftell error: errno 22
Traceback (most recent call last):
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\ouster\cli\core\__init__.py", line 227, in run
    exit_code = cli.main(args=args, standalone_mode=False)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\ouster\cli\plugins\source.py", line 424, in invoke
    super().invoke(ctx)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\ouster\cli\plugins\source.py", line 242, in pcap_info
    ctx.forward(ouster.cli.core.pcap.pcap_info, *args, **kwargs)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\click\core.py", line 804, in forward
    return __self.invoke(__cmd, *args, **kwargs)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\ouster\cli\core\pcap.py", line 96, in pcap_info
    all_infos = pcap._packet_info_stream(file, n, progress_callback, 100)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\ouster\pcap\pcap.py", line 32, in _packet_info_stream
    result = _pcap.get_stream_info(path, progress_callback, callback_frequency, n_packets)
RuntimeError: ftell error: errno 22

This is a sample file downloaded directly from the website (https://ouster.com/resources/lidar-sample-data/os-sample-data/). So, I downloaded the other sample files from the OS0 sensor as well. Here are the ones that work (without error) and the ones that give me the same error code.

OS-0-128_v2.3.0_1024x10_20220419_153621-000.pcap

Filename	working or not
OS-0-128_v2.3.0_1024x10_20220419_153621-000.pcap	not working
OS-0-128_v2.3.0_1024x10_20220419_155133-000.pcap	working
OS-0-128_v3.0.1_1024x10_20230216_172749-000.pcap	not working
OS-0-32-U0_v2.3.0_1024x10_20220420_112351-000.pcap	working
OS-0-32-U0_v2.3.0_1024x10_20220420_113012-000.pcap	working
OS-0-64-U02_v2.3.0_1024x10_20220420_110119-000.pcap	working
OS-0-64-U02_v2.3.0_1024x10_20220420_110749-000.pcap	working

In the same way I try the second command 'ouster-cli --traceback source OS-0-128_v3.0.1_1024x10_20230216_172749-000.pcap viz`, and it stop with this error:

C:\Users\Administrateur\Documents\ouster>ouster-cli --traceback source OS-0-128_v2.3.0_1024x10_20220419_153621-000.pcap viz
Reading metadata from: C:\Users\Administrateur\Documents\ouster\OS-0-128_v2.3.0_1024x10_20220419_153621.json
GL Renderer: Intel(R) HD Graphics 630
GL Version: 3.3.0 - Build 30.0.101.1339 (GLSL: 3.30 - Build 30.0.101.1339)
Press '?' while viz window is focused to print key bindings
Exception in thread Viz processing:
Traceback (most recent call last):
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\threading.py", line 980, in _bootstrap_inner
    self.run()
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\ouster\sdk\viz.py", line 1150, in _process
    self._scan_viz.scan = next(seekable)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\ouster\sdk\viz.py", line 871, in __next__
    t = next(self._it)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\ouster\client\core.py", line 418, in __iter__
    packet = next(it)
  File "C:\Users\Administrateur\anaconda3\envs\ouster\lib\site-packages\ouster\pcap\pcap.py", line 129, in __iter__
    if not _pcap.next_packet_info(self._handle, packet_info):
RuntimeError: ftell error: errno 22
Done

I verify if all the sample file work with this second command line and the results are same than the previous table.

My own files not working too with the same error.

Do you have an idea how to solve that? At first glance, I notice that the larger files are not working. Is this a coincidence? I've checked if my computer's memory and processors are getting saturated during execution, but that's not the case.
Thank you
Pascal

[twslankard]

Hi Pascal,

Thanks for your patience with this issue. I apologize for delaying response to your inquiry. I'm not able to replicate the issue you're experiencing when running ouster-cli source.

Some file systems (e.g. FAT32) have a file size limitation. Do any of the files you're using exceed the file size limit specified by the filesystem you're using?

In any case, could you also tell me the revision of ouster-sdk you're using?

Best Regards,
Tom

[twslankard]

Hi again Pascal,

I tried this again (now on a Windows machine) and can confirm the issue. I'll see if we can come up with a fix and get back to you with an update as soon as possible.

Best Regards,
Tom

[twslankard]

Hi Pascal,

I've created a fix. If you're building from source, you should be able to apply the following patch to resolve the issue.

We'll include this fix in the next release toward the end of September, (which also includes more bug fixes!)

Best Regards,
Tom

diff --git a/ouster_pcap/src/pcap.cpp b/ouster_pcap/src/pcap.cpp
index fd0d5553..38b8950a 100644
--- a/ouster_pcap/src/pcap.cpp
+++ b/ouster_pcap/src/pcap.cpp
@@ -14,6 +14,9 @@
 
 #if defined _WIN32
 #include <winsock2.h>
+#define FTELL _ftelli64
+#define FSEEK _fseeki64
+#elif defined __EMSCRIPTEN__
 #define FTELL ftell
 #define FSEEK fseek
 #else