Writing binary data to flash from IMU device - buffering issue causing acquisition loss

[rjsdotorg]

The buffer for a file opened with default buffering size appears to be 4096 (Micropython on a Pico W clocked at 250mHz, and an MPU6050 over I2C).
I do open(NAME, 'wb', buffering=n) a file.
Every 4096 bytes that are acquired and written, 10 bytes acquired viareadfrom_mem()at a time and written via fh.write(_bytes), causes a gap in the data, as apparently(?) the I2C gets locked/blocked during the block write. I'm testing using flash storage as SD writes are even slower.
With default buffering each readfrom_mem / fh.write call take ~100us leaving 900us idle at 1ksmp/s, but when the write actually takes place it takes a looong time leading to big gaps.
If I set buffering=0, or -1 or 4000000, it is the same behavior.
If I do fh.flush() at all, after any n bytes collected less than 4096 the average sample rate drops to 29.25 smp/s, implying ~34ms time spent.

So, what is the best strategy here? The goal is to readfrom_mem() over I2C and store the data to flash or SD card, potentially megabytes, so I can't just allocate a huge bytearray,
I read that threading might help but have not tested.
I also read that both cores are blocked during write() so core1 is no help.
Is there a DMA option?
No flush() of buffering=n combination I've tested works.

[peterhinch]

The problem here is that I2C transfers and file writes are both blocking. You haven't told us what MPU6050 driver you're using. If it's this one it is possible to read IMU values in a hard interrupt service routine (ISR). If it's your own driver, you might take a look at the code for an example of a non-allocating I2C read.

The interrupt would be triggered by a timer. The ISR could copy the data to a buffer. I would suggest using a two phase buffer. While one buffer is being written to file, the other is being filled by the ISR. With careful design it should be possible to avoid data loss.

[rjsdotorg]

Thank you Peter,
I do use code similar to imu.py, but readfrom_mem() directly in a loop rather than classes and methods. I'll paste the short code, shortly; it was essentially culled from Tim Hanewich's module.
MPU-6000/MPU-6050 Product Specification lists data rate as 1kHz so that's what I was hoping for.
I do have INT wired to GPIO5 and tested an interrupt version, mainly for sample rate stability.
What I fear is that the limiting factor is the I2C-only 6050 - at 400kHz I don't think I can hope to do this.

BTW you have return bytes_toint(self.buf2[0], self.buf2[1]) / 340 + 35 # I think on line 184 of imu.py
MPU-6000 and MPU-6050 Register Map and Descriptions Revision 4.2 shows:

Temperature in degrees C = (TEMP_OUT Register Value as a signed quantity)/340 + 36.53

[rjsdotorg]

I have seen modules which state "I2C / SPI" like this which is confusing since the docs state SPI Timing Characterization (MPU-6000 only) in a number of places.

[rjsdotorg]

For example (or amusement) AQ code:

import io
from os import listdir, chdir
import sys
from time import sleep, ticks_us, ticks_diff

from machine import freq, I2C, Pin, ADC
import micropython
micropython.alloc_emergency_exception_buf(100)
import MPU6050
import uos

AQ_US = const(10 * 1000000) # microseconds
FILE_NAME = f'mpu_data'
USE_IRQ = 0
def data_ready_callback(pin):
    # tested, not used
    global interrupt_flag
    interrupt_flag = 1

led.on()
freq(250000000)

# Set up the I2C interface
i2c = I2C(1, sda=Pin(14), scl=Pin(15))
led.off()

# Set up the MPU6050 class (Tim Hanewich)
mpu = MPU6050.MPU6050(i2c)
# shorter version of  dot notation
readfrom_mem = mpu.i2c.readfrom_mem
# wake up the MPU6050 from sleep
mpu.wake()
# use high ranges
mpu.i2c.writeto_mem(0x68, 0x1B, bytes([0x18]))
mpu.i2c.writeto_mem(0x68, 0x1C, bytes([0x18]))
a_rng = mpu.read_accel_range()
print(f'range:{a_rng}')

dir_list = listdir()

# Create a file and write the sensor readings to it
fh = open(f'{FILE_NAME}_{len(dir_list)}_{VSYS_V}.bin',
          'wb',
          buffering=0)
print(f'fh {fh}')

samp_num = 0
if USE_IRQ:
    # not using irq
    interrupt_flag = 0
    data_buf = bytearray(5000)
    pin = Pin(5, Pin.IN, Pin.PULL_UP)
    pin.irq(trigger=Pin.IRQ_FALLING, handler=data_ready_callback)

led.on()
# enable the MPU6050 INT pin output
mpu.i2c.writeto_mem(0x68, 0x38, bytes([0x1]))
t0 = bef = ticks_us()
d = bytearray(256) # preallocate
# continuous
while True:
    # slow the AQ to ~1000 sps
    while ticks_diff(ticks_us(), bef)<900:
        pass
    bef = ticks_us()
    d = readfrom_mem(0x68, 0x3B, 14)
    # accels, temp, and gyro z
    fh.write(d[0:8] + d[12:14])
    samp_num += 1
    if USE_IRQ and interrupt_flag is 1:
        interrupt_flag=0
        # read 5 bytes
        data_buf[buf_ptr:buf_ptr + 10] = readfrom_mem(0x68, 0x3B, 10)
        samp_num += 1
        buf_ptr += 10
        if buf_ptr>4096:
            # write accel bytes
            fh.write(data_buf[0:buf_ptr])
            buf_ptr = 0

    if ticks_diff(ticks_us(), t0) > AQ_US:
        break
et = ticks_diff(ticks_us(), t0) / 1000000 - .0003 # seconds from us, and 300us lag
fh.flush()
fh.close()
sps = samp_num / et

led.off()
mpu.sleep()

[peterhinch]

This is not ideal. Firstly for fast response you need hard=True in

pin.irq(trigger=Pin.IRQ_FALLING, handler=data_ready_callback)

Worse, the benefit of an ISR is lost because the code polls the interrupt_flag. The benefit of an interrupt is to enable concurrency of device reading and file writing. This is lost in the above code. The ISR needs to buffer samples, not just set a flag.

[rjsdotorg]

Thank you,
True of course and not good example code, but it is the write to file that seems the bottleneck - if it blocks I2C for more than a millisecond data will be missed.
I tried putting readfrom_mem and write() in the isr, but the write is still buffered and the code misses samples due to the extended blocking of I2C for 4096 byte write.
If I just write to a 60kB bytearray in RAM (with hard=False):

def data_ready_callback(pin):
    global samp_num, byte_str
    byte_str[samp_num * 10: samp_num * 10 + 10] = readfrom_mem(0x68, 0x3B, 10))
samp_num += 1

it runs fine for those 6 seconds, no data loss, and I write the 60kB string to flash in one go. Some jitter I'm sure "With soft IRQ the latency is ~35µs, with a range of 30 to 44 µs." but no drops.
If I set hard=True then I get

MemoryError: memory allocation failed, heap is locked

at the 'readfrom_mem()' assignment. Using heap_unlock() and heap_lock() around it throws even more memory errors.
'readfrom_mem_into()' has the same locked heap error.
I tried putting the 'readfrom_mem_into()' in a function called by just a schedule(read_data()) in the callback, but same error.

def read_data():
    global samp_num, byte_str
    byte_str[samp_num * 10: samp_num * 10 + 10] = readfrom_mem(0x68, 0x3B, 10))
    samp_num += 1
def data_ready_callback(pin):
   micropython.schedule(read_data())

I think fundamentally I do not see the way to design this on the Pico, with the goal being megabytes of 1kHz data written to SD from the I2C device.
There appears to be a C I2C>DMA path https://github.com/fivdi/pico-i2c-dma but no uPython that I see.

[peterhinch]

I believe what you are trying to do is feasible, but you need to read up on memory allocation. The method readfrom_mem allocates, but readfrom_mem_into does not. As I suggested above, you need a hard ISR with a double buffer. The buf arg would be a segment of one of the buffers, defined with a memoryview. When one phase of the buffer is full, the ISR would be pointed at the other buffer and the first one written out.

These are relatively advanced techniques so be prepared to do some reading, e.g. this official doc and this one.

[rjsdotorg]

I believe what you are trying to do is feasible, but you need to read up on memory allocation. The method readfrom_mem allocates, but readfrom_mem_into does not. As I suggested above, you need a hard ISR with a double buffer. The buf arg would be a segment of one of the buffers, defined with a memoryview. When one phase of the buffer is full, the ISR would be pointed at the other buffer and the first one written out.

Thanks, I'm aware of the allocation restriction, but had not tried using memoryview for the purpose of pointing to a segment. The design should be interesting.
I learned that replacing readfrom_mem with readfrom_mem_into(0x68, 0x3B, byte_str[samp_num * 10: samp_num * 10 + 10]) still throws "MemoryError: memory allocation failed, heap is locked".
Most people are using IMUs at 25 or 200Hz in RVC mode which does not present much timing challenge. And next week I'll be ordering a newer ST IMU that does 6kHz with 9 axes.

[rjsdotorg]

Incidentally, hard=True is not really needed to reduce jitter here, since the IMU samples at a fixed 1024Hz so the same data is available at 0x3B anytime within the ~1ms window.
I also have not tried to take advantage of the MPU6050 on-chip 1024 byte FIFO, which would reduce the timing constraints by allowing the system processor to read the sensor data in bursts any time within a one second window, so long as it keeps up on average.

[peterhinch]

With hard=False latency can exceed 1ms.

[peterhinch]

byte_str[samp_num * 10: samp_num * 10 + 10]

This allocates because byte_str is a bytearray instead of a memoryview. Consider this code:

a = bytearray(20)
b = a[5:10]
b[0] = 55
print(b)
print(a)

If you run this you'll see that b is the result of an allocation: b is a new array. A memoryview prevents this effect.

[rjsdotorg]

and some code to read the bytestring file and convert to values and plot
I use Thonny to copy the data file to a PC.

import struct
import numpy
import matplotlib.pyplot as plt

acc_scale = 2048. # 16384.0 is for 0, use 2048 for 3
gyr_scale = 16.4

f_path = 'mpu_data.bin'
raw_values = 0

fh = open(f_path, 'rb')
bin_str = fh.read()
fh.close()

data = numpy.empty((int(len(bin_str) / 10), 5))
with open(f_path, 'rb') as fh:
    samp = 0
    while byte_str := fh.read(10):
        ax, ay, az, tmp, gz = struct.unpack('>5h', byte_str)
        if not raw_values:

            ax = (ax / acc_scale - .0366) / 1.000
            ay = (ay / acc_scale + .0700) / 0.941
            az = (az / acc_scale + .0500) / 1.03
            # save temp for gyro compensation
            tmp = (tmp  / 340.0) + 36.53 # C
            #tmp = ((tmp  / 340.0) + 36.53) * (9/5) + 32 # F
            #gx = (gx / gyr_scale + 0.970) / 1 + (tmp - 30) * .03 + 0.9
            #gy = (gy / gyr_scale + 1.790) / 1 + (tmp - 30) * .03 + 0.9
            # save gz for rotation rate
            gz = (gz / gyr_scale + 1.840) / 0.993 + .07 - (tmp * .11) + 2.706
        # accel, temp, rpm
        data[samp] = ax, ay, az, tmp, gz / 6.
        samp += 1
    
# plot
fig, ax1 = plt.subplots()
ax2 = ax1.twinx()
x = numpy.arange(0, len(data)) *10# / 998.
ax1.plot(x, data[:, 0], label='x', color='lime')
ax1.plot(x, data[:, 1], label='y', color='green')
ax1.plot(x, data[:, 2], label='z', color='cyan')
ax2.plot(x, data[:, 3], label='C', color='k')
ax2.plot(x, data[:, 4], label='RPM', color='magenta')
#ax2.plot(x, data[:, 5], label='gy', color='red')
#ax2.plot(x, data[:, 6], label='gz', color='pink')
ax1.set_xlabel('seconds')
ax1.set_ylabel('g')
ax2.set_ylabel('RPM, temp C')
ax1.legend(loc='upper left')
ax2.legend(loc='upper right')
plt.title([round(a, 4) for a in aves.tolist()])
#ax1.set_ylim((-.1, .2))
#ax2.set_ylim((33, 34))
plt.get_current_fig_manager().window.state('zoomed')
plt.show()

``

[jonnor]

If your usecase is not latency sensitive - which it does not seem to be since you are writing to FLASH - then use the FIFO on the IMU to do data transfers in more efficient blocks of samples. It will greatly reduce the risks of missing samples due to allocations/GC/IO/etc. Ref https://github.com/orgs/micropython/discussions/15512

[rjsdotorg]

If your usecase is not latency sensitive - which it does not seem to be since you are writing to FLASH - then use the FIFO on the IMU to do data transfers in more efficient blocks of samples. It will greatly reduce the risks of missing samples due to allocations/GC/IO/etc. Ref https://github.com/orgs/micropython/discussions/15512

Correct, analysis would be post post-capture.
I saw the thread, and although the MPU5060 is not listed, it is similar in that it has a 1024 FIFO one enables at 0x6A and 0x23 etc (as well described in SO and this and this).
It seems that the design should be:
read FIFO (using a memoryview) until RAM bytearray gets >4096 bytes worth
write the bytestring to SD or flash and hope it takes less than a second

[jonnor]

Yep, something like that should work quite well! If writing one block to flash takes more than 1 second then sensor readout is not the major obstacle! Of course, you can also add some in-memory buffers, if there are occasional hiccups. I like the collections.deque module for this.