Performance drops dramatically when flush dirty data to core device

[mslovy]

Question

I find that, when trigger dirty data flushing, the performance of the cas device drops dramatically. I think the each cache line itself has fine-grain lock to manage rwlock for io request, when I flush very small parts of dirty cache lines, it will not be affected other cache-lines read/write ops? Furthermore, flush may need to block write requests, but there is no need to block read requests, right?

Motivation

In our use-case, the cas device would be very busy so that we need to set activity-threshold to 0, in order to trigger the flushing operations. Otherwise, cache device would not be flush any dirty data until the cache fill up with dirty data.

the configuration looks like this:
casadm -X -n cleaning-alru -t 0 -b 128 -s 120 -w 1 -i 1

and when flushing the IOPS on client drops to approximately 5000 from previous 20000. If I flush more data once, like:
casadm -X -n cleaning-alru -t 0 -b 128 -s 1000 -w 1 -i 1

the performance drops lowest to 500, and damping from 500 ~ 5000.

So I would like to know how to minimize the performance impact in this use-case. Any also if there is possible to optimize the lock granularity in codes, can anyone show the critical code blocks related to this?