You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently forte is creating large intermediate files, which are demanded by nf-core's atomized modules. I've identified a few places where the bam storage can be reduced
STAR_FOR_STARFUSION - we do not need to produce a bam for this process to run STARFUSION and we should probably skip writing one by default. the nf-core version of star/align currently requires a bam for the output. there's an easy fix, simply allow the bam output to be optional
Allow all STAR_* processes to accept multiple pairs of fastqs. That way we don't have to use samtools/merge or cat/fastq to merge read data before or after, and we can even maintain distinct read groups this way. STAR is capable of doing this, but nf-core's star/align was not written with this usage case in mind.
Merge the STAR_FOR_ARRIBA and ARRIBA processes together and don't output the bam at the end of the process. If scratch=true or other cleanup protocol is enabled, the bam will be discarded. Currently nf-core does not have a combined process to run arriba.
I think executing all of these will allow us to eliminate up to 5 copies of unused read data per sample, which is a huge savings in terms of storage and will possibly optimize time/compute as well. It may take some time for nf-core to adopt these changes so best to go ahead with local changes for now.
The text was updated successfully, but these errors were encountered:
Currently forte is creating large intermediate files, which are demanded by nf-core's atomized modules. I've identified a few places where the bam storage can be reduced
I think executing all of these will allow us to eliminate up to 5 copies of unused read data per sample, which is a huge savings in terms of storage and will possibly optimize time/compute as well. It may take some time for nf-core to adopt these changes so best to go ahead with local changes for now.
The text was updated successfully, but these errors were encountered: