CITE-seq R1 length issue

[Rimanpreet1]

Hi,
I am trying to analyse raw 10X genomics sequence data on it, using the following code on HPC cluster but it is keep giving me an error and it seems it stops
-R1 SC98_R1_001.fastq.gz
-R2 SC98R2_001.fastq.gz
-t tags.csv
-cbf 1
-cbl 16
-umif 17
-umil 26
-cells 6000

error "[WARNING] Read1 length is 150bp but you are using 26bp for Cell and UMI barcodes combined.
This might lead to wrong cell attribution and skewed umi counts."
could you please help me with this

[Hoohm]

Hello @Rimanpreet1
this is just a warning, you can ignore it.
You say it stops, what's the error you get?

[Rimanpreet1]

Hi Patrick, Thanks here I attached the screenshots of an errors and it seems it stucks ***@***.******@***.******@***.*** From: Patrick Roelli ***@***.***> Sent: Wednesday, August 28, 2024 3:24 PM To: Hoohm/CITE-seq-Count ***@***.***> Cc: Kaur, Rimanpreet ***@***.***>; Mention ***@***.***> Subject: Re: [Hoohm/CITE-seq-Count] CITE-seq R1 length issue (Issue #194) CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Hello @Rimanpreet1<https://github.com/Rimanpreet1> this is just a warning, you can ignore it. You say it stops, what's the error you get? - Reply to this email directly, view it on GitHub<#194 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BCBATKPRC4KW3KFTN6KF5W3ZTYPWHAVCNFSM6AAAAABNISSIF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJWGA4TEMJSGQ>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by e-mail and destroy all copies of the original.

[Rimanpreet1]

Hi Patrick, It is stuck and is not running, could you please help me? I attached the screenshot for your reference ***@***.***thanks, Rimanpreet From: Patrick Roelli ***@***.***> Sent: 28 August 2024 03:24 PM To: Hoohm/CITE-seq-Count ***@***.***> Cc: Kaur, Rimanpreet ***@***.***>; Mention ***@***.***> Subject: Re: [Hoohm/CITE-seq-Count] CITE-seq R1 length issue (Issue #194) CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Hello @Rimanpreet1<https://github.com/Rimanpreet1> this is just a warning, you can ignore it. You say it stops, what's the error you get? - Reply to this email directly, view it on GitHub<#194 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BCBATKPRC4KW3KFTN6KF5W3ZTYPWHAVCNFSM6AAAAABNISSIF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJWGA4TEMJSGQ>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by e-mail and destroy all copies of the original.

[Hoohm]

I think screenshots don't follow if you answer by email. Could you post them directly in the issue or copy paste the text?

[Rimanpreet1]

Hi Patrick, Here I attached the errors in text file, my job on cite-seq-count was stuck and killed eventually. Thanks, Rimanpreet From: Patrick Roelli ***@***.***> Sent: Thursday, August 29, 2024 5:41 AM To: Hoohm/CITE-seq-Count ***@***.***> Cc: Kaur, Rimanpreet ***@***.***>; Mention ***@***.***> Subject: Re: [Hoohm/CITE-seq-Count] CITE-seq R1 length issue (Issue #194) CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. I think screenshots don't follow if you answer by email. Could you post them directly in the issue or copy paste the text? - Reply to this email directly, view it on GitHub<#194 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BCBATKO33XM6EX4KULIMEQTZT3UDXAVCNFSM6AAAAABNISSIF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJXGE3TENRRGY>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by e-mail and destroy all copies of the original. Exception in thread Thread-3: Traceback (most recent call last): File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/threading.py", line 980, in _bootstrap_inner self.run() File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/threading.py", line 917, in run self._target(*self._args, **self._kwargs) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/multiprocess/pool.py", line 576, in _handle_results task = get() File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/multiprocess/connection.py", line 254, in recv return _ForkingPickler.loads(buf.getbuffer()) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 303, in loads return load(file, ignore, **kwds) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 289, in load return Unpickler(file, ignore=ignore, **kwds).load() File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 444, in load obj = StockUnpickler.load(self) MemoryError Processed 1,000,000 reads in 12.02 seconds. Total reads: 11,000,000 in child 22222 Processed 1,000,000 reads in 12.08 seconds. Total reads: 5,000,000 in child 22223 Processed 1,000,000 reads in 12.34 seconds. Total reads: 6,000,000 in child 22197 Processed 1,000,000 reads in 11.75 seconds. Total reads: 2,000,000 in child 22198 Processed 1,000,000 reads in 12.45 seconds. Total reads: 12,000,000 in child 22222 Processed 1,000,000 reads in 11.48 seconds. Total reads: 7,000,000 in child 22197 Processed 1,000,000 reads in 12.91 seconds. Total reads: 6,000,000 in child 22223 Processed 1,000,000 reads in 11.57 seconds. Total reads: 3,000,000 in child 22198 Processed 1,000,000 reads in 12.75 seconds. Total reads: 13,000,000 in child 22222 Processed 1,000,000 reads in 11.55 seconds. Total reads: 8,000,000 in child 22197 Processed 1,000,000 reads in 11.68 seconds. Total reads: 7,000,000 in child 22223 Processed 1,000,000 reads in 12.07 seconds. Total reads: 4,000,000 in child 22198 Processed 1,000,000 reads in 11.45 seconds. Total reads: 14,000,000 in child 22222 Processed 1,000,000 reads in 11.75 seconds. Total reads: 8,000,000 in child 22223 Processed 1,000,000 reads in 13.11 seconds. Total reads: 9,000,000 in child 22197 Processed 1,000,000 reads in 11.89 seconds. Total reads: 5,000,000 in child 22198 Mapping done for process 22222. Processed 14,532,619 reads Processed 1,000,000 reads in 12.84 seconds. Total reads: 9,000,000 in child 22223 Processed 1,000,000 reads in 13.21 seconds. Total reads: 10,000,000 in child 22197 Processed 1,000,000 reads in 12.14 seconds. Total reads: 6,000,000 in child 22198 Processed 1,000,000 reads in 11.55 seconds. Total reads: 10,000,000 in child 22223 Processed 1,000,000 reads in 11.99 seconds. Total reads: 11,000,000 in child 22197 Processed 1,000,000 reads in 12.39 seconds. Total reads: 7,000,000 in child 22198 Processed 1,000,000 reads in 11.57 seconds. Total reads: 11,000,000 in child 22223 Processed 1,000,000 reads in 12.57 seconds. Total reads: 12,000,000 in child 22197 Processed 1,000,000 reads in 11.95 seconds. Total reads: 8,000,000 in child 22198 Processed 1,000,000 reads in 12.42 seconds. Total reads: 12,000,000 in child 22223 Processed 1,000,000 reads in 12.99 seconds. Total reads: 13,000,000 in child 22197 Processed 1,000,000 reads in 11.79 seconds. Total reads: 9,000,000 in child 22198 Processed 1,000,000 reads in 12.88 seconds. Total reads: 13,000,000 in child 22223 Processed 1,000,000 reads in 11.54 seconds. Total reads: 14,000,000 in child 22197 Processed 1,000,000 reads in 13.18 seconds. Total reads: 10,000,000 in child 22198 Mapping done for process 22197. Processed 14,532,619 reads Processed 1,000,000 reads in 11.29 seconds. Total reads: 14,000,000 in child 22223 Processed 1,000,000 reads in 12.34 seconds. Total reads: 11,000,000 in child 22198 Mapping done for process 22223. Processed 14,532,619 reads Processed 1,000,000 reads in 12.03 seconds. Total reads: 12,000,000 in child 22198 Processed 1,000,000 reads in 11.69 seconds. Total reads: 13,000,000 in child 22198 Processed 1,000,000 reads in 1.0 hour, 9.0 minutes, 1.956 seconds. Total reads: 1,000,000 in child 22200 Processed 1,000,000 reads in 13.05 seconds. Total reads: 14,000,000 in child 22198 Mapping done for process 22198. Processed 14,532,619 reads Processed 1,000,000 reads in 11.88 seconds. Total reads: 2,000,000 in child 22200 Processed 1,000,000 reads in 12.1 seconds. Total reads: 3,000,000 in child 22200 Processed 1,000,000 reads in 11.53 seconds. Total reads: 4,000,000 in child 22200 Processed 1,000,000 reads in 10.96 seconds. Total reads: 5,000,000 in child 22200 next error Process ForkPoolWorker-24: Traceback (most recent call last): File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/multiprocess/pool.py", line 131, in worker put((job, i, result)) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/multiprocess/queues.py", line 374, in put obj = _ForkingPickler.dumps(obj) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/multiprocess/reduction.py", line 54, in dumps cls(buf, protocol, *args, **kwds).dump(obj) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 420, in dump StockPickler.dump(self, obj) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 487, in dump self.save(obj) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 414, in save StockPickler.save(self, obj, save_persistent_id) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 886, in save_tuple save(element) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 414, in save StockPickler.save(self, obj, save_persistent_id) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 886, in save_tuple save(element) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 414, in save StockPickler.save(self, obj, save_persistent_id) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 886, in save_tuple save(element) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 414, in save StockPickler.save(self, obj, save_persistent_id) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 1217, in save_module_dict StockPickler.save_dict(pickler, obj) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 971, in save_dict self._batch_setitems(obj.items()) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 997, in _batch_setitems save(v) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 414, in save StockPickler.save(self, obj, save_persistent_id) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 603, in save self.save_reduce(obj=obj, *rv) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 713, in save_reduce self._batch_setitems(dictitems) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 1002, in _batch_setitems save(v) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 414, in save StockPickler.save(self, obj, save_persistent_id) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 603, in save self.save_reduce(obj=obj, *rv) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 692, in save_reduce save(args) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 414, in save StockPickler.save(self, obj, save_persistent_id) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 886, in save_tuple save(element) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 414, in save StockPickler.save(self, obj, save_persistent_id) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 1217, in save_module_dict StockPickler.save_dict(pickler, obj) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 971, in save_dict self._batch_setitems(obj.items()) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 996, in _batch_setitems save(k) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/dill/_dill.py", line 414, in save StockPickler.save(self, obj, save_persistent_id) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 806, in save_bytes self.memoize(obj) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/pickle.py", line 511, in memoize self.memo[id(obj)] = idx, obj MemoryError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/multiprocess/process.py", line 315, in _bootstrap self.run() File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/multiprocess/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/gpfs/software/Anaconda/envs/cite-seq-count/lib/python3.9/site-packages/multiprocess/pool.py", line 134, in worker util.debug("Possible encoding error while sending result: %s" % ( MemoryError Processed 1,000,000 reads in 12.55 seconds. Total reads: 6,000,000 in child 22200 Processed 1,000,000 reads in 12.37 seconds. Total reads: 7,000,000 in child 22200 Processed 1,000,000 reads in 11.81 seconds. Total reads: 8,000,000 in child 22200 Processed 1,000,000 reads in 12.88 seconds. Total reads: 9,000,000 in child 22200 Processed 1,000,000 reads in 12.85 seconds. Total reads: 10,000,000 in child 22200 Processed 1,000,000 reads in 13.44 seconds. Total reads: 11,000,000 in child 22200 Processed 1,000,000 reads in 14.02 seconds. Total reads: 12,000,000 in child 22200 Processed 1,000,000 reads in 15.07 seconds. Total reads: 13,000,000 in child 22200 Processed 1,000,000 reads in 14.5 seconds. Total reads: 14,000,000 in child 22200 Mapping done for process 22200. Processed 14,532,625 reads Killed

[Hoohm]

It might be a memory limitation issue. Can you show me the command you used to run it?

[Rimanpreet1]

Hi Patrick, I am running it on HPC -cluster and how I can manage the memory limitation issue, could you please help me with this, I also attached the script. Rimanpreet From: Patrick Roelli ***@***.***> Sent: Thursday, September 5, 2024 2:30 AM To: Hoohm/CITE-seq-Count ***@***.***> Cc: Kaur, Rimanpreet ***@***.***>; Mention ***@***.***> Subject: Re: [Hoohm/CITE-seq-Count] CITE-seq R1 length issue (Issue #194) CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. It might be a memory limitation issue. Can you show me the command you used to run it? - Reply to this email directly, view it on GitHub<#194 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BCBATKPJWEJOP4W7TKWKCH3ZU727DAVCNFSM6AAAAABNISSIF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZQG4YDKNJWGI>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by e-mail and destroy all copies of the original.

…

-R1 SC98_R1_001.fastq.gz -R2 SC98R2_001.fastq.gz -t tags.csv -cbf 1 -cbl 16 -umif 17 -umil 26 -cells 6000 CITE-seq-Count -R1 /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/SC98_1S_CKDL240025311-1A_22FVTVLT4_S1_L007_R1_001.fastq.gz -R2 /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/SC98_1S_CKDL240025311-1A_22FVTVLT4_S1_L007_R2_001.fastq.gz -t /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/tags.csv -cbf 1 -cbl 16 -umif 17 -umil 26 -cells 6000

[Hoohm]

How many reads are there?

The main way you can change the amount of memory used is by reducing the number of threads. But you might be reaching the maximum memory per thread.

I'm not really sure.

Could you try to test out a beta branch? It's completely rewritten in polars and is using the lazy API, it might solve your memory issue.

beta branch

[Rimanpreet1]

Hi Patrick, I tried with specific threads an on a node with 368 GB memory but it again got killed ***@***.*** sh]$ CITE-seq-Count -R1 /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/SC98_1S_CKDL240025311-1A_22FVTVLT4_S1_L007_R1_001.fastq.gz -R2 /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/SC98_1S_CKDL240025311-1A_22FVTVLT4_S1_L007_R2_001.fastq.gz -t /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/cseqcount_ab_list.csv -cbf 1 -cbl 16 -umif 17 -umil 26 -n_barcodes 47 --bc_collapsing_dist 0 -T 50 Writing mapping input Killed Could you please help me, Rimanpreet From: Patrick Roelli ***@***.***> Sent: Wednesday, September 11, 2024 3:21 PM To: Hoohm/CITE-seq-Count ***@***.***> Cc: Kaur, Rimanpreet ***@***.***>; Mention ***@***.***> Subject: Re: [Hoohm/CITE-seq-Count] CITE-seq R1 length issue (Issue #194) CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. How many reads are there? The main way you can change the amount of memory used is by reducing the number of threads. But you might be reaching the maximum memory per thread. I'm not really sure. Could you try to test out a beta branch? It's completely rewritten in polars and is using the lazy API, it might solve your memory issue. beta branch<https://github.com/Hoohm/CITE-seq-Count/tree/feature/cells_argument> - Reply to this email directly, view it on GitHub<#194 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BCBATKKYUCEJ7NRGS63PEGTZWCJY7AVCNFSM6AAAAABNISSIF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBUGUZDKMJXHE>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by e-mail and destroy all copies of the original.

[Rimanpreet1]

Hi, I tried to run on beta, it killed without giving any error, (cite-seq-count-beta) ***@***.*** sh]$ CITE-seq-Count -R1 /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/SC98_1S_CKDL240025311-1A_22FVTVLT4_S1_L007_R1_001.fastq.gz -R2 /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/SC98_1S_CKDL240025311-1A_22FVTVLT4_S1_L007_R2_001.fastq.gz -t /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/cseqcount_ab_list.csv -cbf 1 -cbl 16 -umif 17 -umil 26 -n_barcodes 47 --bc_collapsing_dist 0 -T 50 Writing mapping input Killed From: Kaur, Rimanpreet Sent: Wednesday, September 18, 2024 4:26 PM To: Hoohm/CITE-seq-Count ***@***.***>; Hoohm/CITE-seq-Count ***@***.***> Cc: Mention ***@***.***> Subject: RE: [Hoohm/CITE-seq-Count] CITE-seq R1 length issue (Issue #194) Hi Patrick, I tried with specific threads an on a node with 368 GB memory but it again got killed ***@***.*** sh]$ CITE-seq-Count -R1 /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/SC98_1S_CKDL240025311-1A_22FVTVLT4_S1_L007_R1_001.fastq.gz -R2 /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/SC98_1S_CKDL240025311-1A_22FVTVLT4_S1_L007_R2_001.fastq.gz -t /gpfs/scratch/rikaur/cancer_raw/seqcount/sh/cseqcount_ab_list.csv -cbf 1 -cbl 16 -umif 17 -umil 26 -n_barcodes 47 --bc_collapsing_dist 0 -T 50 Writing mapping input Killed Could you please help me, Rimanpreet From: Patrick Roelli ***@***.******@***.***>> Sent: Wednesday, September 11, 2024 3:21 PM To: Hoohm/CITE-seq-Count ***@***.******@***.***>> Cc: Kaur, Rimanpreet ***@***.******@***.***>>; Mention ***@***.******@***.***>> Subject: Re: [Hoohm/CITE-seq-Count] CITE-seq R1 length issue (Issue #194) CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. How many reads are there? The main way you can change the amount of memory used is by reducing the number of threads. But you might be reaching the maximum memory per thread. I'm not really sure. Could you try to test out a beta branch? It's completely rewritten in polars and is using the lazy API, it might solve your memory issue. beta branch<https://github.com/Hoohm/CITE-seq-Count/tree/feature/cells_argument> - Reply to this email directly, view it on GitHub<#194 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BCBATKKYUCEJ7NRGS63PEGTZWCJY7AVCNFSM6AAAAABNISSIF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBUGUZDKMJXHE>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by e-mail and destroy all copies of the original.

[Hoohm]

Something is weird.

Can you try to run only 1M reads?
Maybe it's a temp path issue, CSC needs to write data to disk, maybe it doesn't have access on the cluster?