hifisr

HiFi-SR is a Python-based pipeline for the detection of plant mitochondrial structural rearrangements based on the mapping of PacBio high-fidelity (HiFi) reads or Circular Consensus Sequencing (ccs) data, to a reference genome (i.e., the hypothetical master cycle DNA).

Deps

The pipeline has been tested in WSL2 distribution Ubuntu-20.04. It shall work in other Linux operating system, such as CentOS.

• Download the hifisr repository

git clone https://github.com/zouyinstein/hifisr

• Create and activate a conda environment.

conda create -n hifisr python=3.9
conda activate hifisr

• Use Anaconda3 to install required packages.

conda install pigz -c conda-forge
conda install samtools bamtools blast seqkit parafly -c bioconda
# create a soft link to ensure samtools can work
ln -sf ${HOME}/.conda/envs/hifisr/lib/libcrypto.so.1.1 ${HOME}/.conda/envs/hifisr/lib/libcrypto.so.1.0.0  

• Install bcftools demo
• Install minimap2

cd hifisr/deps
curl -L https://github.com/lh3/minimap2/releases/download/v2.24/minimap2-2.24_x64-linux.tar.bz2 | tar -jxvf -
./minimap2-2.24_x64-linux/minimap2

• Install Filtlong

cd hifisr/deps
git clone https://github.com/rrwick/Filtlong.git
cd Filtlong
make -j

export PATH="$PWD/deps/Filtlong/bin:$PATH"

• Install MECAT2

cd hifisr/deps
git clone https://github.com/xiaochuanle/MECAT2.git
cd MECAT2
make -j

export PATH="$PWD/deps/MECAT2/Linux-amd64/bin:$PATH"

• Install metaFlye

cd hifisr/deps
git clone https://github.com/fenderglass/Flye
cd Flye
python setup.py install

• Install required Python packages

pip install biopython pandas openpyxl

Test

# Make sure the HiFi-SR repository has been downloaded (git clone https://github.com/zouyinstein/hifisr).
# Make sure the hifisr environment has been activated (conda activate hifisr).
# Make sure the dependent third-party softwares/packages has been installed.
# Change working directory to hifisr
cd hifisr
# Add minimap2, filtlong, mecat.pl executables to the system PATH
export PATH="$PWD/deps/minimap2-2.24_x64-linux":$PATH
export PATH="$PWD/deps/Filtlong/bin:$PATH"
export PATH="$PWD/deps/MECAT2/Linux-amd64/bin:$PATH"

# Check and unzip the reference sequences
cd references
ls Col_mito.fa Col_plastid.fa Col_ref.fa.gz
pigz -d -p 8 Col_ref.fa.gz

# Check and unzip the input HiFi reads
cd ../data
pigz -d -p 8 Col.fastq.gz

# Change working directory to test and prepare the file input_files.txt
cd ../test
touch input_files.txt
# Contents of input_files.txt are tab-delimilated columns of sample name, input reads, total genome reference, mt genome reference, and pt  genome reference. The information of multiple samples can be added in different lines.

# Prepare the starting files and directories and a job script work.sh: 8 threads for each sample; analyze 1 sample in parallel
python ../scripts/start_project.py input_files.txt 8 1
# Run the job script will start the HiFi-SR pipeline
nohup bash work.sh &

Description of results

demo

Example 1

Analyze of an example wild-type Arabidopsis thaliana dataset Col-CEN (ERR6210723, 14.6 Gb, Naish et al., 2021, Science):

cd hifisr/pre
mkdir CEN  # CEN is the sample name
cd CEN
pigz -d -k -p 16 ERR6210723.fastq.gz
ln -sf ERR6210723.fastq CEN.fastq
# manually change the FASTA headers of mitochondrial and plastid genome refercences into mito and plastid for easily manipulation
cat Athaliana_447_TAIR10.id_Chr{1,2,3,4,5}.fa refs_cp28673mod.id_mito.fas refs_cp28673mod.id_plastid.fas > CEN_ref.fa
ln -sf refs_cp28673mod.id_mito.fas CEN_mito.fa
ln -sf refs_cp28673mod.id_plastid.fas CEN_plastid.fa
# check conda activate hifisr
# check export PATH=/path/to/hifisr/deps/minimap2-2.24_x64-linux:$PATH
python ../../hifisr.py -s CEN -t 10 -i fastq single > $(date +%s).log 2> $(date +%s).err &

Example 2

Analyze of an example wild-type Arabidopsis thaliana dataset Col-XJTU (CRR302668, 22.9 Gb, Wang et al., 2021, GPB):

cd hifisr/pre
mkdir XJTU  # XJTU is the sample name
cd XJTU
pigz -d -k -p 16 CRR302668.fastq.gz
ln -sf CRR302668.fastq XJTU.fastq
# manually change the FASTA headers of mitochondrial and plastid genome refercences into mito and plastid for easily manipulation
cat Athaliana_447_TAIR10.id_Chr{1,2,3,4,5}.fa refs_cp28673mod.id_mito.fas refs_cp28673mod.id_plastid.fas > XJTU_ref.fa
ln -sf refs_cp28673mod.id_mito.fas XJTU_mito.fa
ln -sf refs_cp28673mod.id_plastid.fas XJTU_plastid.fa
# check conda activate hifisr
# check export PATH=/path/to/hifisr/deps/minimap2-2.24_x64-linux:$PATH
python ../../hifisr.py -s XJTU -t 10 -i fastq single > $(date +%s).log 2> $(date +%s).err &

Merge reports of multiple samples

Merge reports of Col-CEN and Col-XJTU:

cd hifisr/pre
echo CEN >> merge_1.txt
echo XJTU >> merge_1.txt
# check conda activate hifisr
# check export PATH=/path/to/hifisr/deps/minimap2-2.24_x64-linux:$PATH
python ../hifisr.py -m merge_1 merge > $(date +%s).log 2> $(date +%s).err &