Getting short reads.
Getting assembly.
command: bwa index /kb/module/work/tmp/flye.contigs.fa
[bwa_index] Pack FASTA... 0.03 sec
[bwa_index] Construct BWT for the packed sequence...
[bwa_index] 1.20 seconds elapse.
[bwa_index] Update BWT... 0.03 sec
[bwa_index] Pack forward-only FASTA... 0.02 sec
[bwa_index] Construct SA from BWT and Occ... 0.37 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index /kb/module/work/tmp/flye.contigs.fa
[main] Real time: 1.648 sec; CPU: 1.645 sec
command: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs.fa /kb/module/work/tmp/a3b6bc8f-059a-4060-ad5f-4dde8b9fa4a7.fwd.fastq > /kb/module/work/tmp/alignments1_debfe222-d36c-4c68-88e2-247996292665.sam
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 67562 sequences (10000059 bp)...
[M::process] read 67574 sequences (10000021 bp)...
[M::mem_process_seqs] Processed 67562 reads in 25.882 CPU sec, 25.797 real sec
[M::process] read 67568 sequences (10000299 bp)...
[M::mem_process_seqs] Processed 67574 reads in 26.468 CPU sec, 26.313 real sec
[M::process] read 67658 sequences (10000028 bp)...
[M::mem_process_seqs] Processed 67568 reads in 26.856 CPU sec, 26.659 real sec
[M::process] read 67606 sequences (10000206 bp)...
[M::mem_process_seqs] Processed 67658 reads in 27.049 CPU sec, 26.875 real sec
[M::process] read 67596 sequences (10000277 bp)...
[M::mem_process_seqs] Processed 67606 reads in 25.955 CPU sec, 25.779 real sec
[M::process] read 67628 sequences (10000008 bp)...
[M::mem_process_seqs] Processed 67596 reads in 26.497 CPU sec, 26.324 real sec
[M::process] read 67660 sequences (10000186 bp)...
[M::mem_process_seqs] Processed 67628 reads in 26.142 CPU sec, 25.974 real sec
[M::process] read 67600 sequences (10000230 bp)...
[M::mem_process_seqs] Processed 67660 reads in 27.454 CPU sec, 27.290 real sec
[M::process] read 67766 sequences (10000063 bp)...
[M::mem_process_seqs] Processed 67600 reads in 26.770 CPU sec, 26.625 real sec
[M::process] read 67710 sequences (10000029 bp)...
[M::mem_process_seqs] Processed 67766 reads in 30.958 CPU sec, 30.782 real sec
[M::process] read 67584 sequences (10000081 bp)...
[M::mem_process_seqs] Processed 67710 reads in 28.203 CPU sec, 28.028 real sec
[M::process] read 67590 sequences (10000242 bp)...
[M::mem_process_seqs] Processed 67584 reads in 26.245 CPU sec, 26.073 real sec
[M::process] read 67584 sequences (10000247 bp)...
[M::mem_process_seqs] Processed 67590 reads in 26.492 CPU sec, 26.368 real sec
[M::process] read 67644 sequences (10000040 bp)...
[M::mem_process_seqs] Processed 67584 reads in 27.128 CPU sec, 26.952 real sec
[M::process] read 67574 sequences (10000002 bp)...
[M::mem_process_seqs] Processed 67644 reads in 27.299 CPU sec, 27.123 real sec
[M::process] read 67614 sequences (10000132 bp)...
[M::mem_process_seqs] Processed 67574 reads in 27.336 CPU sec, 27.154 real sec
[M::process] read 67640 sequences (10000289 bp)...
[M::mem_process_seqs] Processed 67614 reads in 27.044 CPU sec, 26.864 real sec
[M::process] read 67618 sequences (10000030 bp)...
[M::mem_process_seqs] Processed 67640 reads in 28.273 CPU sec, 28.091 real sec
[M::process] read 67634 sequences (10000253 bp)...
[M::mem_process_seqs] Processed 67618 reads in 27.215 CPU sec, 27.070 real sec
[M::process] read 67634 sequences (10000254 bp)...
[M::mem_process_seqs] Processed 67634 reads in 27.440 CPU sec, 27.310 real sec
[M::process] read 67654 sequences (10000079 bp)...
[M::mem_process_seqs] Processed 67634 reads in 26.139 CPU sec, 25.980 real sec
[M::process] read 67726 sequences (10000204 bp)...
[M::mem_process_seqs] Processed 67654 reads in 26.292 CPU sec, 26.135 real sec
[M::process] read 67760 sequences (10000257 bp)...
[M::mem_process_seqs] Processed 67726 reads in 27.242 CPU sec, 27.076 real sec
[M::process] read 67696 sequences (10000201 bp)...
[M::mem_process_seqs] Processed 67760 reads in 27.944 CPU sec, 27.814 real sec
[M::process] read 67664 sequences (10000055 bp)...
[M::mem_process_seqs] Processed 67696 reads in 28.545 CPU sec, 28.355 real sec
[M::process] read 67700 sequences (10000222 bp)...
[M::mem_process_seqs] Processed 67664 reads in 27.715 CPU sec, 27.533 real sec
[M::process] read 67662 sequences (10000256 bp)...
[M::mem_process_seqs] Processed 67700 reads in 27.647 CPU sec, 27.503 real sec
[M::process] read 67628 sequences (10000055 bp)...
[M::mem_process_seqs] Processed 67662 reads in 27.004 CPU sec, 26.832 real sec
[M::process] read 67676 sequences (10000214 bp)...
[M::mem_process_seqs] Processed 67628 reads in 26.629 CPU sec, 26.454 real sec
[M::process] read 67632 sequences (10000285 bp)...
[M::mem_process_seqs] Processed 67676 reads in 26.789 CPU sec, 26.596 real sec
[M::process] read 67660 sequences (10000147 bp)...
[M::mem_process_seqs] Processed 67632 reads in 27.352 CPU sec, 27.207 real sec
[M::process] read 67618 sequences (10000280 bp)...
[M::mem_process_seqs] Processed 67660 reads in 27.101 CPU sec, 26.937 real sec
[M::process] read 67648 sequences (10000189 bp)...
[M::mem_process_seqs] Processed 67618 reads in 26.584 CPU sec, 26.407 real sec
[M::process] read 67622 sequences (10000184 bp)...
[M::mem_process_seqs] Processed 67648 reads in 28.815 CPU sec, 28.691 real sec
[M::process] read 67694 sequences (10000116 bp)...
[M::mem_process_seqs] Processed 67622 reads in 29.700 CPU sec, 29.547 real sec
[M::process] read 67770 sequences (10000282 bp)...
[M::mem_process_seqs] Processed 67694 reads in 27.862 CPU sec, 27.698 real sec
[M::process] read 67690 sequences (10000169 bp)...
[M::mem_process_seqs] Processed 67770 reads in 29.734 CPU sec, 29.570 real sec
[M::process] read 67734 sequences (10000123 bp)...
[M::mem_process_seqs] Processed 67690 reads in 28.284 CPU sec, 28.175 real sec
[M::process] read 67718 sequences (10000240 bp)...
[M::mem_process_seqs] Processed 67734 reads in 33.712 CPU sec, 33.560 real sec
[M::process] read 67706 sequences (10000284 bp)...
[M::mem_process_seqs] Processed 67718 reads in 32.374 CPU sec, 32.222 real sec
[M::process] read 63979 sequences (9452505 bp)...
[M::mem_process_seqs] Processed 67706 reads in 51.398 CPU sec, 51.249 real sec
[M::mem_process_seqs] Processed 63979 reads in 51.005 CPU sec, 50.881 real sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs.fa /kb/module/work/tmp/a3b6bc8f-059a-4060-ad5f-4dde8b9fa4a7.fwd.fastq
[main] Real time: 1202.048 sec; CPU: 1208.741 sec
command: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs.fa /kb/module/work/tmp/8b262464-09e9-471a-8965-5efd8feae319.rev.fastq > /kb/module/work/tmp/alignments2_7e67865d-fec3-40f2-994f-d94d779eca7b.sam
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 67702 sequences (10000090 bp)...
[M::process] read 67734 sequences (10000255 bp)...
[M::mem_process_seqs] Processed 67702 reads in 80.064 CPU sec, 79.977 real sec
[M::process] read 67746 sequences (10000291 bp)...
[M::mem_process_seqs] Processed 67734 reads in 40.034 CPU sec, 39.823 real sec
[M::process] read 67820 sequences (10000299 bp)...
[M::mem_process_seqs] Processed 67746 reads in 63.867 CPU sec, 63.685 real sec
[M::process] read 67766 sequences (10000003 bp)...
[M::mem_process_seqs] Processed 67820 reads in 52.149 CPU sec, 51.991 real sec
[M::process] read 67788 sequences (10000216 bp)...
[M::mem_process_seqs] Processed 67766 reads in 42.408 CPU sec, 42.266 real sec
[M::process] read 67920 sequences (10000297 bp)...
[M::mem_process_seqs] Processed 67788 reads in 42.107 CPU sec, 41.987 real sec
[M::process] read 67884 sequences (10000001 bp)...
[M::mem_process_seqs] Processed 67920 reads in 42.811 CPU sec, 42.651 real sec
[M::process] read 67724 sequences (10000294 bp)...
[M::mem_process_seqs] Processed 67884 reads in 45.339 CPU sec, 45.239 real sec
[M::process] read 68094 sequences (10000258 bp)...
[M::mem_process_seqs] Processed 67724 reads in 43.177 CPU sec, 43.013 real sec
[M::process] read 67870 sequences (10000209 bp)...
[M::mem_process_seqs] Processed 68094 reads in 51.593 CPU sec, 51.432 real sec
[M::process] read 67780 sequences (10000193 bp)...
[M::mem_process_seqs] Processed 67870 reads in 40.664 CPU sec, 40.495 real sec
[M::process] read 67742 sequences (10000177 bp)...
[M::mem_process_seqs] Processed 67780 reads in 67.747 CPU sec, 67.617 real sec
[M::process] read 67724 sequences (10000069 bp)...
[M::mem_process_seqs] Processed 67742 reads in 84.950 CPU sec, 85.093 real sec
[M::process] read 67754 sequences (10000217 bp)...
[M::mem_process_seqs] Processed 67724 reads in 81.066 CPU sec, 81.106 real sec
[M::process] read 67724 sequences (10000261 bp)...
[M::mem_process_seqs] Processed 67754 reads in 79.169 CPU sec, 79.328 real sec
[M::process] read 67742 sequences (10000297 bp)...
[M::mem_process_seqs] Processed 67724 reads in 78.297 CPU sec, 78.463 real sec
[M::process] read 67762 sequences (10000288 bp)...
[M::mem_process_seqs] Processed 67742 reads in 76.013 CPU sec, 76.104 real sec
[M::process] read 67750 sequences (10000179 bp)...
[M::mem_process_seqs] Processed 67762 reads in 55.408 CPU sec, 55.287 real sec
[M::process] read 67734 sequences (10000222 bp)...
[M::mem_process_seqs] Processed 67750 reads in 71.109 CPU sec, 70.958 real sec
[M::process] read 67686 sequences (10000156 bp)...
[M::mem_process_seqs] Processed 67734 reads in 71.333 CPU sec, 71.184 real sec
[M::process] read 67666 sequences (10000235 bp)...
[M::mem_process_seqs] Processed 67686 reads in 62.009 CPU sec, 61.867 real sec
[M::process] read 67708 sequences (10000251 bp)...
[M::mem_process_seqs] Processed 67666 reads in 60.974 CPU sec, 60.929 real sec
[M::process] read 67762 sequences (10000275 bp)...
[M::mem_process_seqs] Processed 67708 reads in 73.368 CPU sec, 73.452 real sec
[M::process] read 67700 sequences (10000121 bp)...
[M::mem_process_seqs] Processed 67762 reads in 75.838 CPU sec, 75.870 real sec
[M::process] read 67628 sequences (10000262 bp)...
[M::mem_process_seqs] Processed 67700 reads in 75.741 CPU sec, 75.701 real sec
[M::process] read 67678 sequences (10000014 bp)...
[M::mem_process_seqs] Processed 67628 reads in 71.066 CPU sec, 71.151 real sec
[M::process] read 67658 sequences (10000214 bp)...
[M::mem_process_seqs] Processed 67678 reads in 71.685 CPU sec, 71.835 real sec
[M::process] read 67618 sequences (10000295 bp)...
[M::mem_process_seqs] Processed 67658 reads in 71.974 CPU sec, 71.936 real sec
[M::process] read 67650 sequences (10000104 bp)...
[M::mem_process_seqs] Processed 67618 reads in 69.961 CPU sec, 70.074 real sec
[M::process] read 67712 sequences (10000069 bp)...
[M::mem_process_seqs] Processed 67650 reads in 72.627 CPU sec, 73.290 real sec
[M::process] read 67670 sequences (10000127 bp)...
[M::mem_process_seqs] Processed 67712 reads in 69.296 CPU sec, 69.224 real sec
[M::process] read 67596 sequences (10000252 bp)...
[M::mem_process_seqs] Processed 67670 reads in 243.667 CPU sec, 243.554 real sec
[M::process] read 67632 sequences (10000164 bp)...
[M::mem_process_seqs] Processed 67596 reads in 334.865 CPU sec, 335.050 real sec
[M::process] read 67618 sequences (10000009 bp)...
[M::mem_process_seqs] Processed 67632 reads in 67.470 CPU sec, 67.330 real sec
[M::process] read 67654 sequences (10000051 bp)...
[M::mem_process_seqs] Processed 67618 reads in 67.056 CPU sec, 66.986 real sec
[M::process] read 67694 sequences (10000119 bp)...
[M::mem_process_seqs] Processed 67654 reads in 58.465 CPU sec, 58.397 real sec
[M::process] read 67648 sequences (10000199 bp)...
[M::mem_process_seqs] Processed 67694 reads in 46.558 CPU sec, 46.412 real sec
[M::process] read 67714 sequences (10000081 bp)...
[M::mem_process_seqs] Processed 67648 reads in 42.931 CPU sec, 42.791 real sec
[M::process] read 67708 sequences (10000146 bp)...
[M::mem_process_seqs] Processed 67714 reads in 44.162 CPU sec, 44.012 real sec
[M::process] read 67682 sequences (10000173 bp)...
[M::mem_process_seqs] Processed 67708 reads in 48.936 CPU sec, 48.779 real sec
[M::process] read 60839 sequences (8992892 bp)...
[M::mem_process_seqs] Processed 67682 reads in 57.656 CPU sec, 57.540 real sec
[M::mem_process_seqs] Processed 60839 reads in 36.671 CPU sec, 36.553 real sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs.fa /kb/module/work/tmp/8b262464-09e9-471a-8965-5efd8feae319.rev.fastq
[main] Real time: 3030.742 sec; CPU: 3032.589 sec
command: polypolish filter --in1 /kb/module/work/tmp/alignments1_debfe222-d36c-4c68-88e2-247996292665.sam --in2 /kb/module/work/tmp/alignments2_7e67865d-fec3-40f2-994f-d94d779eca7b.sam --out1 /kb/module/work/tmp/filtered1_4119fafb-b16c-4fd1-b13b-699bce819032.sam --out2 /kb/module/work/tmp/filtered2_fa99596b-8b3b-4a3e-82c8-388c2055b3b1.sam
[1;4;93mStarting Polypolish filter[0m [2m(2024-06-24 16:27:01)[0m
[2m This runs a pre-processing filter on SAM alignments before they are used to
polish. It looks at each read pair and flags alignments that do not seem to be
part of a concordant pair. This can improve the accuracy Polypolish, especially
near the edges of repeats.[0m
Polypolish version: 0.6.0
Input alignments:
/kb/module/work/tmp/alignments1_debfe222-d36c-4c68-88e2-247996292665.sam
/kb/module/work/tmp/alignments2_7e67865d-fec3-40f2-994f-d94d779eca7b.sam
Output alignments:
/kb/module/work/tmp/filtered1_4119fafb-b16c-4fd1-b13b-699bce819032.sam
/kb/module/work/tmp/filtered2_fa99596b-8b3b-4a3e-82c8-388c2055b3b1.sam
Settings:
--orientation auto
--low 0.1
--high 99.9
[1;4;93mLoading alignments[0m [2m(2024-06-24 16:27:01)[0m
/kb/module/work/tmp/alignments1_debfe222-d36c-4c68-88e2-247996292665.sam: 6,395,804 alignments from 2,837,681 reads
/kb/module/work/tmp/alignments2_7e67865d-fec3-40f2-994f-d94d779eca7b.sam: 6,784,192 alignments from 2,837,681 reads
[1;4;93mFinding insert size thresholds[0m [2m(2024-06-24 16:27:45)[0m
[2m Read pairs with exactly one alignment per read are used to determine the
orientation and insert size thresholds for the read set.[0m
fr: 2,685,519 pairs
rf: 75 pairs
ff: 163 pairs
rr: 144 pairs
Automatically determined correct orientation: fr
Low threshold: 43 (0.1st percentile)
High threshold: 798 (99.9th percentile)
[1;4;93mFiltering SAM files[0m [2m(2024-06-24 16:27:54)[0m
[2m Read alignments that are part of a good pair (correct orientation and
insert size) pass the filter and are written unaltered to the output file. Read
alignments which are not part of good pair are written to the output file with a
"ZP:Z:fail" tag so Polypolish will not use them.[0m
Filtering /kb/module/work/tmp/alignments1_debfe222-d36c-4c68-88e2-247996292665.sam:
3,783,618 pass
2,612,186 fail
Filtering /kb/module/work/tmp/alignments2_7e67865d-fec3-40f2-994f-d94d779eca7b.sam:
3,788,103 pass
2,996,089 fail
[1;4;93mFinished![0m [2m(2024-06-24 17:29:40)[0m
Alignments before filtering: 13,179,996
Alignments after filtering: 7,571,721
Time to run: 1:02:38.919129
command: polypolish polish --fraction_invalid 0.2 --fraction_valid 0.5 --max_errors 10 --min_depth 5 /kb/module/work/tmp/flye.contigs.fa /kb/module/work/tmp/filtered1_4119fafb-b16c-4fd1-b13b-699bce819032.sam /kb/module/work/tmp/filtered2_fa99596b-8b3b-4a3e-82c8-388c2055b3b1.sam > /kb/module/work/tmp/polypolish_output_d2e73f03-f8ed-4ff3-9afb-df3522e55fc2.fasta
[1;4;93mStarting Polypolish polish[0m [2m(2024-06-24 17:29:56)[0m
[2m Polypolish is a tool for polishing genome assemblies with short reads.
Unlike other tools in this category, Polypolish uses SAM files where each read
has been aligned to all possible locations (not just a single best location).
This allows it to repair errors in repeat regions that other alignment-based
polishers cannot fix.[0m
Polypolish version: 0.6.0
Input assembly:
/kb/module/work/tmp/flye.contigs.fa
Input short-read alignments:
/kb/module/work/tmp/filtered1_4119fafb-b16c-4fd1-b13b-699bce819032.sam
/kb/module/work/tmp/filtered2_fa99596b-8b3b-4a3e-82c8-388c2055b3b1.sam
Settings:
--fraction_invalid 0.2
--fraction_valid 0.5
--max_errors 10
--min_depth 5
not logging debugging information
[1;4;93mLoading assembly[0m [2m(2024-06-24 17:29:56)[0m
contig_1 (3,896,935 bp)
[1;4;93mLoading alignments[0m [2m(2024-06-24 17:29:56)[0m
/kb/module/work/tmp/filtered1_4119fafb-b16c-4fd1-b13b-699bce819032.sam: 6,395,804 alignments from 2,837,681 reads
/kb/module/work/tmp/filtered2_fa99596b-8b3b-4a3e-82c8-388c2055b3b1.sam: 6,784,192 alignments from 2,837,681 reads
Filtering for high-quality end-to-end alignments:
6,690,994 alignments kept
6,489,002 alignments discarded
[1;4;93mPolishing assembly sequences[0m [2m(2024-06-24 17:31:07)[0m
[2m For each position in the assembly, Polypolish determines the read depth
at that position and collects all aligned bases. It then polishes the assembly
by looking for positions where the pileup unambiguously supports a different
sequence than the assembly.[0m
Polishing contig_1 (3,896,935 bp):
mean read depth: 210.0x
10 bp have a depth of zero (99.9997% coverage)
56 positions changed (0.0014% of total positions)
estimated pre-polishing sequence accuracy: 99.9986% (Q48.43)
[1;4;93mFinished![0m [2m(2024-06-24 17:31:09)[0m
Polished sequence (to stdout):
contig_1_polypolish (3,896,973 bp)
Time to run: 0:01:13.023559
Generating and saving report.
Polypolish results saved.