Getting short reads.
Getting assembly.
command: bwa index /kb/module/work/tmp/flye.contigs.fa
[bwa_index] Pack FASTA... 0.03 sec
[bwa_index] Construct BWT for the packed sequence...
[bwa_index] 1.39 seconds elapse.
[bwa_index] Update BWT... 0.03 sec
[bwa_index] Pack forward-only FASTA... 0.02 sec
[bwa_index] Construct SA from BWT and Occ... 0.39 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index /kb/module/work/tmp/flye.contigs.fa
[main] Real time: 1.872 sec; CPU: 1.853 sec
command: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs.fa /kb/module/work/tmp/2a1d3644-ec55-4486-b062-6c92d968a89a.fwd.fastq > /kb/module/work/tmp/alignments1_e992e7c6-0923-4ad3-be30-47392a9d2b2a.sam
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 67502 sequences (10000057 bp)...
[M::process] read 67508 sequences (10000253 bp)...
[M::mem_process_seqs] Processed 67502 reads in 23.490 CPU sec, 23.421 real sec
[M::process] read 67552 sequences (10000057 bp)...
[M::mem_process_seqs] Processed 67508 reads in 24.677 CPU sec, 24.575 real sec
[M::process] read 67484 sequences (10000200 bp)...
[M::mem_process_seqs] Processed 67552 reads in 24.918 CPU sec, 24.757 real sec
[M::process] read 67530 sequences (10000292 bp)...
[M::mem_process_seqs] Processed 67484 reads in 23.740 CPU sec, 23.581 real sec
[M::process] read 67542 sequences (10000245 bp)...
[M::mem_process_seqs] Processed 67530 reads in 25.355 CPU sec, 25.196 real sec
[M::process] read 67556 sequences (10000218 bp)...
[M::mem_process_seqs] Processed 67542 reads in 24.925 CPU sec, 24.768 real sec
[M::process] read 67490 sequences (10000293 bp)...
[M::mem_process_seqs] Processed 67556 reads in 24.945 CPU sec, 24.801 real sec
[M::process] read 67458 sequences (10000298 bp)...
[M::mem_process_seqs] Processed 67490 reads in 24.561 CPU sec, 24.453 real sec
[M::process] read 67668 sequences (10000237 bp)...
[M::mem_process_seqs] Processed 67458 reads in 23.891 CPU sec, 23.735 real sec
[M::process] read 67704 sequences (10000023 bp)...
[M::mem_process_seqs] Processed 67668 reads in 28.143 CPU sec, 27.988 real sec
[M::process] read 67510 sequences (10000010 bp)...
[M::mem_process_seqs] Processed 67704 reads in 29.052 CPU sec, 28.897 real sec
[M::process] read 67466 sequences (10000287 bp)...
[M::mem_process_seqs] Processed 67510 reads in 24.704 CPU sec, 24.550 real sec
[M::process] read 67532 sequences (10000243 bp)...
[M::mem_process_seqs] Processed 67466 reads in 24.320 CPU sec, 24.164 real sec
[M::process] read 67516 sequences (10000233 bp)...
[M::mem_process_seqs] Processed 67532 reads in 25.263 CPU sec, 25.153 real sec
[M::process] read 67488 sequences (10000006 bp)...
[M::mem_process_seqs] Processed 67516 reads in 25.778 CPU sec, 25.658 real sec
[M::process] read 67548 sequences (10000055 bp)...
[M::mem_process_seqs] Processed 67488 reads in 24.435 CPU sec, 24.299 real sec
[M::process] read 67524 sequences (10000262 bp)...
[M::mem_process_seqs] Processed 67548 reads in 25.679 CPU sec, 25.535 real sec
[M::process] read 67526 sequences (10000112 bp)...
[M::mem_process_seqs] Processed 67524 reads in 25.164 CPU sec, 25.026 real sec
[M::process] read 67526 sequences (10000043 bp)...
[M::mem_process_seqs] Processed 67526 reads in 25.087 CPU sec, 24.918 real sec
[M::process] read 67540 sequences (10000115 bp)...
[M::mem_process_seqs] Processed 67526 reads in 24.027 CPU sec, 23.868 real sec
[M::process] read 67522 sequences (10000153 bp)...
[M::mem_process_seqs] Processed 67540 reads in 24.355 CPU sec, 24.201 real sec
[M::process] read 67618 sequences (10000190 bp)...
[M::mem_process_seqs] Processed 67522 reads in 23.210 CPU sec, 23.051 real sec
[M::process] read 67556 sequences (10000101 bp)...
[M::mem_process_seqs] Processed 67618 reads in 24.330 CPU sec, 24.214 real sec
[M::process] read 67658 sequences (10000173 bp)...
[M::mem_process_seqs] Processed 67556 reads in 23.517 CPU sec, 23.362 real sec
[M::process] read 67648 sequences (10000213 bp)...
[M::mem_process_seqs] Processed 67658 reads in 23.799 CPU sec, 23.657 real sec
[M::process] read 67638 sequences (10000205 bp)...
[M::mem_process_seqs] Processed 67648 reads in 23.928 CPU sec, 23.769 real sec
[M::process] read 67590 sequences (10000207 bp)...
[M::mem_process_seqs] Processed 67638 reads in 23.455 CPU sec, 23.337 real sec
[M::process] read 67584 sequences (10000048 bp)...
[M::mem_process_seqs] Processed 67590 reads in 23.227 CPU sec, 23.072 real sec
[M::process] read 67542 sequences (10000061 bp)...
[M::mem_process_seqs] Processed 67584 reads in 22.692 CPU sec, 22.537 real sec
[M::process] read 67646 sequences (10000189 bp)...
[M::mem_process_seqs] Processed 67542 reads in 23.234 CPU sec, 23.105 real sec
[M::process] read 67644 sequences (10000013 bp)...
[M::mem_process_seqs] Processed 67646 reads in 23.017 CPU sec, 22.859 real sec
[M::process] read 67512 sequences (10000021 bp)...
[M::mem_process_seqs] Processed 67644 reads in 23.610 CPU sec, 23.476 real sec
[M::process] read 67548 sequences (10000219 bp)...
[M::mem_process_seqs] Processed 67512 reads in 22.873 CPU sec, 22.715 real sec
[M::process] read 67558 sequences (10000156 bp)...
[M::mem_process_seqs] Processed 67548 reads in 23.147 CPU sec, 23.048 real sec
[M::process] read 67554 sequences (10000259 bp)...
[M::mem_process_seqs] Processed 67558 reads in 22.945 CPU sec, 22.780 real sec
[M::process] read 67680 sequences (10000063 bp)...
[M::mem_process_seqs] Processed 67554 reads in 22.944 CPU sec, 22.818 real sec
[M::process] read 67558 sequences (10000018 bp)...
[M::mem_process_seqs] Processed 67680 reads in 23.787 CPU sec, 23.628 real sec
[M::process] read 67658 sequences (10000228 bp)...
[M::mem_process_seqs] Processed 67558 reads in 23.014 CPU sec, 22.877 real sec
[M::process] read 67622 sequences (10000074 bp)...
[M::mem_process_seqs] Processed 67658 reads in 24.739 CPU sec, 24.579 real sec
[M::process] read 67610 sequences (10000241 bp)...
[M::mem_process_seqs] Processed 67622 reads in 24.125 CPU sec, 23.997 real sec
[M::process] read 55008 sequences (8135032 bp)...
[M::mem_process_seqs] Processed 67610 reads in 23.515 CPU sec, 23.374 real sec
[M::mem_process_seqs] Processed 55008 reads in 19.592 CPU sec, 19.501 real sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs.fa /kb/module/work/tmp/2a1d3644-ec55-4486-b062-6c92d968a89a.fwd.fastq
[main] Real time: 1009.474 sec; CPU: 1015.365 sec
command: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs.fa /kb/module/work/tmp/5bb8b20b-6d01-44a6-b295-1e9cbd96626a.rev.fastq > /kb/module/work/tmp/alignments2_5a86d64e-9eae-4348-89f9-a7ecbde15af5.sam
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 67580 sequences (10000002 bp)...
[M::process] read 67648 sequences (10000038 bp)...
[M::mem_process_seqs] Processed 67580 reads in 29.523 CPU sec, 29.447 real sec
[M::process] read 67674 sequences (10000097 bp)...
[M::mem_process_seqs] Processed 67648 reads in 30.498 CPU sec, 30.351 real sec
[M::process] read 67590 sequences (10000256 bp)...
[M::mem_process_seqs] Processed 67674 reads in 31.622 CPU sec, 31.444 real sec
[M::process] read 67658 sequences (10000027 bp)...
[M::mem_process_seqs] Processed 67590 reads in 28.816 CPU sec, 28.668 real sec
[M::process] read 67716 sequences (10000028 bp)...
[M::mem_process_seqs] Processed 67658 reads in 31.479 CPU sec, 31.318 real sec
[M::process] read 67856 sequences (10000098 bp)...
[M::mem_process_seqs] Processed 67716 reads in 31.345 CPU sec, 31.183 real sec
[M::process] read 67718 sequences (10000120 bp)...
[M::mem_process_seqs] Processed 67856 reads in 31.352 CPU sec, 31.179 real sec
[M::process] read 67574 sequences (10000081 bp)...
[M::mem_process_seqs] Processed 67718 reads in 31.413 CPU sec, 31.242 real sec
[M::process] read 67998 sequences (10000090 bp)...
[M::mem_process_seqs] Processed 67574 reads in 28.756 CPU sec, 28.593 real sec
[M::process] read 67892 sequences (10000028 bp)...
[M::mem_process_seqs] Processed 67998 reads in 35.477 CPU sec, 35.310 real sec
[M::process] read 67686 sequences (10000071 bp)...
[M::mem_process_seqs] Processed 67892 reads in 35.119 CPU sec, 34.955 real sec
[M::process] read 67616 sequences (10000083 bp)...
[M::mem_process_seqs] Processed 67686 reads in 28.960 CPU sec, 28.793 real sec
[M::process] read 67704 sequences (10000269 bp)...
[M::mem_process_seqs] Processed 67616 reads in 28.637 CPU sec, 28.479 real sec
[M::process] read 67686 sequences (10000070 bp)...
[M::mem_process_seqs] Processed 67704 reads in 28.896 CPU sec, 28.777 real sec
[M::process] read 67582 sequences (10000290 bp)...
[M::mem_process_seqs] Processed 67686 reads in 28.331 CPU sec, 28.164 real sec
[M::process] read 67716 sequences (10000133 bp)...
[M::mem_process_seqs] Processed 67582 reads in 26.710 CPU sec, 26.581 real sec
[M::process] read 67614 sequences (10000270 bp)...
[M::mem_process_seqs] Processed 67716 reads in 28.212 CPU sec, 28.074 real sec
[M::process] read 67648 sequences (10000018 bp)...
[M::mem_process_seqs] Processed 67614 reads in 27.837 CPU sec, 27.686 real sec
[M::process] read 67576 sequences (10000296 bp)...
[M::mem_process_seqs] Processed 67648 reads in 28.451 CPU sec, 28.316 real sec
[M::process] read 67622 sequences (10000217 bp)...
[M::mem_process_seqs] Processed 67576 reads in 26.347 CPU sec, 26.204 real sec
[M::process] read 67572 sequences (10000153 bp)...
[M::mem_process_seqs] Processed 67622 reads in 26.927 CPU sec, 26.799 real sec
[M::process] read 67590 sequences (10000142 bp)...
[M::mem_process_seqs] Processed 67572 reads in 25.254 CPU sec, 25.116 real sec
[M::process] read 67512 sequences (10000149 bp)...
[M::mem_process_seqs] Processed 67590 reads in 27.131 CPU sec, 26.974 real sec
[M::process] read 67606 sequences (10000189 bp)...
[M::mem_process_seqs] Processed 67512 reads in 25.570 CPU sec, 25.440 real sec
[M::process] read 67610 sequences (10000015 bp)...
[M::mem_process_seqs] Processed 67606 reads in 26.729 CPU sec, 26.604 real sec
[M::process] read 67602 sequences (10000279 bp)...
[M::mem_process_seqs] Processed 67610 reads in 26.084 CPU sec, 25.920 real sec
[M::process] read 67562 sequences (10000206 bp)...
[M::mem_process_seqs] Processed 67602 reads in 25.053 CPU sec, 24.882 real sec
[M::process] read 67572 sequences (10000231 bp)...
[M::mem_process_seqs] Processed 67562 reads in 24.691 CPU sec, 24.577 real sec
[M::process] read 67508 sequences (10000266 bp)...
[M::mem_process_seqs] Processed 67572 reads in 25.048 CPU sec, 24.886 real sec
[M::process] read 67630 sequences (10000127 bp)...
[M::mem_process_seqs] Processed 67508 reads in 24.909 CPU sec, 24.761 real sec
[M::process] read 67642 sequences (10000211 bp)...
[M::mem_process_seqs] Processed 67630 reads in 24.459 CPU sec, 24.324 real sec
[M::process] read 67476 sequences (10000198 bp)...
[M::mem_process_seqs] Processed 67642 reads in 78.968 CPU sec, 78.809 real sec
[M::process] read 67526 sequences (10000254 bp)...
[M::mem_process_seqs] Processed 67476 reads in 183.123 CPU sec, 182.971 real sec
[M::process] read 67536 sequences (10000166 bp)...
[M::mem_process_seqs] Processed 67526 reads in 25.593 CPU sec, 25.430 real sec
[M::process] read 67510 sequences (10000125 bp)...
[M::mem_process_seqs] Processed 67536 reads in 25.936 CPU sec, 25.811 real sec
[M::process] read 67552 sequences (10000271 bp)...
[M::mem_process_seqs] Processed 67510 reads in 26.506 CPU sec, 26.299 real sec
[M::process] read 67536 sequences (10000273 bp)...
[M::mem_process_seqs] Processed 67552 reads in 26.939 CPU sec, 26.775 real sec
[M::process] read 67592 sequences (10000291 bp)...
[M::mem_process_seqs] Processed 67536 reads in 24.850 CPU sec, 24.704 real sec
[M::process] read 67572 sequences (10000136 bp)...
[M::mem_process_seqs] Processed 67592 reads in 25.780 CPU sec, 25.607 real sec
[M::process] read 67550 sequences (10000059 bp)...
[M::mem_process_seqs] Processed 67572 reads in 24.704 CPU sec, 24.568 real sec
[M::process] read 52514 sequences (7772335 bp)...
[M::mem_process_seqs] Processed 67550 reads in 24.862 CPU sec, 24.741 real sec
[M::mem_process_seqs] Processed 52514 reads in 18.908 CPU sec, 18.839 real sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs.fa /kb/module/work/tmp/5bb8b20b-6d01-44a6-b295-1e9cbd96626a.rev.fastq
[main] Real time: 1359.785 sec; CPU: 1365.959 sec
command: polypolish filter --in1 /kb/module/work/tmp/alignments1_e992e7c6-0923-4ad3-be30-47392a9d2b2a.sam --in2 /kb/module/work/tmp/alignments2_5a86d64e-9eae-4348-89f9-a7ecbde15af5.sam --out1 /kb/module/work/tmp/filtered1_a825800c-9819-4158-a95b-5c0a17f836b2.sam --out2 /kb/module/work/tmp/filtered2_3aaa2874-4034-477b-820f-16cce6e2b77f.sam
[1;4;93mStarting Polypolish filter[0m [2m(2024-06-24 03:03:55)[0m
[2m This runs a pre-processing filter on SAM alignments before they are used to
polish. It looks at each read pair and flags alignments that do not seem to be
part of a concordant pair. This can improve the accuracy Polypolish, especially
near the edges of repeats.[0m
Polypolish version: 0.6.0
Input alignments:
/kb/module/work/tmp/alignments1_e992e7c6-0923-4ad3-be30-47392a9d2b2a.sam
/kb/module/work/tmp/alignments2_5a86d64e-9eae-4348-89f9-a7ecbde15af5.sam
Output alignments:
/kb/module/work/tmp/filtered1_a825800c-9819-4158-a95b-5c0a17f836b2.sam
/kb/module/work/tmp/filtered2_3aaa2874-4034-477b-820f-16cce6e2b77f.sam
Settings:
--orientation auto
--low 0.1
--high 99.9
[1;4;93mLoading alignments[0m [2m(2024-06-24 03:03:55)[0m
/kb/module/work/tmp/alignments1_e992e7c6-0923-4ad3-be30-47392a9d2b2a.sam: 3,829,378 alignments from 2,825,124 reads
/kb/module/work/tmp/alignments2_5a86d64e-9eae-4348-89f9-a7ecbde15af5.sam: 8,156,857 alignments from 2,825,124 reads
[1;4;93mFinding insert size thresholds[0m [2m(2024-06-24 03:04:27)[0m
[2m Read pairs with exactly one alignment per read are used to determine the
orientation and insert size thresholds for the read set.[0m
fr: 2,685,582 pairs
rf: 30 pairs
ff: 115 pairs
rr: 105 pairs
Automatically determined correct orientation: fr
Low threshold: 43 (0.1st percentile)
High threshold: 816 (99.9th percentile)
[1;4;93mFiltering SAM files[0m [2m(2024-06-24 03:04:32)[0m
[2m Read alignments that are part of a good pair (correct orientation and
insert size) pass the filter and are written unaltered to the output file. Read
alignments which are not part of good pair are written to the output file with a
"ZP:Z:fail" tag so Polypolish will not use them.[0m
Filtering /kb/module/work/tmp/alignments1_e992e7c6-0923-4ad3-be30-47392a9d2b2a.sam:
3,607,867 pass
221,511 fail
Filtering /kb/module/work/tmp/alignments2_5a86d64e-9eae-4348-89f9-a7ecbde15af5.sam:
3,616,342 pass
4,540,515 fail
[1;4;93mFinished![0m [2m(2024-06-24 03:06:22)[0m
Alignments before filtering: 11,986,235
Alignments after filtering: 7,224,209
Time to run: 0:02:27.241884
command: polypolish polish --fraction_invalid 0.2 --fraction_valid 0.5 --max_errors 10 --min_depth 5 /kb/module/work/tmp/flye.contigs.fa /kb/module/work/tmp/filtered1_a825800c-9819-4158-a95b-5c0a17f836b2.sam /kb/module/work/tmp/filtered2_3aaa2874-4034-477b-820f-16cce6e2b77f.sam > /kb/module/work/tmp/polypolish_output_a62743d4-aa15-41ce-a959-649f5ca0623a.fasta
[1;4;93mStarting Polypolish polish[0m [2m(2024-06-24 03:06:35)[0m
[2m Polypolish is a tool for polishing genome assemblies with short reads.
Unlike other tools in this category, Polypolish uses SAM files where each read
has been aligned to all possible locations (not just a single best location).
This allows it to repair errors in repeat regions that other alignment-based
polishers cannot fix.[0m
Polypolish version: 0.6.0
Input assembly:
/kb/module/work/tmp/flye.contigs.fa
Input short-read alignments:
/kb/module/work/tmp/filtered1_a825800c-9819-4158-a95b-5c0a17f836b2.sam
/kb/module/work/tmp/filtered2_3aaa2874-4034-477b-820f-16cce6e2b77f.sam
Settings:
--fraction_invalid 0.2
--fraction_valid 0.5
--max_errors 10
--min_depth 5
not logging debugging information
[1;4;93mLoading assembly[0m [2m(2024-06-24 03:06:35)[0m
contig_1 (3,833,949 bp)
contig_2 (6,629 bp)
[1;4;93mLoading alignments[0m [2m(2024-06-24 03:06:35)[0m
/kb/module/work/tmp/filtered1_a825800c-9819-4158-a95b-5c0a17f836b2.sam: 3,829,378 alignments from 2,825,124 reads
/kb/module/work/tmp/filtered2_3aaa2874-4034-477b-820f-16cce6e2b77f.sam: 8,156,857 alignments from 2,825,124 reads
Filtering for high-quality end-to-end alignments:
7,117,930 alignments kept
4,868,305 alignments discarded
[1;4;93mPolishing assembly sequences[0m [2m(2024-06-24 03:07:28)[0m
[2m For each position in the assembly, Polypolish determines the read depth
at that position and collects all aligned bases. It then polishes the assembly
by looking for positions where the pileup unambiguously supports a different
sequence than the assembly.[0m
Polishing contig_1 (3,833,949 bp):
mean read depth: 213.8x
3 bp have a depth of zero (99.9999% coverage)
43 positions changed (0.0011% of total positions)
estimated pre-polishing sequence accuracy: 99.9989% (Q49.50)
Polishing contig_2 (6,629 bp):
mean read depth: 198.9x
4 bp have a depth of zero (99.9397% coverage)
5 positions changed (0.0754% of total positions)
estimated pre-polishing sequence accuracy: 99.9246% (Q31.22)
[1;4;93mFinished![0m [2m(2024-06-24 03:07:30)[0m
Polished sequence (to stdout):
contig_1_polypolish (3,833,975 bp)
contig_2_polypolish (6,630 bp)
Time to run: 0:00:54.855012
Generating and saving report.
Polypolish results saved.