Getting short reads.
Getting assembly.
command: bwa index /kb/module/work/tmp/flye.contigs_Colony9.fa
[bwa_index] Pack FASTA... 0.04 sec
[bwa_index] Construct BWT for the packed sequence...
[bwa_index] 1.62 seconds elapse.
[bwa_index] Update BWT... 0.03 sec
[bwa_index] Pack forward-only FASTA... 0.02 sec
[bwa_index] Construct SA from BWT and Occ... 0.44 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index /kb/module/work/tmp/flye.contigs_Colony9.fa
[main] Real time: 2.160 sec; CPU: 2.143 sec
command: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs_Colony9.fa /kb/module/work/tmp/8882bcff-46b0-400a-b71d-8d90b640ef9e.fwd.fastq > /kb/module/work/tmp/alignments1_53886e02-0157-4994-b628-b6ac1d748834.sam
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 67488 sequences (10000285 bp)...
[M::process] read 67548 sequences (10000264 bp)...
[M::mem_process_seqs] Processed 67488 reads in 32.055 CPU sec, 31.984 real sec
[M::process] read 67474 sequences (10000075 bp)...
[M::mem_process_seqs] Processed 67548 reads in 30.729 CPU sec, 30.555 real sec
[M::process] read 67538 sequences (10000104 bp)...
[M::mem_process_seqs] Processed 67474 reads in 28.211 CPU sec, 28.042 real sec
[M::process] read 67524 sequences (10000020 bp)...
[M::mem_process_seqs] Processed 67538 reads in 28.556 CPU sec, 28.362 real sec
[M::process] read 67500 sequences (10000141 bp)...
[M::mem_process_seqs] Processed 67524 reads in 30.498 CPU sec, 30.305 real sec
[M::process] read 67498 sequences (10000062 bp)...
[M::mem_process_seqs] Processed 67500 reads in 29.149 CPU sec, 28.961 real sec
[M::process] read 67526 sequences (10000264 bp)...
[M::mem_process_seqs] Processed 67498 reads in 28.820 CPU sec, 28.672 real sec
[M::process] read 67494 sequences (10000280 bp)...
[M::mem_process_seqs] Processed 67526 reads in 30.337 CPU sec, 30.184 real sec
[M::process] read 67548 sequences (10000028 bp)...
[M::mem_process_seqs] Processed 67494 reads in 28.926 CPU sec, 28.735 real sec
[M::process] read 67554 sequences (10000216 bp)...
[M::mem_process_seqs] Processed 67548 reads in 29.378 CPU sec, 29.222 real sec
[M::process] read 67698 sequences (10000253 bp)...
[M::mem_process_seqs] Processed 67554 reads in 30.322 CPU sec, 30.180 real sec
[M::process] read 67532 sequences (10000064 bp)...
[M::mem_process_seqs] Processed 67698 reads in 33.627 CPU sec, 33.460 real sec
[M::process] read 67508 sequences (10000280 bp)...
[M::mem_process_seqs] Processed 67532 reads in 29.548 CPU sec, 29.352 real sec
[M::process] read 67464 sequences (10000243 bp)...
[M::mem_process_seqs] Processed 67508 reads in 28.574 CPU sec, 28.386 real sec
[M::process] read 67482 sequences (10000184 bp)...
[M::mem_process_seqs] Processed 67464 reads in 27.889 CPU sec, 27.692 real sec
[M::process] read 67484 sequences (10000213 bp)...
[M::mem_process_seqs] Processed 67482 reads in 28.857 CPU sec, 28.666 real sec
[M::process] read 67500 sequences (10000192 bp)...
[M::mem_process_seqs] Processed 67484 reads in 28.095 CPU sec, 27.896 real sec
[M::process] read 67524 sequences (10000268 bp)...
[M::mem_process_seqs] Processed 67500 reads in 28.214 CPU sec, 28.018 real sec
[M::process] read 67542 sequences (10000120 bp)...
[M::mem_process_seqs] Processed 67524 reads in 28.053 CPU sec, 27.860 real sec
[M::process] read 67540 sequences (10000242 bp)...
[M::mem_process_seqs] Processed 67542 reads in 27.868 CPU sec, 27.707 real sec
[M::process] read 67510 sequences (10000005 bp)...
[M::mem_process_seqs] Processed 67540 reads in 27.595 CPU sec, 27.402 real sec
[M::process] read 67508 sequences (10000219 bp)...
[M::mem_process_seqs] Processed 67510 reads in 28.210 CPU sec, 28.015 real sec
[M::process] read 67498 sequences (10000127 bp)...
[M::mem_process_seqs] Processed 67508 reads in 27.302 CPU sec, 27.105 real sec
[M::process] read 67498 sequences (10000039 bp)...
[M::mem_process_seqs] Processed 67498 reads in 28.092 CPU sec, 27.885 real sec
[M::process] read 67536 sequences (10000300 bp)...
[M::mem_process_seqs] Processed 67498 reads in 27.572 CPU sec, 27.425 real sec
[M::process] read 67624 sequences (10000148 bp)...
[M::mem_process_seqs] Processed 67536 reads in 27.200 CPU sec, 27.039 real sec
[M::process] read 67636 sequences (10000180 bp)...
[M::mem_process_seqs] Processed 67624 reads in 28.350 CPU sec, 28.201 real sec
[M::process] read 67546 sequences (10000197 bp)...
[M::mem_process_seqs] Processed 67636 reads in 29.400 CPU sec, 29.227 real sec
[M::process] read 67642 sequences (10000158 bp)...
[M::mem_process_seqs] Processed 67546 reads in 26.400 CPU sec, 26.249 real sec
[M::process] read 67606 sequences (10000277 bp)...
[M::mem_process_seqs] Processed 67642 reads in 28.300 CPU sec, 28.157 real sec
[M::process] read 67512 sequences (10000216 bp)...
[M::mem_process_seqs] Processed 67606 reads in 28.292 CPU sec, 28.132 real sec
[M::process] read 67592 sequences (10000207 bp)...
[M::mem_process_seqs] Processed 67512 reads in 27.981 CPU sec, 27.831 real sec
[M::process] read 67558 sequences (10000021 bp)...
[M::mem_process_seqs] Processed 67592 reads in 27.359 CPU sec, 27.190 real sec
[M::process] read 67616 sequences (10000082 bp)...
[M::mem_process_seqs] Processed 67558 reads in 27.969 CPU sec, 27.814 real sec
[M::process] read 67636 sequences (10000223 bp)...
[M::mem_process_seqs] Processed 67616 reads in 27.739 CPU sec, 27.576 real sec
[M::process] read 67592 sequences (10000156 bp)...
[M::mem_process_seqs] Processed 67636 reads in 29.901 CPU sec, 29.775 real sec
[M::process] read 67508 sequences (10000231 bp)...
[M::mem_process_seqs] Processed 67592 reads in 27.778 CPU sec, 27.628 real sec
[M::process] read 67592 sequences (10000141 bp)...
[M::mem_process_seqs] Processed 67508 reads in 28.851 CPU sec, 28.716 real sec
[M::process] read 67564 sequences (10000108 bp)...
[M::mem_process_seqs] Processed 67592 reads in 28.832 CPU sec, 28.699 real sec
[M::process] read 67524 sequences (10000058 bp)...
[M::mem_process_seqs] Processed 67564 reads in 30.476 CPU sec, 30.345 real sec
[M::process] read 67622 sequences (10000054 bp)...
[M::mem_process_seqs] Processed 67524 reads in 31.694 CPU sec, 31.562 real sec
[M::process] read 67618 sequences (10000264 bp)...
[M::mem_process_seqs] Processed 67622 reads in 29.606 CPU sec, 29.476 real sec
[M::process] read 67554 sequences (10000208 bp)...
[M::mem_process_seqs] Processed 67618 reads in 29.322 CPU sec, 29.190 real sec
[M::process] read 67602 sequences (10000276 bp)...
[M::mem_process_seqs] Processed 67554 reads in 28.359 CPU sec, 28.186 real sec
[M::process] read 67596 sequences (10000262 bp)...
[M::mem_process_seqs] Processed 67602 reads in 28.024 CPU sec, 27.872 real sec
[M::process] read 67560 sequences (10000023 bp)...
[M::mem_process_seqs] Processed 67596 reads in 28.415 CPU sec, 28.224 real sec
[M::process] read 67610 sequences (10000291 bp)...
[M::mem_process_seqs] Processed 67560 reads in 29.922 CPU sec, 29.719 real sec
[M::process] read 41892 sequences (6196682 bp)...
[M::mem_process_seqs] Processed 67610 reads in 27.161 CPU sec, 26.970 real sec
[M::mem_process_seqs] Processed 41892 reads in 16.575 CPU sec, 16.427 real sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs_Colony9.fa /kb/module/work/tmp/8882bcff-46b0-400a-b71d-8d90b640ef9e.fwd.fastq
[main] Real time: 1392.467 sec; CPU: 1400.552 sec
command: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs_Colony9.fa /kb/module/work/tmp/bc048394-b10b-4957-90cc-63181dfce03e.rev.fastq > /kb/module/work/tmp/alignments2_b6c00fdb-d7b4-4d00-a87d-59a491e8427b.sam
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 67622 sequences (10000048 bp)...
[M::process] read 67756 sequences (10000003 bp)...
[M::mem_process_seqs] Processed 67622 reads in 32.371 CPU sec, 32.300 real sec
[M::process] read 67644 sequences (10000172 bp)...
[M::mem_process_seqs] Processed 67756 reads in 34.548 CPU sec, 34.392 real sec
[M::process] read 67730 sequences (10000212 bp)...
[M::mem_process_seqs] Processed 67644 reads in 33.032 CPU sec, 32.796 real sec
[M::process] read 67744 sequences (10000106 bp)...
[M::mem_process_seqs] Processed 67730 reads in 34.442 CPU sec, 34.238 real sec
[M::process] read 67660 sequences (10000213 bp)...
[M::mem_process_seqs] Processed 67744 reads in 36.260 CPU sec, 36.050 real sec
[M::process] read 67698 sequences (10000093 bp)...
[M::mem_process_seqs] Processed 67660 reads in 34.342 CPU sec, 34.139 real sec
[M::process] read 67864 sequences (10000018 bp)...
[M::mem_process_seqs] Processed 67698 reads in 34.821 CPU sec, 34.598 real sec
[M::process] read 67718 sequences (10000265 bp)...
[M::mem_process_seqs] Processed 67864 reads in 35.350 CPU sec, 35.118 real sec
[M::process] read 67766 sequences (10000180 bp)...
[M::mem_process_seqs] Processed 67718 reads in 34.646 CPU sec, 34.462 real sec
[M::process] read 67812 sequences (10000155 bp)...
[M::mem_process_seqs] Processed 67766 reads in 34.716 CPU sec, 34.503 real sec
[M::process] read 68108 sequences (10000280 bp)...
[M::mem_process_seqs] Processed 67812 reads in 36.597 CPU sec, 36.417 real sec
[M::process] read 67704 sequences (10000152 bp)...
[M::mem_process_seqs] Processed 68108 reads in 41.637 CPU sec, 41.450 real sec
[M::process] read 67744 sequences (10000141 bp)...
[M::mem_process_seqs] Processed 67704 reads in 35.175 CPU sec, 34.973 real sec
[M::process] read 67586 sequences (10000140 bp)...
[M::mem_process_seqs] Processed 67744 reads in 35.479 CPU sec, 35.275 real sec
[M::process] read 67700 sequences (10000079 bp)...
[M::mem_process_seqs] Processed 67586 reads in 33.420 CPU sec, 33.215 real sec
[M::process] read 67662 sequences (10000243 bp)...
[M::mem_process_seqs] Processed 67700 reads in 34.302 CPU sec, 34.095 real sec
[M::process] read 67620 sequences (10000248 bp)...
[M::mem_process_seqs] Processed 67662 reads in 35.994 CPU sec, 35.779 real sec
[M::process] read 67730 sequences (10000140 bp)...
[M::mem_process_seqs] Processed 67620 reads in 32.813 CPU sec, 32.607 real sec
[M::process] read 67656 sequences (10000291 bp)...
[M::mem_process_seqs] Processed 67730 reads in 34.490 CPU sec, 34.295 real sec
[M::process] read 67664 sequences (10000170 bp)...
[M::mem_process_seqs] Processed 67656 reads in 34.463 CPU sec, 34.247 real sec
[M::process] read 67640 sequences (10000078 bp)...
[M::mem_process_seqs] Processed 67664 reads in 34.897 CPU sec, 34.694 real sec
[M::process] read 67630 sequences (10000009 bp)...
[M::mem_process_seqs] Processed 67640 reads in 36.018 CPU sec, 35.810 real sec
[M::process] read 67610 sequences (10000038 bp)...
[M::mem_process_seqs] Processed 67630 reads in 33.362 CPU sec, 33.152 real sec
[M::process] read 67582 sequences (10000121 bp)...
[M::mem_process_seqs] Processed 67610 reads in 34.186 CPU sec, 33.992 real sec
[M::process] read 67550 sequences (10000273 bp)...
[M::mem_process_seqs] Processed 67582 reads in 33.656 CPU sec, 33.432 real sec
[M::process] read 67626 sequences (10000044 bp)...
[M::mem_process_seqs] Processed 67550 reads in 30.840 CPU sec, 30.622 real sec
[M::process] read 67684 sequences (10000266 bp)...
[M::mem_process_seqs] Processed 67626 reads in 31.447 CPU sec, 31.195 real sec
[M::process] read 67560 sequences (10000288 bp)...
[M::mem_process_seqs] Processed 67684 reads in 32.942 CPU sec, 32.738 real sec
[M::process] read 67692 sequences (10000273 bp)...
[M::mem_process_seqs] Processed 67560 reads in 32.237 CPU sec, 32.020 real sec
[M::process] read 67652 sequences (10000011 bp)...
[M::mem_process_seqs] Processed 67692 reads in 34.273 CPU sec, 34.067 real sec
[M::process] read 67528 sequences (10000264 bp)...
[M::mem_process_seqs] Processed 67652 reads in 34.688 CPU sec, 34.468 real sec
[M::process] read 67632 sequences (10000275 bp)...
[M::mem_process_seqs] Processed 67528 reads in 32.692 CPU sec, 32.492 real sec
[M::process] read 67550 sequences (10000134 bp)...
[M::mem_process_seqs] Processed 67632 reads in 32.877 CPU sec, 32.656 real sec
[M::process] read 67600 sequences (10000103 bp)...
[M::mem_process_seqs] Processed 67550 reads in 30.870 CPU sec, 30.663 real sec
[M::process] read 67710 sequences (10000009 bp)...
[M::mem_process_seqs] Processed 67600 reads in 30.265 CPU sec, 30.073 real sec
[M::process] read 67638 sequences (10000028 bp)...
[M::mem_process_seqs] Processed 67710 reads in 32.010 CPU sec, 31.804 real sec
[M::process] read 67486 sequences (10000141 bp)...
[M::mem_process_seqs] Processed 67638 reads in 75.327 CPU sec, 75.104 real sec
[M::process] read 67574 sequences (10000046 bp)...
[M::mem_process_seqs] Processed 67486 reads in 203.960 CPU sec, 203.764 real sec
[M::process] read 67538 sequences (10000213 bp)...
[M::mem_process_seqs] Processed 67574 reads in 60.619 CPU sec, 60.421 real sec
[M::process] read 67532 sequences (10000247 bp)...
[M::mem_process_seqs] Processed 67538 reads in 30.868 CPU sec, 30.699 real sec
[M::process] read 67588 sequences (10000051 bp)...
[M::mem_process_seqs] Processed 67532 reads in 30.639 CPU sec, 30.479 real sec
[M::process] read 67576 sequences (10000022 bp)...
[M::mem_process_seqs] Processed 67588 reads in 30.792 CPU sec, 30.626 real sec
[M::process] read 67564 sequences (10000104 bp)...
[M::mem_process_seqs] Processed 67576 reads in 30.849 CPU sec, 30.686 real sec
[M::process] read 67590 sequences (10000232 bp)...
[M::mem_process_seqs] Processed 67564 reads in 31.082 CPU sec, 30.906 real sec
[M::process] read 67594 sequences (10000141 bp)...
[M::mem_process_seqs] Processed 67590 reads in 31.532 CPU sec, 31.330 real sec
[M::process] read 67586 sequences (10000126 bp)...
[M::mem_process_seqs] Processed 67594 reads in 34.135 CPU sec, 33.999 real sec
[M::process] read 67626 sequences (10000197 bp)...
[M::mem_process_seqs] Processed 67586 reads in 33.510 CPU sec, 33.355 real sec
[M::process] read 36992 sequences (5471788 bp)...
[M::mem_process_seqs] Processed 67626 reads in 32.551 CPU sec, 32.388 real sec
[M::mem_process_seqs] Processed 36992 reads in 16.602 CPU sec, 16.462 real sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa mem -a -k1 -T7 -A1 -B1 -O1 -E1 -L100 /kb/module/work/tmp/flye.contigs_Colony9.fa /kb/module/work/tmp/bc048394-b10b-4957-90cc-63181dfce03e.rev.fastq
[main] Real time: 1859.504 sec; CPU: 1869.066 sec
command: polypolish filter --in1 /kb/module/work/tmp/alignments1_53886e02-0157-4994-b628-b6ac1d748834.sam --in2 /kb/module/work/tmp/alignments2_b6c00fdb-d7b4-4d00-a87d-59a491e8427b.sam --out1 /kb/module/work/tmp/filtered1_baabcf30-e923-4de4-a73b-36e32408f230.sam --out2 /kb/module/work/tmp/filtered2_1546ef4b-37f0-4a5f-8d99-e80436c0679b.sam
[1;4;93mStarting Polypolish filter[0m [2m(2024-06-24 03:23:20)[0m
[2m This runs a pre-processing filter on SAM alignments before they are used to
polish. It looks at each read pair and flags alignments that do not seem to be
part of a concordant pair. This can improve the accuracy Polypolish, especially
near the edges of repeats.[0m
Polypolish version: 0.6.0
Input alignments:
/kb/module/work/tmp/alignments1_53886e02-0157-4994-b628-b6ac1d748834.sam
/kb/module/work/tmp/alignments2_b6c00fdb-d7b4-4d00-a87d-59a491e8427b.sam
Output alignments:
/kb/module/work/tmp/filtered1_baabcf30-e923-4de4-a73b-36e32408f230.sam
/kb/module/work/tmp/filtered2_1546ef4b-37f0-4a5f-8d99-e80436c0679b.sam
Settings:
--orientation auto
--low 0.1
--high 99.9
[1;4;93mLoading alignments[0m [2m(2024-06-24 03:23:20)[0m
/kb/module/work/tmp/alignments1_53886e02-0157-4994-b628-b6ac1d748834.sam: 17,072,297 alignments from 3,284,318 reads
/kb/module/work/tmp/alignments2_b6c00fdb-d7b4-4d00-a87d-59a491e8427b.sam: 22,142,627 alignments from 3,284,318 reads
[1;4;93mFinding insert size thresholds[0m [2m(2024-06-24 03:24:38)[0m
[2m Read pairs with exactly one alignment per read are used to determine the
orientation and insert size thresholds for the read set.[0m
fr: 3,073,924 pairs
rf: 32 pairs
ff: 139 pairs
rr: 147 pairs
Automatically determined correct orientation: fr
Low threshold: 43 (0.1st percentile)
High threshold: 830 (99.9th percentile)
[1;4;93mFiltering SAM files[0m [2m(2024-06-24 03:24:46)[0m
[2m Read alignments that are part of a good pair (correct orientation and
insert size) pass the filter and are written unaltered to the output file. Read
alignments which are not part of good pair are written to the output file with a
"ZP:Z:fail" tag so Polypolish will not use them.[0m
Filtering /kb/module/work/tmp/alignments1_53886e02-0157-4994-b628-b6ac1d748834.sam:
5,387,876 pass
11,684,421 fail
Filtering /kb/module/work/tmp/alignments2_b6c00fdb-d7b4-4d00-a87d-59a491e8427b.sam:
5,405,645 pass
16,736,982 fail
[1;4;93mFinished![0m [2m(2024-06-24 06:23:42)[0m
Alignments before filtering: 39,214,924
Alignments after filtering: 10,793,521
Time to run: 3:00:22.290579
command: polypolish polish --fraction_invalid 0.2 --fraction_valid 0.5 --max_errors 10 --min_depth 5 /kb/module/work/tmp/flye.contigs_Colony9.fa /kb/module/work/tmp/filtered1_baabcf30-e923-4de4-a73b-36e32408f230.sam /kb/module/work/tmp/filtered2_1546ef4b-37f0-4a5f-8d99-e80436c0679b.sam > /kb/module/work/tmp/polypolish_output_790dda4f-e3cc-43bb-a39a-99b6ba7d2a7e.fasta
[1;4;93mStarting Polypolish polish[0m [2m(2024-06-24 06:24:03)[0m
[2m Polypolish is a tool for polishing genome assemblies with short reads.
Unlike other tools in this category, Polypolish uses SAM files where each read
has been aligned to all possible locations (not just a single best location).
This allows it to repair errors in repeat regions that other alignment-based
polishers cannot fix.[0m
Polypolish version: 0.6.0
Input assembly:
/kb/module/work/tmp/flye.contigs_Colony9.fa
Input short-read alignments:
/kb/module/work/tmp/filtered1_baabcf30-e923-4de4-a73b-36e32408f230.sam
/kb/module/work/tmp/filtered2_1546ef4b-37f0-4a5f-8d99-e80436c0679b.sam
Settings:
--fraction_invalid 0.2
--fraction_valid 0.5
--max_errors 10
--min_depth 5
not logging debugging information
[1;4;93mLoading assembly[0m [2m(2024-06-24 06:24:03)[0m
contig_1 (8,125 bp)
contig_2 (3,915,280 bp)
contig_3 (6,404 bp)
[1;4;93mLoading alignments[0m [2m(2024-06-24 06:24:03)[0m
/kb/module/work/tmp/filtered1_baabcf30-e923-4de4-a73b-36e32408f230.sam: 17,072,297 alignments from 3,284,318 reads
/kb/module/work/tmp/filtered2_1546ef4b-37f0-4a5f-8d99-e80436c0679b.sam: 22,142,627 alignments from 3,284,318 reads
Filtering for high-quality end-to-end alignments:
8,566,322 alignments kept
30,648,602 alignments discarded
[1;4;93mPolishing assembly sequences[0m [2m(2024-06-24 06:26:49)[0m
[2m For each position in the assembly, Polypolish determines the read depth
at that position and collects all aligned bases. It then polishes the assembly
by looking for positions where the pileup unambiguously supports a different
sequence than the assembly.[0m
Polishing contig_1 (8,125 bp):
mean read depth: 278.7x
2 bp have a depth of zero (99.9754% coverage)
1 position changed (0.0123% of total positions)
estimated pre-polishing sequence accuracy: 99.9877% (Q39.10)
Polishing contig_2 (3,915,280 bp):
mean read depth: 239.7x
4 bp have a depth of zero (99.9999% coverage)
63 positions changed (0.0016% of total positions)
estimated pre-polishing sequence accuracy: 99.9984% (Q47.93)
Polishing contig_3 (6,404 bp):
mean read depth: 232.9x
2 bp have a depth of zero (99.9688% coverage)
0 positions changed (0.0000% of total positions)
estimated pre-polishing sequence accuracy: 100.0000% (Q∞)
[1;4;93mFinished![0m [2m(2024-06-24 06:26:51)[0m
Polished sequence (to stdout):
contig_1_polypolish (8,124 bp)
contig_2_polypolish (3,915,292 bp)
contig_3_polypolish (6,404 bp)
Time to run: 0:02:47.893471
Generating and saving report.
Polypolish results saved.