Assembler for SOLiD data: by Improving memory management of Velvet assembler Sajia Akhter, Robert A Edwards Computational Science Research Center, San Diego State University 2009 The SOLiD™ System is a highly accurate next-generation sequencing technology that supports a wide range of applications. This platform delivers very short length high volume sequences which are significantly different from the traditional sequencing data in both read length and volume. So it is challenging to adapt these data to many applications, including de novo assembly. There are several different assemblers like VELVET, SHARCGS, EULER-SR, which can assemble Solexa data. As both SOLiD data and Solexa data have the same read length but the volume of the data from SOLiD technology is much higher than form Solexa, these assemblers might not perform very well for SOLiD data. The algorithms for the velvet assembler work very fast but memory is a big issue for this assembler. Here we report the improvement of the velvet assembler so that it can assemble Solid data in moderate memory requirement. Velvet: Sequencing Modified Hashing Algorithm 2: • Convert Sequence to base 4 integer, that is ACGT –> (0123)4 = 27 • Advantages • Less memory - 40 length read requires 10 bytes instead of 40 bytes • Computation faster - Makes hashing efficient • Does not use splay tree for hashing • As time complexity for splay tree is nlog(n), and time complexity of any standard sorting algorithm is also nlog(n), we use quick sort for sorting all kmers Memory requirement: Sequencing Velvet assembler Steps of Velvet assembler struct TightString{ char *seq i32 length i32 arraylength } Total size = (8+10) +8 = 26 byte Modified Velvet assembler struct TightString{ char seq[10] i8 length i8 arraylength } Total size = 12 byte For each read save 54% For 337 mi reads Memory requirement for velvet (337 mi * 26) ~ 15 GB Modified Velvet (337 mi * 12) ~ 4 GB Algorithm 1. For all sequences, store sequence id and all possible start positions of kmer 2. Sort the list from step 1 according to kmer value 1. For each comparison of sorting, kmer value is dynamically calculated 3. Use binary search for identifying the repeated sequence. 1. Store the relative position for non repeated sequence Memory requirement: • #sequences * #kmer in each sequence * sizeof(node) • For 337 mi reads, length = 35 and kmer = 31 • (337 mi * 5) * 8 ~ 13.5 GB • if kmer in each sequence > 8 then more memory required struct node{ i32 seqID i8 position i8 relative_position[3] } Total size = 8 byte Time Complexity • Time complexity is same as original Velvet. However, as we calculate kmer value dynamically, this algorithm is 15 times slower than the original velvet. Velvet: Hashing • Each read converted to k length • That means, for 35 read length and kmer = 31, total 5 k length strings are possible for that read. • Finding repeated kmer • Keeps track for previous sequence if current sequence matches • Use splay tree • Height balancing binary search tree • Make faster insertion and searching Memory Requirement: Hashing Velvet Assembler struct splayNode{ i64 kmer i32 seqID i32 position i64 *lc i64 *rc } Total size = 32 byte Velvet: algorithms for de novo short read assembly using de Bruijn graphs. D.R. Zerbino and E. Birney. Genome Research 18:821-829. Modified Hashing Algorithm 1 struct splayNode{ i32 seqID i8 position i32 lc i32 rc } Total size = 13 byte • Memory requirement: • #sequences * #kmer in each sequence * sizeof( splay node) • For 337 mi reads, length = 35 and kmer = 31 • Worst case: if there is no repeats: (337 mi * 5) * 32 ~ 54 GB • For Modified Hashing Algorithm 1: • Save 19 byte per node • Worst case memory ~ 22 GB [(337 mi * 5) * 13] • Time complexity – Increased as kmer value is dynamically • computed • Not work if #node > 4 billion Modified Hashing Algorithm 3: • To improve time complexity, we can store the 1st kmer value and then calculate the next kmer values using the 1st kmer value. For example, length = n and kmer = k, we store kmer value for position 1 to k and then dynamically calculate kmer value for position 2 to k+1, 3 to k+2, and so on. Memory requirement: • #sequences * #kmer in each sequence * sizeof(node) + #sequence * 8 [to store kmer value] • For 337 mi reads, length = 35 and kmer = 31 • (337 mi * 5) * 8 ~ 13.5 GB + 2.7 GB Time Complexity • Time complexity is same as original Velvet. However, as we store 1st kmer value, this algorithm is faster than modification algorithm 2 but still about 6 times slower than the original velvet. • Time performance can be improved by implementing iterative quick sort. Real data Statistics (upto hashing step) Data Size Velvet Modification in Sequencing Modification Algorithm 3 526,996 kmer = 31, read length = 36 Time: 6 sec Memory: 105 MB Time: 6 sec Memory: 98 MB Time : 30 sec Memory: 36 MB 49,423,731 kmer = 31, read length = 36 Time: 12m 29 sec Memory: 8.7 GB Time: 12m 29 sec Memory: 7.8 GB Time: 85 min 10 sec Memory: 3.9 GB 337,075,448 kmer = 31, read length = 35 Could not run in memory limitation of 24GB Could not run in memory limitation of 24GB Time: 694 min 15 sec Memory: 21.5 GB
© Copyright 2024