upcoming tutorials All sessions to be held in 138 Cargill register at msi.umn.edu

Transcription

1

2

3 upcoming tutorials Today, December 10 2:30 PM Friday, December 12 1:00 PM Wednesday, December 17 1:00 PM Wednesday, January 7 1:00 PM Tuesday, January 13 10:00 AM All sessions to be held in 138 Cargill register at msi.umn.edu

4

5 the overall assembly strategy is the same 1 pre-processing 2 assembly 3 finishing/polishing Short Reads (Illumina) - graph assembly T Long Reads (PacBio) - HGAP assembly adapter removal quality trimming error correction T A T 1 reads T read length read self-correction de Bruijn or string graph construction 2 overlap-layout-consensus assembly assembled genome contigs read pairs scaffolding read mapping 3 consensus calling with quiver ATCGTT-CCGAGTCTCCCCGCAATCGCAAGCG-TTTCAT CGAGTCT-CGCGCAATCGCAAGCG-TTTC ATCGTT-CCGAGTCTCCCCGCCATC TT-CCGAGACTCCCCGCAATCGCAAGCGATT GTTTCCGAGTCTCCCCGCAATCGCTAGCG-TTGCAT NNNNNN but the data and tools are fundamentally different

6

7

8 .h5 files contain a lot more than just basecalls and quality scores $ h5dump n <smrtcell_data_file>.bax.h5

9 Typical (size-selected) read length distribution, P4-C2 chemistry 2500 data from 1 SMRTcell subreads Mb > subread length subread length

10 quality scores across all bases position in read 25000

11

12 PacBio data is noisy PacBio data cannot (currently) be assembled in its raw state several strategies exist for correcting reads prior to assembly correction without complementary technology used to be difficult until recently, was limited by computational power and SMRT cell throughput Koren & Philippy Curr Op Micro 2014

13

14

15

16 size matters data from 1 SMRTcell, ~4 Mb genome subreads before size selection mean 1,527 bp N50 1,866 bp Mb > subread length subread length

17 size matters 2500 data from 1 SMRTcell, ~4 Mb genome subreads after size selection mean 4,505 bp N50 6,591 bp Mb > subread length subread length

18

19

20

21

22 other options for assembling PacBio reads

23 other options for assembling PacBio reads

24 other options for assembling PacBio reads

25 other options for assembling PacBio reads Assembly-with-PacBio-Long-Reads

26

27 data delivery and organization files typically transferred as gzipped tarballs (.tgz) deposited by Matt Bockol (Mayo) onto MSI to /project/scratch/bockolm2 MSI has plans to streamline data delivery recommend organizing chronologically by run create separate project/sample directories with symbolic links SMRT cell directory names are not informative within untarred parent directory, run $ get_smrtcell_info.sh badalame@login02 [/home/bonddr/shared/pacbio_data/runs/ ] % get_smrtcell_info.sh A01_1 WT_Gsul_BPSS_repeat_0.050_nM A01_2 WT_Gsul_BPSS_repeat_0.050_nM B01_1 WTL_BPSS_repeat_0.050_nM B01_2 WTL_BPSS_repeat_0.050_nM C01_1 CES_3_BPSS_0.050_nM C01_2 CES_3_BPSS_0.050_nM D01_1 JG233_raw_BPSS_0.050_nM D01_2 JG233_raw_BPSS_0.050_nM E01_1 JG233_S_C_BPSS_0.050_nM E01_2 JG233_S_C_BPSS_0.050_nM

28 typical workflow de novo microbial assembly gather, organize, and verify data start isub session within NX Client on MSI import SMRT cells run subread filtering / standard QC run HGAP with length cutoff to provide 100x coverage interpret results / re-run with modifications circularize chromosome(s) and plasmids reorient to begin at replication origin if desired upload as new reference sequence run base modification and motif detection iteratively run quiver until QV > 50 final polish with short reads (if available) using Pilon annotate pull reads into other pipeline(s)

29 import SMRT cells any readable file path can be scanned three options for importing data 1. physically move or copy SMRT cell data to /smrtanalysis/userdata/inputs_dropbox this is dangerous if you ever need to remove smrtanalysis from your home directory 2. create symbolic links to SMRT cell data in inputs_dropbox $ ln s /path/to/smrtcells ~/smrtanalysis/userdata/inputs_dropbox better option 3. scan defined file path(s) for SMRT cell data, e.g. /home/pijoe/shared/pacbio_data/projects/sampleid/smrtcells best option allows for personalized data organization outside SMRT Portal once imported, SMRT cells cannot be (easily) removed from the available list

30

31 QC adapter removal and subread filtering Travers et al. Nucl. Acids Res. (2010)! key terms: SMRT bell library polymerase read subreads full pass subread filtered subreads subreads passing specified length and quality filters adapter

32 CCS reads 1. generate amplicon 5 forward strand 3 3 reverse strand 5 2. ligate adaptors SMRTbell 3. sequence template DNA polymerase 4. data analysis raw long read processed long read single-molecule fragments circular consensus sequence (ccs) 1 o analysis Fichot et al. Microbiome 1:10 (2013)!

33

34

35

36 running HGAP always run with 100x coverage of longest reads key parameters: minimum subread length set to value that provides ~100x coverage based on subread filtering curve minimum polymerase read quality some pipelines default to 0.75, but I always set to 0.8 unless limited by coverage anticipated genome size your best guess based on related species

37

38 pre-assembly HGAP automatically sets length cutoff providing 30x coverage in longest reads blasr maps shorter reads to longer reads pbdagcon calls consensus and spits out corrected long reads these can be useful for other pipelines some long reads get shorter! pre-assembled yield the fraction of total seed bases (i.e. 30x in longest reads) that survived self correction can result from ends being truncated and/or long reads unable to be corrected Read Length Polymerase Read Bases 370,004,973 Length Cutoff 12,944 Seed Bases 114,063,209 Pre-Assembled bases 76,895,237 Pre-Assembled Yield Pre-Assembled Reads 7,719 Pre-Assembled Reads Length 9,961 Pre-Assembled N50 13,168

39 raw uncorrected read (26,076 bp)

40

41 interpreting HGAP results final assembly my assembly has always has a ~4kb plasmid?! (not really) check coverage plots should be even, without large spikes (collapsed repeats) or dips check for plasmids previous versions sent plasmids to separate file why might you have multiple contigs? (for microbes) anticipated genome size was incorrect long, unresolvable/complex repeats low pre-assembled yield some contigs with abnormally low or high coverage might be spurious and can possibly be ignored BLAST any small contigs sum lengths of contigs and re-run HGAP if necessary try HGAP.2 (slower, but more accurate)

42 when an assembly returns a circular genome script for separating contigs for individual circularization: $ extract_contigs.sh See

43 uploading reference sequences unlike SMRT cell data, reference sequences cannot by symlinked! make copies of reference genomes in ~/smrtanalysis/userdata/references_dropbox larger genomes can take several minutes to finish uploading sequence(s) should be in a single.fasta file (including plasmids)

44 base modification and motif detection makes use of real-time kinetic data to evaluate potential based modifications based on long active site residence time interpulse duration pipeline also runs RS_Resequencing (i.e. quiver) by default Pipe-Reference-Guide-v2.2.0

45

46

47 quiver isn t perfect using Pilon to polish remaining indels makes use of short read mapping to identify potential indels, SNPs, ambiguous bases, local misassemblies $ java -Xmx16G jar path/to/pilon-1.8.jar \ --genome path/to/fasta --unpaired path/to/mapping.bam \ --output sample_name --changes --variant --tracks \ --mindepth 100 Pilon removed 128 remaining indels in 3.8 Mbp genome despite Quiver calling > QV 55 consensus

48 quiver isn t perfect using Pilon to polish remaining indels

49 quiver isn t perfect using Pilon to polish remaining indels

50 quiver isn t perfect using Pilon to polish remaining indels final quiver polish 3,820,756 bp % (QV 60) pilon 128 indels detected 3,820,884 bp re-run quiver 3,820,866 bp Sequence Position Variant Type Coverage Confidence Genotype unitig_0 quiver quiver quiver quiver pilon delA DEL haploid unitig_0 quiver quiver quiver quiver pilon delG DEL haploid unitig_0 quiver quiver quiver quiver pilon delC DEL haploid unitig_0 quiver quiver quiver quiver pilon delG DEL haploid unitig_0 quiver quiver quiver quiver pilon delG DEL haploid unitig_0 quiver quiver quiver quiver pilon delG DEL haploid unitig_0 quiver quiver quiver quiver pilon delG DEL haploid unitig_0 quiver quiver quiver quiver pilon delT DEL haploid unitig_0 quiver quiver quiver quiver pilon delC DEL haploid unitig_0 quiver quiver quiver quiver pilon delA DEL haploid unitig_0 quiver quiver quiver quiver pilon delG DEL haploid unitig_0 quiver quiver quiver quiver pilon delG DEL haploid unitig_0 quiver quiver quiver quiver pilon delC DEL haploid unitig_0 quiver quiver quiver quiver pilon delC DEL haploid unitig_0 quiver quiver quiver quiver pilon delG DEL haploid unitig_0 quiver quiver quiver quiver pilon delC DEL haploid unitig_0 quiver quiver quiver quiver pilon delG DEL haploid unitig_0 quiver quiver quiver quiver pilon delC DEL haploid

51 where is my data? all secondary analysis data resides in ~/smrtanalysis/userdata/jobs each job is assigned a six-digit ID corresponding to its directory name graphs and HTML files are in /results filtered subreads, assembly.fasta files and anything else are in /data

52 typical run times on MSI Pipeline Genome size (Mbp) # contigs # SMRT cells coverage wall time RS_Subreads 3.7 n/a 8 580x 34 m RS_HGAP x 2h 45 m RS_HGAP x 13h 45 m RS_Modification_and Motif x 10h 12 m RS_Resequencing x 2h 7m NOTE: all pipelines begin with adapter removal and subread filtering

53 visualizing results in SMRT View start an isub session with extra memory: $ isub m 32gb click on the SMRT View button within the job launches a java web application must be run from an active SMRT Portal session

54 troubleshooting stop SMRT Portal $ /panfs/roc/pacbio/stop_user_portal.sh LAST RESORT $ pbsave.sh $ /panfs/roc/pacbio/delete_user_portal.sh THIS WILL REMOVE ALL ANALYSIS DATA UNLESS SAVED! contact help@msi.umn.edu

55 resources and additional information visit @ctitusbrown data for 5 organisms just released freely available