Project organization

[k8hertweck]

Data in /shared/biodata/example_data

What are recommendations for:

• primary data storage? with SR? copy to own folders?
• large intermediate data files (SAM, BAM, etc)?

What type of directory structure to model for pre-processed files?

[k8hertweck]

Primary (raw) data storage: fastq

• stays in original location (path) as shared by SR
• read-only
• unaligned -> project name (user/pool) -> gzipped sample, per read and sample
• create variable to reference?
• group permissions determined by lab

Reference data: shared biodata

• genomic data
• indexes

Metadata:

Output: files sent to scratch, instructions to copy to home (default on rhino) or fast?

• one directory per sample, including alignment file and logs

• raw_data_qc

• trimmed_data

• trimmed_data_qc

• mapped_reads (BAM): retain alignment, renaming file to view in IGV (and logs)

• mapped_qc, across all samples, plots and samtools output

• counts files: individual (richly annotated), and combined (fewer annotations)

Suggested structure from HBC:

rnaseq
├── logs
├── meta
├── raw_data
├── results
└── scripts

Structure used by Data Carpentry, modified for RNAseq:

• dc_workshop
  • data
    • untrimmed_fastq
    • trimmed_fastq
  • results
    • sam
    • bam
    • counts