HPC bunches in Exploration Processing.

Uploaded on:
Category: Music / Dance
Linux based registering bunches for huge scale investigative calculation or information examination ... same information set in family-based affiliation testing, Nature Genetics ...
Slide 1

HPC bunches in Research Computing HPC (group) Services offered by Research Computing to the Partners Research Community Dennis Gurgul Jan 25 th 2007 Research Computing

Slide 2

The Services Linux based registering bunches for extensive scale experimental calculation or information examination Offer processing energy to Partners research group and colleagues About 150 records, continually dynamic records 20-30 Average 300-500 occupations entries for every day User bolster: Non-standard applications support : Open source programming , in-house created applications … and so on User direction for framework get to and work accommodation, inconvenience shooting, execution investigating. Application/framework mix with HPC Beyond our group, general HPC support. E.g. Proteomics, DNA sequencing information preparing.. Accomplices Research Computing

Slide 3

Brief History - Intro - how could we have been able to we begin helix.mgh.harvard.edu (GCG) >> newmbcrr.dfci.harvard.edu helix.mgh.harvard.edu email >> Partners.org "How about we assemble a group & see what happens " — Martinson, Jon 2002 Partners Research Computing

Slide 4

Brief History - Intro Research Computing Infrastructure Brent Richter Dennis Gurgul Steve Roylance Jerry Xu Aaron Zschau HPCGG, RPDR, I2B2, *NIX, CLUSTERS Partners Research Computing

Slide 5

Brief History - Intro Agenda: Intro Resources Detailed Technical Description (Hardware/OS/and so on) Apps/Users Success Stories On-going Changes & Future Plans How to Get an Account Partners Research Computing

Slide 6

Computational Resources hpres.mgh.harvard.edu 61 hubs, LSF, 8 TB temp stockpiling, AMD/Opteron rescluster2.mgh.harvard.edu 25 hubs, SGE, 0.5 TB temp stockpiling, Intel 32 bit (most seasoned – Matlab) rescluster.mgh.harvard.edu 15 hubs, LSF, 0.8 TB temp stockpiling, AMD/Opteron Partners Research Computing

Slide 7

System Descriptions: hpres Deployed 4 th Q 04 HP XC4000 bunch mgt. bundle (de-marked RHEL), which incorporates LSF, Two HP Proliant DL585 login hubs w 2 AMD/Opteron, 4 GB RAM, two reflected 72 G SCSI One HP ProLiant DL585 head hub 61 HP ProLiant DL145 process hubs w 2 AMD/Opteron, 4 GB RAM, 160 GB ATA HP MSA1500 SAN w 42 300 GB SCSI as .8 TB LUNS, ADG, 3 hot extras over fiber HBA to login hubs NFS over GB Enet by means of HP ProCurve switches. Accomplices Research Computing

Slide 8

HPRES Architecture GB association PHS system Root switches The real figuring power gave by our gathering Partners Research Computing

Slide 9

System Descriptions: rescluster2 Deployed in 4 th Q 2003 Rocks with included SGE on CentOS Dell 2650 PowerEdge head/login w 2 Intel 32 bit, 4 GB RAM, 0.5 TB RAID5 SCSI 25 Dell 1750 PowerEdge register hubs w 1 Intel 32 bit, 2 GB RAM, 72 G SCSI NFS over Gig Enet by means of Dell PowerConnect Sw. Matlab Biopara CHARMM Partners Research Computing

Slide 10

System Descriptions: rescluster Time Deployed: first quarter, 2006 Computing Resource: 1 Headnode: Dell 2850 PowerEdge, 2 Intel Xeon 3.0 GHz, Memory: 4GB 15 figuring hubs: HP ProLiant DL145 double AMD Opteron 64/32 bit processors, 1.0 GHz, Memory 4GB Storage: 0.8 TB RAID5, SCSI Network: Gigabit Ethernet File System: NFS over Gig Enet through Dell PowerConnect Sw Rocks Clusters with LSF Partners Research Computing

Slide 11

Funding rescluster financed & utilized essentially by Steel Lab at MGH. Plan to grow equipment and open to group rescluster2 financed principally by Biostatistics Lab at MGH. Fundamentally for Matlab hpres supported by association with HP Partners Research Computing

Slide 12

System Level Apps System Extra Math Libraries Basic Compilers Partners Research Computing

Slide 13

Primary Apps/Key Users I2B2, the RPDR Analysis Next Cluster: Parallel File System HP EVA Gateway Partners Research Computing

Slide 14

General Apps/Users Must be shared. Just kept up by System Admin Localized Public Databases (> 500GB ) Sequence Alignment (Accept work from both clump and web) Partners Research Computing

Slide 15

General Apps/Users Protein-related Applications Language, Statistical Applications and Others Partners Research Computing

Slide 16

General Apps/Users Other Applications Databases MySQL (Inside group) Postgres (inside bunch) MS SQLserver (outside bunch, can be come to through PHS system) Oracle (outside, can be come to through PHS system) Partners Research Computing

Slide 17

Achievements and User Feedback Department of Biological Chemistry and Molecular Pharmacology, HMS Derti Adnan, Roth FP, Church GM, Wu CT. "Mammalian ultraconserved components are emphatically exhausted among segmental duplications and duplicate number variants." Nature Genetics . 2006 Oct;38(10):1216-20 Quote: " The Partners group permitted my associates and me to find that ultraconserved successions in the human genome (essentially no transformations in a huge number of years) are altogether exhausted inside copied arrangements … .. Without inquiry, this work would not have been conceivable without the Partners group . " Partners Research Computing

Slide 18

Achievements and User Feedback Children\'s Hospital, Boston Atul Butte and Isaac Kohane " Creation and ramifications of a phenome-genome network." Nature Biotechnolgy . 2006 Jan;24(1):55-62. http://www.nature.com/nbt/diary/v24/n1/pdf/nbt1150.pdf Quote: " You all were wonderful. Pleasant when complex innovation works without issues… . Finally, this paper at last JUST was distributed… I have had only supreme accomplishment in utilizing your bunch ." Partners Research Computing

Slide 19

Achievements and User Feedback Steele Lab, HMS & MGH Selected Submitted Papers: M. M. Dupin, I. Halliday and C. M. Care, L. Munn, "Simple Hybrid technique for the reenactment of red blood cells", submitted November 2006. Chosen Paper In planning M. M. Dupin, ..and so on , "New proficiency arranged, cross breed approach for displaying deformable particles in three dimension", in planning 2007. Stipends Awarded: NIH R01 HL64240 (Munn; research) Quote: " Dennis Gurgul has been astonishing in giving exclusive requirement specialized backing , both on the equipment side and programming side. … .. The outcomes and papers we have (and anticipated) would not have been conceivable without the Partners group help ." Partners Research Computing

Slide 20

Achievements and User Feedback Children\'s Hospital Informatics Program at the Harvard-MIT Division of Health Sciences and Technology Carter SL, Eklund AC, Kohane IS, Harris LN, Szallasi Z. "A mark of chromosomal shakiness derived from quality expression profiles predicts clinical result in different human cancers." Nature Genetics . 2006 Sep;38(9):1043 Quote: " ALL of the information facilitating and computational investigation for this paper was done on hpres!! much obliged folks! " Partners Research Computing

Slide 21

Achievements and User Feedback Department of Biostatistics, Harvard School of Public Health R.Hoffmann T, Lange C. (chosen productions) "P2BAT: a monstrous parallel execution of PBAT for all inclusive affiliation contemplates in bioinformatics", Bioinformatics . 2006 Dec 15;22(24):3103-5. Family-based plans in the time of huge scale quality affiliation contemplates. Laird NM, Lange C. Nature Review Genetics. 2006 May;7(5):385-94. Genomic screening and replication utilizing the same information set as a part of family-based affiliation testing, Nature Genetics 37, 683 - 691 (2005) Quote: " We figured out how to get 3 papers distributed in Nature and one in Science in light of the group calculations. We additionally won a major gift " Partners Research Computing

Slide 22

Achievements and User Feedback MGH, Center for Cancer Research Ben Witnner Quote: " we utilize the group to do bootstrap-based measurable examination… . that would not be conceivable without a group. We likewise utilize ..CellProfiler, a system that breaks down pictures delivered by the high-throughput microscopy. This too would not be conceivable withtout a bunch " Partners Research Computing

Slide 23

Achievements and User Feedback Quote: " It is to a great degree supportive in giving access to a linux framework on which I can run my atomic demonstrating programs,which request quick processor execution.Having this accessible has made numerous runs conceivable which I just couldn\'t do before . " John Penniston , MD, Protein Folding, Harvard Medical School Kenneth Auerbach Sr. Venture Manager, Neurology (BWH) Quote: " Enabled me to do parallel handling of substantial quantities of applicant little particles … and forthcoming restricting destinations in proteins. Without the group these counts would have taken weeks to perform . " Many other empowering remarks… … Partners Research Computing

Slide 24

Future and On-going Changes Expanding Computing Resources- - equipment: Rescluster Add 15 – 20 hubs Change headnode Add stockpiling Make accessible to Partners general utilization Rescluster2 Reconfigure (new equipment, and so on) Make Matlab/biopara/SAS server Use Matlab scheduler Use as vessel stay Partners Research Computing

Slide 25

Future and On-going Changes Expanding Computing Resources- - equipment: Upgrading Existing HPRES Upgrade/move the capacity framework Windows Cluster ? Coming XC3000 Class-c Blades (64 hubs) Dual Core Woodcrest/8 GB Cluster Gateway FS EVA capacity (34 TB crude) One beginning client at first Partners Research Computing

Slide 26

The Coming RESCLU Architecture Hope can beat I/O bottleneck One starting client at first (RPDR investigation) Will be open for general utilization at later date Partners Research Computing

Slide 27

On-going Changes and Future Plan Expanding Computing Resources- - programming: Upgrading Management Tools Commercial Compilers Commercial measurements programming (Matlab tool kit, SAS, SPSS… and so forth . Licenses?) Partners Research Computing

Slide 28

On-going Changes and Future Plans Expanding Services—through association Better comprehend/su

View more...