👓 Engineering bioinformatics in seconds, not hours | Ryan Barrett

Read Engineering bioinformatics in seconds, not hours by Ryan BarrettRyan Barrett (snarfed.org)

It was winter 2014. Pharrell had just dropped Happy, the Rosetta probe landed on a comet, President Obama was opening diplomatic relations with Cuba…

‚Ķand¬†here at Color, the bioinformatics team had a problem. Our¬†pipeline‚Ää‚ÄĒ‚Ääthe data processing system that crunches raw DNA data from our lab into the variants we report to patients‚Ää‚ÄĒ‚Ääwas slow. 12 to 24 hours slow.

This wasn‚Äôt a problem in and of itself‚Ää‚ÄĒ‚Ääbioinformatics pipelines routinely run for hours or even days‚Ää‚ÄĒ‚Ääbut it was a royal pain for development. We‚Äôd write new pipeline code, start it running, go home, and return the next morning to find it had crashed halfway through because we‚Äôd¬†missed a semicolon. Argh. Or worse, since we hadn‚Äôt launched yet, our live pipeline would hit similar bugs in production R&D samples, which would delay them until we could debug, test, and deploy the fix. No good.

Leave a Reply

Your email address will not be published. Required fields are marked *