Pegasus (workflow management)

Pegasus
DevelopersUniversity of Southern California, Information Sciences Institute, University of Wisconsin-Madison
Stable release
5.0 Beta1 / July 27, 2020 (2020-07-27)
Written inJava, Python, C
Operating systemmacOS, Linux
TypeWorkflow management system
LicenseApache License 2.0
Websitepegasus.isi.edu

Pegasus is an open-source workflow management system.[1][2][3] It provides the necessary abstractions for scientists to create scientific workflows[4] and allows for transparent execution of these workflows on a range of computing platforms including high performance computing clusters, clouds, and national cyberinfrastructure.[5][6] In Pegasus, workflows are described abstractly as directed acyclic graphs (DAGs) using a provided API for Jupyter Notebooks, Python, R, or Java.[7] During execution, Pegasus translates the constructed abstract workflow into an executable workflow[8][9] which is executed and managed by HTCondor.[10][11]

Pegasus is being used in a number of different disciplines including astronomy, gravitational-wave physics, bioinformatics, earthquake engineering, and helioseismology.[12] Notably, the LIGO Scientific Collaboration has used it to directly detect a gravitational wave for the first time.[8][13][14]

Area of applications

Application examples:[5][6]

  • Gravitational-Wave Physics
  • Earthquake Science
  • Bioinformatics
  • Workflows for Volcanic Mass Flows
  • Diffusion Image Processing and Analysis
  • Spallation Neutron Source (SNS)

History

The development of Pegasus started in 2001.

See also

References

  1. ↑ E. Deelman, K. Vahi, G. Juve, M. Rynge, S. Callaghan, P. J. Maechling, R. Mayani, W. Chen, R. Ferreira da Silva, M. Livny, and K. Wenger, "Future Generation Computer Systems", Elsevier; 46, pp. 17-35 (2015)
  2. ↑ E.A. Huerta, R. Haas, E. Fajardo, D.S. Katz, S. Anderson, P. Couvares, J. Willis, T. Bouvet, J. Enos, W.T.C. Kramer, H.W. Leong, and D. Wheeler, "BOSS-LDG: A Novel Computational Framework That Brings Together Blue Waters, Open Science Grid, Shifter and the LIGO Data Grid to Accelerate Gravitational Wave Discovery", 2017 IEEE 13th International Conference on e-Science (e-Science); pp. 335-344 (2017)
  3. ↑ B. Riedel, B. Bauermeister, L. Bryant, J. Conrad, P. de Perio, R. W. Gardner, L. Grandi, F. Lombardi, A. Rizzo, G. Sartorelli, M. Selvi, E. Shockley, J. Stephen, S. Thapa, and C. Tunnell "Distributed Data and Job Management for the XENON1T Experiment", PEARC '18: Proceedings of the Practice and Experience on Advanced Research Computing;9, pp. 1-8 (2018)
  4. ↑ G. Amalarethinam, T. Lucia, A. Beena, “Scheduling Framework for Regular Scientific Workflows in Cloud”, International Journal of Applied Engineering Research; 10, no. 82 (2015)
  5. 1 2 E. Deelman, G. Singh, M. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, B. G. Berriman, J. Good, A. Laity, J. C. Jacob, and D. S. Katz, “Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems”, Scientific Programming; 13, pp. 19 (2005)
  6. 1 2 The Scientific Workflow Integrity with Pegasus (SWIP), by Center for Applied Cybersecurity Research; published 16 September 2016; retrieved 1 May 2020
  7. ↑ D. Weitzel, B. Bockelman, D. Brown, P. Couvares, F. Würthwein, and E.F. Hernandez, “Data Access for LIGO on the OSG”, Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact - PEARC17; 24, no. 1-6 (2017)
  8. 1 2 "Testing LIGO's Sensitivity". Research.gov. September 1, 2007. Retrieved April 30, 2020.
  9. ↑ Duncan A. Brown and Ewa Deelman, "Looking for gravitational waves: A computing perspective", at Science Node; published June 8, 2011; retrieved April 30, 2020
  10. ↑ $1M NSF award goes to IU-led data integrity project, by Indiana University; published 16 September 2016; retrieved 1 May 2020
  11. ↑ Brian Mattmiller, "High Throughput Computing helps LIGO confirm Einstein's last unproven theory", at Morgridge Institute for Research; published March 7, 2016; retrieved May 1, 2020
  12. ↑ Sanden Totten, "Caltech Wasn't the Only SoCal School Helping Discover Gravitational Waves", at KPCC; published 11 February 2016; retrieved May 1, 2020
  13. ↑ D.A. Brown, P.R. Brady, A. Dietz, J. Cao, B. Johnson, J. McNabb, “A Case Study on the Use of Workflow Technologies for Scientific Analysis: Gravitational Wave Data Analysis. In: I.J Taylor, E. Deelman, D.B. Gannon, M. Shields (eds) Workflows for e-Science”, Springer, London; 13, pp. 39-59 (2007)
  14. ↑ D. Davis, T. Massinger, A. Lundgren, J.C. Driggers, A.L. Urban, and L. Nuttall, “Improving the sensitivity of Advanced LIGO using noise subtraction”, Classical and Quantum Gravity; 36, no. 5 (2019)
This page is issued from Wikipedia. The text is available under Creative Commons Attribution-Share Alike 4.0 unless otherwise noted. Additional terms may apply for the media files.