Minimizing financial cost of scientific workflows under deadline constraints in multi-cloud environments

Tianyu Gao, Yongqiang Wang, Chase Wu, Ruxia Li, Aiqin Hou, Mingrui Xu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In recent years, cloud platforms have been rapidly developed and deployed around the globe and many large-scale scientific workflows have been migrated to multiple clouds for cost-effective data analysis. In such cloud-based workflow applications, financial cost is a major concern in addition to traditional performance requirements such as execution time. In this paper, we formulate a workflow mapping problem to minimize the financial cost of deadline-constrained scientific workflows executed in multi-cloud environments, referred to as MinCost-MC, which is shown to be NP-complete. Within a generic three-layer workflow execution framework, we propose a Workflow Mapping algorithm for Financial Cost Optimization, referred to as WMFCO. This algorithm takes in consideration storage requirements, I /O operations, and data transfers to minimize the financial cost of a given workflow within a specified deadline. Extensive simulation results show that WMFCO exhibits a superior performance over existing algorithms in terms of financial cost in multi-cloud environments.

Original languageEnglish (US)
Title of host publicationProceedings of the ACM Symposium on Applied Computing
PublisherAssociation for Computing Machinery
Pages114-121
Number of pages8
ISBN (Print)9781450359337
DOIs
StatePublished - Jan 1 2019
Event34th Annual ACM Symposium on Applied Computing, SAC 2019 - Limassol, Cyprus
Duration: Apr 8 2019Apr 12 2019

Publication series

NameProceedings of the ACM Symposium on Applied Computing
VolumePart F147772

Conference

Conference34th Annual ACM Symposium on Applied Computing, SAC 2019
CountryCyprus
CityLimassol
Period4/8/194/12/19

Fingerprint

Costs
Data transfer

All Science Journal Classification (ASJC) codes

  • Software

Keywords

  • Cloud computing
  • Cost optimization
  • Workflow mapping

Cite this

Gao, T., Wang, Y., Wu, C., Li, R., Hou, A., & Xu, M. (2019). Minimizing financial cost of scientific workflows under deadline constraints in multi-cloud environments. In Proceedings of the ACM Symposium on Applied Computing (pp. 114-121). (Proceedings of the ACM Symposium on Applied Computing; Vol. Part F147772). Association for Computing Machinery. https://doi.org/10.1145/3297280.3297293
Gao, Tianyu ; Wang, Yongqiang ; Wu, Chase ; Li, Ruxia ; Hou, Aiqin ; Xu, Mingrui. / Minimizing financial cost of scientific workflows under deadline constraints in multi-cloud environments. Proceedings of the ACM Symposium on Applied Computing. Association for Computing Machinery, 2019. pp. 114-121 (Proceedings of the ACM Symposium on Applied Computing).
@inproceedings{7476fb0ddb334bcbbf963e072d3618ad,
title = "Minimizing financial cost of scientific workflows under deadline constraints in multi-cloud environments",
abstract = "In recent years, cloud platforms have been rapidly developed and deployed around the globe and many large-scale scientific workflows have been migrated to multiple clouds for cost-effective data analysis. In such cloud-based workflow applications, financial cost is a major concern in addition to traditional performance requirements such as execution time. In this paper, we formulate a workflow mapping problem to minimize the financial cost of deadline-constrained scientific workflows executed in multi-cloud environments, referred to as MinCost-MC, which is shown to be NP-complete. Within a generic three-layer workflow execution framework, we propose a Workflow Mapping algorithm for Financial Cost Optimization, referred to as WMFCO. This algorithm takes in consideration storage requirements, I /O operations, and data transfers to minimize the financial cost of a given workflow within a specified deadline. Extensive simulation results show that WMFCO exhibits a superior performance over existing algorithms in terms of financial cost in multi-cloud environments.",
keywords = "Cloud computing, Cost optimization, Workflow mapping",
author = "Tianyu Gao and Yongqiang Wang and Chase Wu and Ruxia Li and Aiqin Hou and Mingrui Xu",
year = "2019",
month = "1",
day = "1",
doi = "https://doi.org/10.1145/3297280.3297293",
language = "English (US)",
isbn = "9781450359337",
series = "Proceedings of the ACM Symposium on Applied Computing",
publisher = "Association for Computing Machinery",
pages = "114--121",
booktitle = "Proceedings of the ACM Symposium on Applied Computing",

}

Gao, T, Wang, Y, Wu, C, Li, R, Hou, A & Xu, M 2019, Minimizing financial cost of scientific workflows under deadline constraints in multi-cloud environments. in Proceedings of the ACM Symposium on Applied Computing. Proceedings of the ACM Symposium on Applied Computing, vol. Part F147772, Association for Computing Machinery, pp. 114-121, 34th Annual ACM Symposium on Applied Computing, SAC 2019, Limassol, Cyprus, 4/8/19. https://doi.org/10.1145/3297280.3297293

Minimizing financial cost of scientific workflows under deadline constraints in multi-cloud environments. / Gao, Tianyu; Wang, Yongqiang; Wu, Chase; Li, Ruxia; Hou, Aiqin; Xu, Mingrui.

Proceedings of the ACM Symposium on Applied Computing. Association for Computing Machinery, 2019. p. 114-121 (Proceedings of the ACM Symposium on Applied Computing; Vol. Part F147772).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Minimizing financial cost of scientific workflows under deadline constraints in multi-cloud environments

AU - Gao, Tianyu

AU - Wang, Yongqiang

AU - Wu, Chase

AU - Li, Ruxia

AU - Hou, Aiqin

AU - Xu, Mingrui

PY - 2019/1/1

Y1 - 2019/1/1

N2 - In recent years, cloud platforms have been rapidly developed and deployed around the globe and many large-scale scientific workflows have been migrated to multiple clouds for cost-effective data analysis. In such cloud-based workflow applications, financial cost is a major concern in addition to traditional performance requirements such as execution time. In this paper, we formulate a workflow mapping problem to minimize the financial cost of deadline-constrained scientific workflows executed in multi-cloud environments, referred to as MinCost-MC, which is shown to be NP-complete. Within a generic three-layer workflow execution framework, we propose a Workflow Mapping algorithm for Financial Cost Optimization, referred to as WMFCO. This algorithm takes in consideration storage requirements, I /O operations, and data transfers to minimize the financial cost of a given workflow within a specified deadline. Extensive simulation results show that WMFCO exhibits a superior performance over existing algorithms in terms of financial cost in multi-cloud environments.

AB - In recent years, cloud platforms have been rapidly developed and deployed around the globe and many large-scale scientific workflows have been migrated to multiple clouds for cost-effective data analysis. In such cloud-based workflow applications, financial cost is a major concern in addition to traditional performance requirements such as execution time. In this paper, we formulate a workflow mapping problem to minimize the financial cost of deadline-constrained scientific workflows executed in multi-cloud environments, referred to as MinCost-MC, which is shown to be NP-complete. Within a generic three-layer workflow execution framework, we propose a Workflow Mapping algorithm for Financial Cost Optimization, referred to as WMFCO. This algorithm takes in consideration storage requirements, I /O operations, and data transfers to minimize the financial cost of a given workflow within a specified deadline. Extensive simulation results show that WMFCO exhibits a superior performance over existing algorithms in terms of financial cost in multi-cloud environments.

KW - Cloud computing

KW - Cost optimization

KW - Workflow mapping

UR - http://www.scopus.com/inward/record.url?scp=85065668175&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065668175&partnerID=8YFLogxK

U2 - https://doi.org/10.1145/3297280.3297293

DO - https://doi.org/10.1145/3297280.3297293

M3 - Conference contribution

SN - 9781450359337

T3 - Proceedings of the ACM Symposium on Applied Computing

SP - 114

EP - 121

BT - Proceedings of the ACM Symposium on Applied Computing

PB - Association for Computing Machinery

ER -

Gao T, Wang Y, Wu C, Li R, Hou A, Xu M. Minimizing financial cost of scientific workflows under deadline constraints in multi-cloud environments. In Proceedings of the ACM Symposium on Applied Computing. Association for Computing Machinery. 2019. p. 114-121. (Proceedings of the ACM Symposium on Applied Computing). https://doi.org/10.1145/3297280.3297293