No Access

Improving Resilience of Software Systems: A Case Study in 3D-Online Game System

School of Software Engineering, Beijing Jiaotong University, Beijing, P. R. China

Corresponding author. In addition, Wei Lu and Weidong Wang contributed equally to this work and should be considered as co-first authors.

Search for more papers by this author

Weidong Wang

School of Software Engineering, Beijing Jiaotong University, Beijing, P. R. China

E-mail Address: wangweidong@bjtu.edu.cn

Corresponding author. In addition, Wei Lu and Weidong Wang contributed equally to this work and should be considered as co-first authors.

Search for more papers by this author

Ergude Bao

School of Software Engineering, Beijing Jiaotong University, Beijing, P. R. China

E-mail Address: baoe@bjtu.edu.cn

Search for more papers by this author

Weiwei Xing

School of Software Engineering, Beijing Jiaotong University, Beijing, P. R. China

E-mail Address: wwxing@bjtu.edu.cn

Search for more papers by this author

, and

Kai Zhu

Beijing Chukong Aipu Technology Co., Ltd., Beijing, P. R. China

E-mail Address: zhukaixy@163.com

Search for more papers by this author

https://doi.org/10.1142/S0218194017500012Cited by:1 (Source: Crossref)

Abstract

Resilience is the property that enables a system to continue operating properly when one or more faults occur. Nowadays, as software systems become more and more complex, their hardware execution platforms also become more heterogenous with larger scale. Software systems may fail due to some faults such as node breakdown, communication failure, or data processing failure. In this paper, we propose a ring-based resilience mechanism, which implements fault detection and recovery. (1) To solve the problem that the central server may have high burden of network traffic, we design a ring-based heartbeat algorithm for crash fault detection. (2) We also design a light-weight recovery mechanism to recover from crash faults as compared with the current system-specific mechanisms. To evaluate our mechanism, we use a 3D-online game system as a case study. By injecting faults, we test the effectiveness and overhead of the proposed mechanism. Compared with other mechanisms, the experimental results show that our mechanism can support resilience very well and is better at dealing with the crash fault caused by high cluster workload with acceptable overhead.

Keywords:

References

1. H. Okamura, T. Dohi and S. Osaki , Software reliability growth models with normal failure time distributions, Reliab. Eng. Syst. Saf. 116 (8) (2013) 135–141. Crossref, Web of Science, Google Scholar
2. L. Rashid, K. Pattabiraman and S. Gopalakrishnan , Characterizing the impact of intermittent hardware faults on programs, IEEE Trans. Reliab. 64 (1) (2015) 297–310. Crossref, Web of Science, Google Scholar
3. A. Höller, G. Macher, T. Rauter, J. Iber and C. Kreiner , A virtual fault injection framework for reliability-aware software development, in IEEE Int. Conf. Dependable Systems and Networks Workshops, 2015, pp. 69–74. Google Scholar
4. E. Erturk and E. A. Sezer , A comparison of some soft computing methods for software fault prediction, Expert Syst. Appl. 42 (4) (2015) 1872–1879. Crossref, Web of Science, Google Scholar
5. B. J. Ma, H. P. Zhang, G. Q. Chen, Y. P. Zhao and B. Baesens , Investigating associative classification for software fault prediction: An experimental perspective, Int. J. Softw. Eng. Knowl. Eng. 24 (1) (2014) 61–90. Link, Web of Science, Google Scholar
6. W. D. Wang, L. Q. Wang and W. Lu , A resilient framework for fault handling in web service oriented systems, in Proc. 22nd Int. Conf. Web Services, 2015, pp. 61–68. Google Scholar
7. C. Engelmann and T. Naughton , Toward a performance/resilience tool for hardware/software co-design of high-performance computing systems, in Proc. 42nd Int. Conf. Parallel Processing, 2013, pp. 960–969. Google Scholar
8. C. W. Axelrod , Investing in software resiliency, J. Def. Softw. Eng. 22 (6) (2009) 20–25. Google Scholar
9. H. Hoffmann, J. Eastep, M. D. Santambrogio, J. E. Miller and A. Agarwal , Application heartbeats for software performance and health, in Proc. 15th ACM SIGPLAN Symp. Principles and Practice of Parallel Programming, 2010, pp. 347–348. Google Scholar
10. C. Zhou, R. Kumar and S. Jiang , Analysis of runtime data-log for software fault localization, in Proc. American Control Conf., 2011, pp. 5127–5132. Google Scholar
11. M. Cinque, D. Cotroneo, R. Natella and A. Pecchia , Assessing and improving the effectiveness of logs for the analysis of software faults, in IEEE/IFIP Int. Conf. Dependable Systems and Networks, 2010, pp. 457–466. Google Scholar
12. N. Delgado, A. Q. Gates and S. Roach , A taxonomy and catalog of runtime software-fault monitoring tools, IEEE Trans. Softw. Eng. 30 (12) (2004) 859–872. Crossref, Web of Science, Google Scholar
13. Z. Hou, Y. Huang, S. Zheng, X. Dong and B. Wang , Design and implementation of heartbeat in multi-machine environment, in 17th Int. Conf. Advanced Information Networking and Applications, 2003, pp. 583–586. Google Scholar
14. M. Kumar, S. Neubert, S. Behrendt, A. Rieger, M. Weippert, N. Stoll, K. Thurow and R. Stoll , Stress monitoring based on stochastic fuzzy analysis of heartbeat intervals, IEEE Trans. Fuzzy Syst. 20 (4) (2012) 746–759. Crossref, Web of Science, Google Scholar
15. J. H. Janssen, J. N. Bailenson, W. A. IJsselsteijn and J. H. D. M. Westerink , Intimate heartbeats: Opportunities for affective communication technology, IEEE Trans. Affect. Comput. 1 (2) (2010) 72–80. Crossref, Web of Science, Google Scholar
16. W. D. Wang, C. H. Liao, L. Q. Wang, D. J. Quinlan and W. Lu, HBTM: A heartbeat-based behavior detection mechanism for POSIX threads and OpenMP applications, arXiv:1512.00665, 2015, pp. 1–7. Google Scholar
17. J. Duell, P. Hargrove and E. Roman, The design and implementation of Berkeley lab’s linux checkpoint/restart, Technical report LBNL-54941, Lawrence Berkeley National Laboratory, 2005. Google Scholar
18. R. Gioiosa, J. C. Sancho, S. Jiang and F. Petrini , Transparent incremental checkpointing at kernel level: A foundation for fault tolerance for parallel computers, in Proc. ACM IEEE Conf. Supercomputing, 2005, pp. 9–22. Google Scholar
19. P. H. Hargrove and J. C. Duell , Berkeley lab checkpoint/restart (BLCR) for linux clusters, J. Phys., Conf. Ser. 46 (2006) 494–499. Google Scholar
20. M. M. Ali, J. Southern, P. Strazdins and B. Harding , Application level fault recovery: Using fault-tolerant open MPI in a PDE solver, in IEEE Int. Parallel & Distributed Processing Symposium Workshops, 2014, pp. 1169–1178. Google Scholar
21. N. Losada, G. Rodríguez, M. Martín and P. González , Extending an application-level checkpointing tool to provide fault tolerance support to OpenMP applications, J. Univers. Comput. Sci. 20 (12) (2014) 1352–1372. Google Scholar
22. W. D. Yu , A software fault prevention approach in coding and root cause analysis, Bell Labs Tech. J. 3 (2) (2002) 3–21. Crossref, Web of Science, Google Scholar
23. P. H. S. Brito, R. Lemos, E. Martins, R. Moraes and C. M. F. Rubira , Architectural-based validation of fault-tolerant software, in 4th Latin-American Symp. Dependable Computing, 2009, pp. 103–111. Google Scholar
24. J. Guthoff and V. Sieh , Combining software-implemented and simulation-based fault injection into a single fault injection method, in Proc. 25th Int. Symp. Fault-Tolerant Computing, 1995, pp. 196–206. Google Scholar
25. H. Huang, L. Q. Wang, B. C. Tak, L. Wang and C. Q. Tang , CAP3: A cloud auto-provisioning framework for parallel processing using on-demand and spot instances, in IEEE 6th Int. Conf. Cloud Computing, 2013, pp. 228–235. Google Scholar
26. H. Y. Ma, L. Q. Wang and K. Krishnamoorthy , Detecting thread-safety violations in hybrid OpenMP/MPI programs, in IEEE Int. Conf. Cluster Computing, 2015, pp. 460–463. Google Scholar