Systematic mapping on the agility evaluation in software development organizations
Keywords:
agile approaches, agility, agility evaluation, software process evaluation, software process improvement, systematic literature mapping
Abstract
Agile software development approaches provide great benefits for organizations; however, their implementation is subject to many challenges. For organizations is important to know the level of agility achieved and the gaps in their transformation process in order to implement improvements in their processes and take advantage of the agile approaches. Keeping this in mind, this article presents a systematic mapping of the literature about the evaluation of agility in software development organizations. As a result, 18 studies were found, which were analyzed and compared taking into account the type of study, the relationship of the evaluation with the principles and agile values, the defined evaluation criteria, the validation methods used, and the proposed tools. In addition, the factors that influence the assessment of agility and the aspects that motivate the interest in this research stream are exposed.
Downloads
Download data is not yet available.
References
[1] L. Williams, “Agile Software Development Methodologies and Practices,” in Advances in Computers, vol. 80, M. V Zelkowitz, Ed. Elsevier, 2010, pp. 1-44. https://doi.org/10.1016/S0065-2458(10)80001-4
[2] J. Highsmith and A. Cockburn, “Agile software development: the business of innovation,” Computer, vol. 34, no. 9, pp. 120-127, 2001.
[3] VersionOne Inc., “The 11th annual State of Agile,” 2017. [Online]. Available: https://explore.versionone.com/state-of-agile. [Accessed: 01-Jun-2018].
[4] A. S. Campanelli and F. S. Parreiras, “Agile methods tailoring - A systematic literature review,” Journal of Systems and Software, vol. 110, pp. 85-100, 2015. https://doi.org/10.1016/j.jss.2015.08.035
[5] K. Schwaber, “SCRUM Development Process,” in Business Object Design and Implementation: OOPSLA ’95 Workshop Proceedings 16 October 1995, Austin, Texas, J. Sutherland, C. Casanave, J. Miller, P. Patel, and G. Hollowell, Eds. London: Springer London, 1997, pp. 117-134. https://doi.org/10.1007/978-1-4471-0947-1_11
[6] C. A. K.Beck, Extreme Programming Explained. Pearson Education, 1999.
[7] A. Cockburn, Crystal Clear: A Human-Powered Methodology for Small Teams. Pearson Education, 2004.
[8] M. Poppendieck and T. Poppendieck, Lean Software Development: An Agile Toolkit (The Agile Software Development Series). 2003.
[9] J. A. Highsmith, Adaptive Software Development: A Collaborative Approach to Managing Complex Systems, vol. 12. Dorset House Publishing Co Inc., 2000.
[10] J. Stapleton, DSDM, Dynamic Systems Development Method: The Method in Practice. Addison-Wesley Professional, 1997.
[11] S. R. Palmer and M. Felsing, A Practical Guide to Feature Driven Development. Prentice Hall, 2002.
[12] S. W. Ambler, “Agile Unified Process.” [Online]. Available: http://www.ambysoft.com/unifiedprocess/agileUP.html. [Accessed: 01-Jun-2018].
[13] “KanBan Fundamentals.” [Online]. Available: http://www.scrumhub.com/kanban-fundamentals/. [Accessed: 01-Jun-2018].
[14] P. Ranganath, “Elevating teams from ‘Doing’ agile to ‘Being’ and ‘Living’ agile,” in Proceedings -2011 Agile Conference, Agile 2011, 2011, pp. 187-194. https://doi.org/10.1109/AGILE.2011.40
[15] O. Ozcan-Top and O. Demirörs, “A reference model for software agility assessment: AgilityMod,” in Communications in Computer and Information Science, 2015, vol. 526, pp. 145-158. https://doi.org/ 10.1007/978-3-319-19860-6_12
[16] E. Rolón, J. Cardoso, F. García, F. Ruiz, and M. Piattini, “Analysis and Validation of Control-Flow Complexity Measures with BPMN Process Models,” in Enterprise, Business-Process and Information Systems Modeling, 2009, pp. 58-70. https://doi.org/10.1007/978-3-642-01862-6_6
[17] T. J. Gandomani, H. Zulzalil, A. A. A. Ghani, A. B. M. Sultan, and M. Z. Nafchi, “Obstacles in moving to agile software development methods; At a Glance,” Journal of Computer Science, vol. 9, no. 5, pp. 620-625, 2013. https://doi.org/10.3844/jcssp.2013.620.625
[18] K. Beck et al., “Manifesto for Agile Software Development,” 2001. [Online]. Available: http://agilemanifesto.org/. [Accessed: 01-Jun-2018].
[19] M. Taromirad and R. Ramsin, “An appraisal of existing evaluation frameworks for agile methodologies,” in Proceedings - Fifteenth IEEE International Conference and Workshops on the Engineering of Computer-Based Systems, ECBS 2008, 2008, pp. 418-427. https://doi.org/10.1109/ECBS.2008.32
[20] O. Ozcan-Top and O. Demirörs, “Assessment of Agile Maturity Models: A Multiple Case Study,” in Software Process Improvement and Capability Determination, 2013, pp. 130-141. https://doi.org/10.1007/978-3-642-38833-0_12
[21] M. Z. Nafchi, H. Zulzalil, and T. J. Gandomani, “On the current agile assessment methods and approaches,” in 2014 8th Malaysian Software Engineering Conference, MySEC 2014, 2014, pp. 251-254. https://doi.org/10.1109/MySec.2014.6986023
[22] M. Muñoz, J. Mejía, B. Corona, J. A. Calvo-Manzano, T. San Feliu, and J. Miramontes, “Analysis of Tools for Assessing the Implementation and Use of Agile Methodologies in SMEs,” in Software Process Improvement and Capability Determination, 2016, pp. 123-134. https://doi.org/10.1007/978-3-319-38980-6_10
[23] O. E. Adali, Ö. Özcan-Top, and O. Demirörs, “Evaluation of Agility Assessment Tools: A Multiple Case Study,” in Software Process Improvement and Capability Determination, 2016, pp. 135-149. https://doi.org/10.1007/978-3-319-38980-6_11
[24] K. Petersen, S. Vakkalanka, and L. Kuzniarz, “Guidelines for conducting systematic mapping studies in software engineering: An update,” Information and Software Technology, vol. 64, pp. 1-8, 2015. https://doi.org/10.1016/j.infsof.2015.03.007
[25] F. McCaffery, P. S. Taylor, and G. Coleman, “Adept: A unified assessment method for small software companies,” IEEE Software, vol. 24, no. 1, pp. 24-31, 2007. https://doi.org/10.1109/MS.2007.3
[26] A. Sidky, J. Arthur, and S. Bohner, “A disciplined approach to adopting agile practices: The agile adoption framework,” Innovations in Systems and Software Engineering, vol. 3, no. 3, pp. 203-216, 2007. https://doi.org/10.1007/s11334-007-0026-z
[27] B. Shen and D. Ju, “On the Measurement of Agility in Software Process,” in Software Process Dynamics and Agility, 2007, pp. 25-36. https://doi.org/10.1007/978-3-540-72426-1_3
[28] A. Qumer and B. Henderson-Sellers, “An evaluation of the degree of agility in six agile methods and its applicability for method engineering,” Information and Sof tware Technology, vol. 50, no. 4, pp. 280-295, 2007. https://doi.org/10.1016/j.infsof.2007.02.002
[29] M. Taromirad and R. Ramsin, “CEFAM: Comprehensive evaluation framework for agile methodologies,” in 32nd Annual IEEE Software Engineering Workshop, SEW-32 2008, 2009, pp. 195-04. https://doi.org/10.1109/SEW.2008.19
[30] F. Mafakheri, F. Nasiri, and M. Mousavi, “Project agility assessment: an integrated decision analysis approach,” Production Planning & Control, vol. 19, no. 6, pp. 567-576, 2008. https://doi.org/10.1080/09537280802360884
[31] M. Ikoma, M. Ooshima, T. Tanida, M. Oba, and S. Sakai, “Using a validation model to measure the agility of software development in a large software development organization,” in 2009 31st International Conference on Software Engineering - Companion Volume, 2009, pp. 91-100. https://doi.org/10.1109/icse-companion.2009.5070967
[32] D. M. Shawky and A. F. Ali, “A practical measure for the agility of software development processes,” in 2010 2nd International Conference on Computer Technology and Development, 2010, pp. 230-234. https://doi.org/10.1109/icctd.2010.5645881
[33] TongShi, JianbinChen, and J. Zhu, “Study on assessment framework of software process in agile,” in 2010 2nd International Conference on Industrial and Information Systems, 2010, vol. 1, pp. 498-501. https://doi.org/10.1109/indusis.2010.5565802
[34] S. Soundararajan, J. D. Arthur, and O. Balci, “A Methodology for Assessing Agile Software Development Methods,” in 2012 Agile Conference, 2012, pp. 51-54. https://doi.org/10.1109/Agile.2012.24
[35] V. Escobar-Sarmiento and M. Linares-Vásquez, “A model for measuring agility in small and medium software development enterprises,” in 38th Latin America Conference on Informatics, CLEI 2012 - Conference Proceedings, 2012, pp. 1-10. https://doi.org/10.1109/clei.2012.6427226
[36] K. M. Poonacha and S. Bhattacharya, “Towards a framework for assessing agility,” in Proceedings of the Annual Hawaii International Conference on System Sciences, 2012, pp. 5329- 338. https://doi.org/10.1109/hicss.2012.599
[37] R. Wendler, “Development of the organizational agility maturity model,” in 2014 Federated Conference on Computer Science and Information Systems, 2014, pp. 119-1206. https://doi.org/10.15439/2014F79
[38] O. E. Adali, O. O. Top, and O. Demirors, “Assessment of agility in software organizations with a web-based agility assessment tool,” in Proceedings - 43rd Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2017, 2017, pp. 88-95. https://doi.org/10.1109/SEAA.2017.61
[39] M. Florêncio, F. Sambinelli, and M. A. Francisco Borges, “ASA: Agile Software Development Self-assessment Method,” in Agile Methods, 2018, pp. 21-30. https://doi.org/10.1007/978-3-319-73673-0_2
[40] H. Kniberg, “Squad Health Check model – visualizing what to improve,” 2014. [Online]. Available: https://labs.spotify.com/2014/09/16/squad-health-checkmodel/. [Accessed: 07-May-2019].
[41] “Scrum Level,” 2018. [Online]. Available: https:// scrumlevel.com. [Accessed: 07-May-2019].
[2] J. Highsmith and A. Cockburn, “Agile software development: the business of innovation,” Computer, vol. 34, no. 9, pp. 120-127, 2001.
[3] VersionOne Inc., “The 11th annual State of Agile,” 2017. [Online]. Available: https://explore.versionone.com/state-of-agile. [Accessed: 01-Jun-2018].
[4] A. S. Campanelli and F. S. Parreiras, “Agile methods tailoring - A systematic literature review,” Journal of Systems and Software, vol. 110, pp. 85-100, 2015. https://doi.org/10.1016/j.jss.2015.08.035
[5] K. Schwaber, “SCRUM Development Process,” in Business Object Design and Implementation: OOPSLA ’95 Workshop Proceedings 16 October 1995, Austin, Texas, J. Sutherland, C. Casanave, J. Miller, P. Patel, and G. Hollowell, Eds. London: Springer London, 1997, pp. 117-134. https://doi.org/10.1007/978-1-4471-0947-1_11
[6] C. A. K.Beck, Extreme Programming Explained. Pearson Education, 1999.
[7] A. Cockburn, Crystal Clear: A Human-Powered Methodology for Small Teams. Pearson Education, 2004.
[8] M. Poppendieck and T. Poppendieck, Lean Software Development: An Agile Toolkit (The Agile Software Development Series). 2003.
[9] J. A. Highsmith, Adaptive Software Development: A Collaborative Approach to Managing Complex Systems, vol. 12. Dorset House Publishing Co Inc., 2000.
[10] J. Stapleton, DSDM, Dynamic Systems Development Method: The Method in Practice. Addison-Wesley Professional, 1997.
[11] S. R. Palmer and M. Felsing, A Practical Guide to Feature Driven Development. Prentice Hall, 2002.
[12] S. W. Ambler, “Agile Unified Process.” [Online]. Available: http://www.ambysoft.com/unifiedprocess/agileUP.html. [Accessed: 01-Jun-2018].
[13] “KanBan Fundamentals.” [Online]. Available: http://www.scrumhub.com/kanban-fundamentals/. [Accessed: 01-Jun-2018].
[14] P. Ranganath, “Elevating teams from ‘Doing’ agile to ‘Being’ and ‘Living’ agile,” in Proceedings -2011 Agile Conference, Agile 2011, 2011, pp. 187-194. https://doi.org/10.1109/AGILE.2011.40
[15] O. Ozcan-Top and O. Demirörs, “A reference model for software agility assessment: AgilityMod,” in Communications in Computer and Information Science, 2015, vol. 526, pp. 145-158. https://doi.org/ 10.1007/978-3-319-19860-6_12
[16] E. Rolón, J. Cardoso, F. García, F. Ruiz, and M. Piattini, “Analysis and Validation of Control-Flow Complexity Measures with BPMN Process Models,” in Enterprise, Business-Process and Information Systems Modeling, 2009, pp. 58-70. https://doi.org/10.1007/978-3-642-01862-6_6
[17] T. J. Gandomani, H. Zulzalil, A. A. A. Ghani, A. B. M. Sultan, and M. Z. Nafchi, “Obstacles in moving to agile software development methods; At a Glance,” Journal of Computer Science, vol. 9, no. 5, pp. 620-625, 2013. https://doi.org/10.3844/jcssp.2013.620.625
[18] K. Beck et al., “Manifesto for Agile Software Development,” 2001. [Online]. Available: http://agilemanifesto.org/. [Accessed: 01-Jun-2018].
[19] M. Taromirad and R. Ramsin, “An appraisal of existing evaluation frameworks for agile methodologies,” in Proceedings - Fifteenth IEEE International Conference and Workshops on the Engineering of Computer-Based Systems, ECBS 2008, 2008, pp. 418-427. https://doi.org/10.1109/ECBS.2008.32
[20] O. Ozcan-Top and O. Demirörs, “Assessment of Agile Maturity Models: A Multiple Case Study,” in Software Process Improvement and Capability Determination, 2013, pp. 130-141. https://doi.org/10.1007/978-3-642-38833-0_12
[21] M. Z. Nafchi, H. Zulzalil, and T. J. Gandomani, “On the current agile assessment methods and approaches,” in 2014 8th Malaysian Software Engineering Conference, MySEC 2014, 2014, pp. 251-254. https://doi.org/10.1109/MySec.2014.6986023
[22] M. Muñoz, J. Mejía, B. Corona, J. A. Calvo-Manzano, T. San Feliu, and J. Miramontes, “Analysis of Tools for Assessing the Implementation and Use of Agile Methodologies in SMEs,” in Software Process Improvement and Capability Determination, 2016, pp. 123-134. https://doi.org/10.1007/978-3-319-38980-6_10
[23] O. E. Adali, Ö. Özcan-Top, and O. Demirörs, “Evaluation of Agility Assessment Tools: A Multiple Case Study,” in Software Process Improvement and Capability Determination, 2016, pp. 135-149. https://doi.org/10.1007/978-3-319-38980-6_11
[24] K. Petersen, S. Vakkalanka, and L. Kuzniarz, “Guidelines for conducting systematic mapping studies in software engineering: An update,” Information and Software Technology, vol. 64, pp. 1-8, 2015. https://doi.org/10.1016/j.infsof.2015.03.007
[25] F. McCaffery, P. S. Taylor, and G. Coleman, “Adept: A unified assessment method for small software companies,” IEEE Software, vol. 24, no. 1, pp. 24-31, 2007. https://doi.org/10.1109/MS.2007.3
[26] A. Sidky, J. Arthur, and S. Bohner, “A disciplined approach to adopting agile practices: The agile adoption framework,” Innovations in Systems and Software Engineering, vol. 3, no. 3, pp. 203-216, 2007. https://doi.org/10.1007/s11334-007-0026-z
[27] B. Shen and D. Ju, “On the Measurement of Agility in Software Process,” in Software Process Dynamics and Agility, 2007, pp. 25-36. https://doi.org/10.1007/978-3-540-72426-1_3
[28] A. Qumer and B. Henderson-Sellers, “An evaluation of the degree of agility in six agile methods and its applicability for method engineering,” Information and Sof tware Technology, vol. 50, no. 4, pp. 280-295, 2007. https://doi.org/10.1016/j.infsof.2007.02.002
[29] M. Taromirad and R. Ramsin, “CEFAM: Comprehensive evaluation framework for agile methodologies,” in 32nd Annual IEEE Software Engineering Workshop, SEW-32 2008, 2009, pp. 195-04. https://doi.org/10.1109/SEW.2008.19
[30] F. Mafakheri, F. Nasiri, and M. Mousavi, “Project agility assessment: an integrated decision analysis approach,” Production Planning & Control, vol. 19, no. 6, pp. 567-576, 2008. https://doi.org/10.1080/09537280802360884
[31] M. Ikoma, M. Ooshima, T. Tanida, M. Oba, and S. Sakai, “Using a validation model to measure the agility of software development in a large software development organization,” in 2009 31st International Conference on Software Engineering - Companion Volume, 2009, pp. 91-100. https://doi.org/10.1109/icse-companion.2009.5070967
[32] D. M. Shawky and A. F. Ali, “A practical measure for the agility of software development processes,” in 2010 2nd International Conference on Computer Technology and Development, 2010, pp. 230-234. https://doi.org/10.1109/icctd.2010.5645881
[33] TongShi, JianbinChen, and J. Zhu, “Study on assessment framework of software process in agile,” in 2010 2nd International Conference on Industrial and Information Systems, 2010, vol. 1, pp. 498-501. https://doi.org/10.1109/indusis.2010.5565802
[34] S. Soundararajan, J. D. Arthur, and O. Balci, “A Methodology for Assessing Agile Software Development Methods,” in 2012 Agile Conference, 2012, pp. 51-54. https://doi.org/10.1109/Agile.2012.24
[35] V. Escobar-Sarmiento and M. Linares-Vásquez, “A model for measuring agility in small and medium software development enterprises,” in 38th Latin America Conference on Informatics, CLEI 2012 - Conference Proceedings, 2012, pp. 1-10. https://doi.org/10.1109/clei.2012.6427226
[36] K. M. Poonacha and S. Bhattacharya, “Towards a framework for assessing agility,” in Proceedings of the Annual Hawaii International Conference on System Sciences, 2012, pp. 5329- 338. https://doi.org/10.1109/hicss.2012.599
[37] R. Wendler, “Development of the organizational agility maturity model,” in 2014 Federated Conference on Computer Science and Information Systems, 2014, pp. 119-1206. https://doi.org/10.15439/2014F79
[38] O. E. Adali, O. O. Top, and O. Demirors, “Assessment of agility in software organizations with a web-based agility assessment tool,” in Proceedings - 43rd Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2017, 2017, pp. 88-95. https://doi.org/10.1109/SEAA.2017.61
[39] M. Florêncio, F. Sambinelli, and M. A. Francisco Borges, “ASA: Agile Software Development Self-assessment Method,” in Agile Methods, 2018, pp. 21-30. https://doi.org/10.1007/978-3-319-73673-0_2
[40] H. Kniberg, “Squad Health Check model – visualizing what to improve,” 2014. [Online]. Available: https://labs.spotify.com/2014/09/16/squad-health-checkmodel/. [Accessed: 07-May-2019].
[41] “Scrum Level,” 2018. [Online]. Available: https:// scrumlevel.com. [Accessed: 07-May-2019].
Published
2019-06-19
How to Cite
Ortega-Ordóñez, W., Pardo-Calvache, C., & Pino-Correa, F. (2019). Systematic mapping on the agility evaluation in software development organizations. ITECKNE, 16(1), 64-76. https://doi.org/https://doi.org/10.15332/.v16i1.2162
Issue
Section
Research and Innovation Articles
