Improved Software development methods driving industry competitiveness

Scaling agile methods

Page 3

Short summary of the impact

Agile software development methods are the most popular used today. However, agile methods were initially thought to be limited to small projects with co-located teams, in non-safety-critical domains. In the past 10 years, our research has focused on the first two constraints; agile methods are now often used for large projects with distributed development teams. The use of agile methods in regulated environments represents the final frontier. Recently, we embarked on a study of the use of agile methods in QUMAS, a leading supplier of Quality Management and Compliance Solutions for the life-sciences/pharma domain. We created a new agile method which has yielded positive impacts including quicker customer feedback and a shorter regulatory conformance assessment process. Furthermore, QUMAS were recently acquired by a larger overseas company, and used this research to prove the expertise of the Irish software development function, strengthening the case for retaining high value jobs in Ireland.

Underpinning research

Although the field of software development is barely 50 years old, software development continues to be a problematic endeavour in organisations today. The term ‘Software Crisis’ was coined in 1968 to refer to software taking too long to develop, costing more than budgeted, and not meeting customer needs. More recently, we coined the term Software Crisis 2.0 (Fitzgerald 2011) to refer to the failure of improvements in software development processes to keep pace with the exponential performance improvements in hardware technology. This causes a crisis in light of the amount of new software needed to cope with the avalanche of data that are produced by Internet of Things (IoT) devices.

Software development represents a complex and knowledge-intensive set of activities which require that team members have shared goals and aspirations, as well as a common understanding of the problem being solved. In the past, long development cycles exacerbated miscommunication among developers, and also between customers and developers. As a result, all too often the software eventually delivered no longer met business needs, as these had changed in the interim. Such problems have led to a continual search for development methods that shorten delivery timeframes and improve results.

Agile methods emerged from software development practice; their primary aims were to improve communication between customers and developers, and to shorten development life cycles. Under agile methods, chunks of working software are delivered in shorter timeframes and customer feedback is sought. The Agile Manifesto² identifies four fundamental value propositions:

Individuals and interactions over Processes and tools.
Working software over Comprehensive documentation.
Customer collaboration over Contract negotiation.
Responding to change over Following a plan.

Agile methods have been readily adopted by practitioners. However, it was initially believed that agile methods were suited only to small development projects with co-located teams of developers (given the need for daily face-to-face meetings, for example), and in non-critical domains. Over the past 15 years our research has helped challenge these three assumptions. We now see agile methods adopted in large projects (Ref 3), and with distributed non co-located development teams (Ref 5, Ref 6). The final frontier, the use of agile methods in safety-critical domains, is the one we discuss here.

Regulated environments include industries such as financial services, automotive, aviation, food, medical devices, nuclear, pharmaceutical and railway; increasing the effectiveness of software development processes in these domains can greatly benefit consumers and society. However, highly regulated domains pose particular challenges for agile software development. Agile advocates acknowledge the elements on the right (in the four value propositions above) as having value, but give more weight to the elements on the left. However, in regulated environments the elements on the right represent values of particular importance. Regulators ensure that processes compliant with regulations are documented and followed; thus, documentation looms large in regulated environments. Although agile methods shorten time-to-market, in regulated domains time-to-market is secondary to safety; releasing a defective product could literally mean a multi-million euro loss if products have to be recalled.

QUMAS had selected a widely-used agile approach known as Scrum, which defines a number of roles, ceremonies and artifacts in a software development project. Roles define the various responsibilities in a software development project: for instance, the Scrum Master’s role is to remove any barriers that the development team might face. Ceremonies are recurring activities: for instance, the daily stand-up meeting is a short meeting usually held at the start of day to discuss progress and emerging issues. Artifacts include the software product, which is kept in a runnable state as much as possible, and the product backlog, which is a list of outstanding issues.

The challenge of the current research was to tailor this ‘vanilla’ agile approach to make the resulting development process acceptable to regulators and auditors. We extended Scrum by developing additional roles, ceremonies and artifacts. We named the resulting development method ‘R-Scrum’: Scrum for Regulated Environments. R-Scrum defines a separate role for QA, as regulations require that the QA function be independent from the development team. The ‘dev check’, an additional ceremony, ensures that finished tasks are compliant. An additional artifact is the non-compliance report, which is filed if a QA checkpoint identifies any non-compliance issues.

Table 1: Proportion of academic staff at professorial level in Irish public Universities who are women 1993/2013 ** Smyth, 1996; *** HEA (2006); ****HEA (2013)

References to the research

Fitzgerald B, Stol KJ, O’Sullivan R and O’Brien D (2013) Scaling Agile Methods to Regulated Environments: An Industry Case Study, Proceedings of 35^th International Conference on Software Engineering (ICSE), IEEE Publications, San Francisco, USA, pp. 863-872.
Fitzgerald B (2012) Software Crisis 2.0, IEEE Computer, Vol 45 No 4, April 2012.
Fitzgerald B, Hartnett G and Conboy K (2006) Customising agile methods to software practices at Intel Shannon, European Journal of Information Systems, Vol 15, No 2, pp. 197-210.
Ågerfalk PJ and Fitzgerald B (2008) Outsourcing to an unknown workforce: exploring opensourcing as a global sourcing strategy, MIS Quarterly, Vol 32, No. 3, pp. 385-410.
Ågerfalk P and Fitzgerald B (2006) Flexible and distributed software processes: old petunias in new bowls? Communications of the ACM, Vol 49, No 10, pp. 26-34.
Holmström H, Ó Conchuir E, Ågerfalk PJ and Fitzgerald B (2008) Two-stage offshoring: an investigation of the Irish bridge, MIS Quarterly, Vol 32, No 2, pp. 201- 224.

Research Awards

Fitzgerald, B. (Co-PI), Lero, Research Centre, 2015-2020, funded by Science Foundation Ireland, €3m to Prof. Fitzgerald

Fitzgerald, B. and Stol, K. SCALARE (Scaling Software), ITEA2 project co-funded by Enterprise Ireland, 2013-2016, €320,000

Stol, K. €2,000 Funded by the Royal Irish Academy, 2015

Fitzgerald, B. (PI) B4-STEP, 2002-2006, funded by Science Foundation Ireland, €1 million to Prof. Fitzgerald

Fitzgerald, B. (Co-PI) Lero, CSET, 2005-2010, Funded by Science Foundation Ireland, €2 million to Prof Fitzgerald

Fitzgerald, B. (Co-PI) Lero, CSET2, 2011-2015 Funded by Science Foundation Ireland, €1.6 million to Prof Fitzgerald

Details of the impact

Since the adoption of their agile approach, QUMAS has enjoyed several benefits. The company has been able to pre-sell their new flagship product to a number of clients. This would have been unthinkable during their previous development model; their customers are extremely cautious, as they are also subject to regulation.

QUMAS’s QA department is now better able to keep up with verifying and maintaining compliance – a concept called “continuous compliance”. QA no longer have to do a full product audit and check at the very end of the development process, which could take weeks; instead they can continuously verify that the process is traceable, which makes the process much easier. QUMAS’s VP of QA commented: “QA audits are done at the end of each sprint, which allows for improved visibility, traceability and measurement, so we have no unexpected exceptions to address at final release.” (Source 1).

QUMAS has also benefitted from the publication of our research, which has helped establish its software development team as experts in agile development within regulated environments. The company has been acquired twice within the last 12 months, and recognition of the Irish expertise is of the utmost importance to retaining jobs within Ireland.

In addition, QUMAS customers benefit from the company’s process improvements. Given the agility of the process, customers can now provide early feedback on the product. In the traditional software development model, a provider could take several months to respond to customer input; with R-Scrum QUMAS is able to respond and deliver within 5 weeks.

Regulating authorities and other auditors are also impacted. Product audits used to require the presence of product and quality managers for at least two full days; now audits now take only a few hours. In the past, product and quality managers responsible for process compliance would always attend such audits, but they no longer feel the need to attend as the new process runs so smoothly.

In short, R-Scrum demonstrates that agile methods can be used in regulated domains. Other companies in Ireland and beyond have also shown an interest in the adoption of agile methods, demonstrating the potential impact of this research in improving software development worldwide.

In addition, the topic of applying agile methods in regulated domains is attracting considerable attention within academia; two separate academic workshops have been organised to discuss early research results of this research team and other researchers internationally (Source 2 and 6).

R-Scrum is also coming to the attention of practitioners. A summary of our research was published in the ITEA magazine, which is read by practitioners involved in projects worth hundreds of millions of euros. J&J (pharmaceutical) and Siemens (medical technology) and Kugler-Maag (automotive) have expressed interest in R-Scrum as a result of our dissemination activities (Sources 5, 7 and 8).

Furthermore, this study has helped to establish research collaborations and acquire new funding to study the topic of “scaling methods” in larger contexts where software plays an increasingly important role. As a partner in a European research project, Enterprise Ireland is funding the SCALARE (Scaling Software) project, which has resulted in a €320,000 research grant for the University of Limerick. Based on his expertise as a researcher in this area, Dr Stol was awarded a smaller but highly competitive grant of €2,000 from the Royal Irish Academy.

More recently, Prof Fitzgerald and Dr Stol have established a collaboration with SINTEF, a large research centre in Norway, to pursue further research in this area. SINTEF researchers are focusing specifically on safety-critical systems that must comply with safety standards. Other collaborations include work with Kugler-Maag Cie, a German consultancy company active in the automotive domain.

Regulated environments such as financial services, transportation, food, medical devices and pharmaceuticals touch the lives of people of all ages and in all walks of life. R-Scrum can potentially influence software development in all these life-critical domains, by increasing the speed and accuracy of the development process and making it more responsive to future changes and needs.

Education & Dissemination

The new R-Scrum method has been integrated to the curriculum on the Masters in Software Engineering program at the University of Limerick.

This research was disseminated through a number of top-tier outlets. Firstly, the primary paper reporting the case was published and presented at the International Conference on Software Engineering (ICSE), held in San Francisco, California. The research was also presented at the 2013 annual meeting of the International Society for Pharmaceutical Engineering (ISPE) in Dublin, for a practitioner audience.

Furthermore, the research featured as the topic in an industry workshop which included speakers from Sweden and Germany who are active in the automotive and pharma domains. Participants included companies from as far afield as the USA and Australia.

Prof Brian Fitzgerald and Dr Klaas-Jan Stol have presented the research at several other occasions. Prof Fitzgerald delivered a keynote at the Conference on Researching Agile development of Information Systems (RAISE) in London, UK (Source 3). Dr Stol has presented the research as an invited speaker at the University of Bolzano, Italy, and has published a feature article on this topic in the popular press in the Netherlands (Source 4).

In summary, this research is improving software development methods globally and impacts industry competitiveness and education in the field. This research also addresses the critical issues of safety, flexibility and time-to-market in this competitive industry which ultimately has a significant impact on job retention and competitiveness.

Sources to corroborate the impact

Testimonial Vice President of QA at QUMAS.

ASSD – First international workshop on agile secure software development, organised by ARES, Toulouse, August 2015. http://www.ares-conference.eu/conference/workshops/assd-2015/
Invited keynote address at international conference, RaiseConference – Researching Agile development of Information Systems http://raiseconference.org/programme
Opinion piece in Dutch media AutomatiseringGibs, 5 June 2013, http://www.automatiseringgids.nl/achtergrond/2013/11/het-schijnbare-tekort-van-scrum
Written testimonial from Frances Paulisch, Software Excellence, Siemens AG.
Invited keynote address at XP2015, 16^th International Conference on Agile Software Development, Helsinki, May 2015 http://www.xp2015.org/agile-methods-applied-to-development-and-certification-of-safety-critical-software-ascs/
Written testimonial from James Gunning, Director, Quality CSV, J&J Quality Systems.
Written testimonial from Horst Hientz, Principal, Co-Founder & Board Member, Kugler-Maag Cie.
Case Study Team: Prof Brian Fitzgerald and Dr Klaas-Jan Stol of Lero – the Irish Software Research Centre. Donal O’Brien, Joanne O’Driscoll, Ryan O’Sullivan, QUMAS.