Extended Software Development Workshop: Improving bundle libraries
CECAM-HQ-EPFL, Lausanne, Switzerland
The landscape of Electronic Structure Calculations is evolving rapidly. On one hand, the adoption of common libraries greatly accelerates the availability of new theoretical developments and can have a significant impact on multiple scientific communities at once [LibXC, PETSc]. On the other hand, electronic-structure codes are increasingly used as "force drivers" within broader calculations [Flos,IPi], a use case for which they have initially not been designed. Recent modelling approaches designed to address limitations with system sizes, while preserving consistency with what is currently available, have also become relevant players in the field. For instance, Second-Principles Density Functional Theory [SPDFT], a systematic approximation built on top of the First-Principles DFT approach, provides a similar level of accuracy to the latter and makes it possible to run calculations on more than 100,000 atoms [ScaleUp, Multibinit]. At a broader level, the European Materials Modelling Council (EMMC) has been organizing various events to establish guidelines and roadmaps around the collaboration of Academy and Industry, to meet prominent challenges in the modelling of realistic systems and the economic sustainability of such endeavours, as well as proposing new career paths for people with hybrid scientific/software engineer profiles [EMMC1,EMMC2].
All these trends further push the development of electronic-structure software more and more towards the provision of standards, libraries, APIs, and flexible software components. At a social level, they are also bringing different communities together and reinforce existing collaborations within the communities themselves. Ongoing efforts include an increasing part of coordination of the developments, enhanced integration of libraries into main codes, and consistent distribution of the software modules. They have been made possible in part by the successful adaptation of Lean, Agile and DevOps approaches to the context of scientific software development and the construction of highly-automated infrastructures [EtsfCI, OctopusCI, SiestaPro]. A key enabler in all this process has been the will to get rid of the former silo mentality, both at a scientific level (one research group, one code) as well as a business model level (libre software vs. open-source vs. proprietary), allowing collaborations between communities and making new public-private partnerships possible.
In this context, an essential component of the Electronic Structure Library [esl, els-gitlab] is the ESL Bundle, a consistent set of libraries broadly used within the Electronic Structure Community that can be installed together. This bundle solves various installation issues for end users and enables a smoother integration of the shipped libraries into external codes. In order to maintain the compatibility of the bundle with the main electronic-structure codes on the long run, its development has been accompanied by the creation of the ESL Steering Committee, which includes representatives of both the individual ESL components and the codes using them. As a consequence, the visibility of the ESL expands and the developers are exposed to an increasing amount of feedback, as well as requests from third-party applications. Since many of these developers are contributing to more than one software package, this constitutes an additional source of pressure, on top of research publications and fundraising duties, that is not trivial to manage.
Establishing an infrastructure allowing code developers to efficiently act upon the feedback received and still guarantee the long-term usability of the ESL components, both individually and as a bundle, has become a necessary step. This requires an efficient coordination between various elements:
Set up a common and consistent code development infrastructure / training in terms of compilation, installation, testing and documentation, that can be used seamlessly beyond the electronic structure community, and learn from solutions adopted by other communities.
Agree on metadata and metrics that are relevant for users of ESL components as well as third-party software, not necessarily related to electronic structure in a direct way.
Creating long-lasting synergies between stakeholders of all communities involved and making it attractive for Industry to contribute.
Since 2014, the ESL has been paving the way towards broader and broader collaborations, with a wiki, a data-exchange standard, refactoring code of global interest into integrated modules, and regularly organising workshops, within a wider movement lead by the European eXtreme Data and Computing Initiative [exdci].
Nick Papior (Technical University of Denmark) - Organiser
Micael Oliveira (Max Planck Institute for the Structure and Dynamics of Matter) - Organiser & speaker
Emilio Artacho (Nanogune, Ikerbasque and University of Cambridge) - Organiser
Yann Pouillon (Simune Atomistics) - Organiser
Volker Blum (Duke University) - Organiser