The need for better verification of electronic structure codes and for better validation of the methods and techniques they implement is starting to be acutely felt in the electronic-structure community.  

Verification (i.e. of the correctness of computer codes) is becoming increasingly important and delicate due to the growth in complexity of electronic-structure codes. Such growth is a consequence on the one hand of the implementation of more sophisticated methods and theories; on the other hand, of the need to adapt to new computer architectures, especially massively-parallel and GPU-based ones. Of course this increases the likelihood to introduce bugs and numerical instabilities. 

Validation (i.e. of the correctness of theoretical methods) is an ever-present problem of all computer simulations. Only experience and the accumulation of results can provide information on the validity of the theoretical method. Advanced theoretical methods, such as many-body perturbation theory and time-dependent DFT, are affected both by method-specific approximations and by the approximations made in the ground-state calculations they are based upon: the overall effect on the reliability of the final results has to be ascertained.

The field of Verification and Validation (V&V) is very wide and touches many different aspects. For plane-wave based calculations, validation, and even verification, is made problematic by the pseudopotential approximation. The introduction of hybrid and more advanced functionals, for which pseudopotentials are seldom available, make the problem more serious. For molecular dynamics simulations, comparing results is not obvious by construction, since MD trajectories are in practice irreproducible.

V&V would profit a lot from the availability of common formats that allow data exchange between different computer codes. V&V would also profit from the development of high-throughput computing, i.e. a systematic search for all possible materials having some given characteristics or property; V&V is in turn essential to have reliable results from high-throughput computing. Finally, V&V would profit from a better organization and diffusion of available data: the problem of searching and retrieving information is crucial for V&V.

CECAM is currently coordinating and supporting a V&V activity in the field of electronic structure calculations. There has never been a CECAM workshop explicitly dedicated to V&V. We think that such a workshop is an opportunity to present the results achieved so far in the CECAM community, to collect and discuss experience worldwide, and to plan further developments and activities in this field. 

Validation and verification of electronic-structure camputer codes is less well-established than e.g. in quantum chemistry and classical molecular dyamics /Monte Carlo. The main reason for this is likely that the field of electronic-structure computing is more diverse. There is not even a "standard" kind of calculation: many problems require a specially crafted approach and sometimes specialized pieces of code. One can however assume that the backbone of electronic structure computing is the calculation of the ground state in DFT. Even for this case there are competing methods and no "standards", and even within the same theoretical approach, V&V is often made by individual researchers, with duplication of efforts and waste of resources.

The CECAM V&V initiative [1] proposes as a first step to collect and disseminate results of electronic structure calculations obtained with various codes for a set of benchmark problems with the aim of achieving the following goals:

1. Make results of electronic structure calculations widely available

2. Establish and discuss consistency of results obtained with various codes

3. Analyze differences observed between codes and methods and identify the reasons for the differences observed.

4. Provide validation data for specific benchmarks.

In order to achieve goal 1 and at least part of points 2 and 3, the results of reference calculations are stored into a web site [2] running the ESTEST [3] software. ESTEST allows simple storage, search, retrieval of input and output data produced by different code, comparison of results. Several output data file converters into the internal XML representation already exist, while others are under development.

An important V&V aspect specific to calculations based on plane waves is the quality of the pseudopotentials. In order to collect data on this aspect and to achieve a better assessment of available and future pseudopotentials, the CECAM V&V initiative is collecting data from both all-electron (currently FLAPW) and plane-wave calculations. 

In particular, there is an ongoing activity in the CECAM community on the comparisons of pseudopotential and all-electron calculations for various compounds and structures.

The subject of data exchange across different codes, and notably for pseudopotentials and crystal structures, arises naturally. Past efforts to establish standard formats having a wide scope have proven that such a goal is elusive. Less ambitious goals and more restricted scope may yield a more fruitful and durable approach. Efforts in such directions are currently underway at various CECAM nodes.

Concerning more specifically the subject of validation, much work has been done during the years in several groups, especially with respect to performances of the various exchange-correlation functionals. This is especially true for quantum chemistry (for a recente example, see [4]) but results are starting to appear also in the field of solid-state physics (for a recent example, see [5]). While really useful to the community, these results are typicaly available only under the form of a published paper. The quantum chemistry community is however building databases of calculations [6].