Abstract
Current pixel-array detectors produce diffraction images at extreme data rates (of up to 2 TB h(-1)) that make severe demands on computational resources. New multiprocessing frameworks are required to achieve rapid data analysis, as it is important to be able to inspect the data quickly in order to guide the experiment in real time. By utilizing readily available web-serving tools that interact with the Python scripting language, it was possible to implement a high-throughput Bragg-spot analyzer (cctbx.spotfinder) that is presently in use at numerous synchrotron-radiation beamlines. Similarly, Python interoperability enabled the production of a new data-reduction package (cctbx.xfel) for serial femtosecond crystallography experiments at the Linac Coherent Light Source (LCLS). Future data-reduction efforts will need to focus on specialized problems such as the treatment of diffraction spots on interleaved lattices arising from multi-crystal specimens. In these challenging cases, accurate modeling of close-lying Bragg spots could benefit from the high-performance computing capabilities of graphics-processing units.
Highlights
It is widely recognized that modular and reusable code speeds up software development
The collaborative efforts embodied in the CCP4 software suite have been greatly facilitated by the availability of libraries (Winn et al, 2002, 2011) that provide common file formats for describing atomic structures, structure factors and electron-density maps
The ability to customize scripted tools for use in new contexts, and the capacity of various software toolboxes to work together, are key ingredients in developing new programs that support the crystallography experiment as it evolves over time
Summary
It is widely recognized that modular and reusable code speeds up software development. Two additional features of software design have been important: object-oriented programming (Grosse-Kunstleve et al, 2002; Cowtan, 2003), which allows computational problems to be attacked at a high level of abstraction, and scripting, which enables the rapid testing of new ideas Modules that implement these principles, such as those in the Computational Crystallography Toolbox (cctbx; Grosse-Kunstleve et al, 2002; Bourhis et al, 2007), have been used as efficient building blocks for numerous crystallographic applications, including small-molecule (Dolomanov et al, 2009) and macromolecule (Adams et al, 2010) structure solution, indexing (Sauter et al, 2004) and scaling (Evans, 2006), and data-processing workflows (Winter, 2010). Any failure of a test script shows up on a nightly web page, along with a stack trace identifying the point of failure, allowing corrective action.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
