Speaker
Dr
Alexander Kryukov
(SINP MSU)
Description
Currently, a number of experimental facilities in the field of particle astrophysics of the mega-siences class are being built and are already operating in the world. An important feature of this class of projects is the huge flow of data produced, the participation of many organizations and, as a result, the distributed nature of data processing and analysis.
To meet similar requests in high energy physics, a WLCG grid was deployed as part of the LHC project. This solution, on the one hand, showed high efficiency, and, on the other hand, it turned out to be a rather heavy solution that requires high administrative costs, highly qualified staff and a very homogeneous environment on which applications operate.
The paper considers the architecture of a distributed storage for astrophysical experiments — AstroDS using the example of the KASCADE and TAIGA experiments. The main ideas of the proposed approach are as follows:
• unification of access to local storage data without changing their structure based on corresponding adapter modules;
• use of local data access policies;
• data transfer only at the moment of actual access to them;
• search and aggregation of the necessary data on user requests to metadata.
A feature of the system is its orientation of storing source data as well as primary processed data, for example, data after calibration, using the Write-One-Read-Many method. Adding data to local repositories should be done through special local services that provide, among other things, semi-automatic collection of meta-information in the process of downloading new data.
At present, a prototype of the system has been deployed on the basis of the SINP MSU, which develop the technology of building distributed storages for particle astrophysics.
This work was supported by the RSF Grant No. 18-41-06003.
Primary author
Dr
Alexander Kryukov
(SINP MSU)
Co-authors
Andrey Mikhailov
(Matrosov Institute for System Dynamics and Control Theory, SB RAS)
Andrey Shigarov
(Matrosov Institute for System Dynamics and Control Theory, SB RAS)
Prof.
Igor Bychkov
(Matrosov Institute for System Dynamics and Control Theory, SB RAS)
Minh-Duc Nguyen
(M.V.Lomonosov Moscow State University)
