====== The bibliography ====== * [[https://arxiv.org/pdf/1911.00534v1.pdf|ArXiv:1911.00534v1]]. (the struggle) Towards an open source policy. Y. G.Grange1, et al. * [[https://arxiv.org/pdf/1912.09877v1.pdf|ArXiv:1912.09877v1]]. CTLearn: Deep Learning for Gamma-ray Astronomy. D. Nieto, et al. * [[https://arxiv.org/pdf/2001.08619.pdf|ArXiv:2001.08619]]. SciServer: a Science Platform for Astronomy and Beyond. Manuchehr Taghizadeh-Poppa, et al. * Peter Nemeth, [[https://arxiv.org/abs/1711.08522|Astroserver - Research Services in the Stellar Webshop]] * [[https://arxiv.org/abs/1512.02019|Status Report of the DPHEP Collaboration: A Global Effort for Sustainable Data Preservation in High Energy Physics]], Dec. 2015. ===== GAN: fast generator ===== * [[https://arxiv.org/pdf/1905.11825.pdf|ArXiv:1905.11825]] Fast Data-Driven Simulation of Cherenkov DetectorsUsing Generative Adversarial Networks, A Maevskiy, D Derkach, N Kazeev, A Ustyuzhanin,M Artemev and L Anderlini on behalf of the LHCb collaboration. * [[https://arxiv.org/pdf/1909.12160.pdf|ArXiv:1909.12160]]. Galaxy Image Simulation Using Progressive GANs. Mohamad Dia, Elodie Savary, Martin Melchior, and Frederic Courbin ===== ANN and noise reduction ===== * {{ archive:appds:gutta_kadimesetty_ieeetrpms_2019.pdf |Kadimesetty 2019}} Convolutional Neural Network-Based Robust Denoising of Low-Dose Computed Tomography Perfusion Maps. Venkata S. Kadimesetty, Sreedevi Gutta, Sriram Ganapathy, and Phaneendra K. Yalavarthy. * [[https://arxiv.org/pdf/1912.13171.pdf|ArXiv:1912.13171]] Deep Learning on Image Denoising: An Overview. Chunwei Tian, Lunke Fei, Wenxian Zheng, Yong Xu, Wangmeng Zuo, Chia-Wen Lin * [[https://arxiv.org/pdf/1910.09435.pdf|ArXiv:1910.09435]] Background Rejection in Atmospheric CherenkovTelescopes using Recurrent Convolutional NeuralNetworks. R.D. Parsons, S. Ohm * arXiv: 1708.00961 {{ archive:appds:1708.00961.pdf |Low Dose CT Image Denoising Using a Generative Adversarial Network with Wasserstein Distance and Perceptual Loss}} * arXiv: 1807:08176 {{ archive:appds:1807.08176-cnn.pdf |A Convolutional Neural Networks Denoising Approach for Salt and Pepper Noise}} * Jingwen Chen and et. al. {{ archive:appds:chen_image_blind_denoising_cvpr_2018_paper.pdf |Image Blind Denoising With Generative Adversarial Network Based Noise Modeling}} * Yasushi Amari and et. al. {{ archive:appds:impuls_noise_reduction.pdf |A Study on Impulse Noise Reduction Using CNN Learned by Divided Images}} * ERIC KVIST {{ archive:appds:mlp_vs_cnn.pdf |A comparative study between MLP and CNN for noise reduction on images}} * Kartik Audhkhasi, Osonde Osoba, Bart Kosko, {{ archive:appds:n-cnn-published-may-2016.pdf |Noise-enhanced convolutional neural networks}}, Neural Networks 78 (2016) 15–23 * Fabian Dietrichson {{ archive:appds:ius_2018_proceedings_ultrasound_speckle_reduction_using_generative_adversial_networks.pdf |Ultrasound speckle reduction using generative adversial networks}} * C. J. Díaz Baso, J. de la Cruz Rodríguez, and S. Danilovic, {{ archive:appds:1908.02815-solar_denoising.pdf |Solar image denoising with convolutional neural networks}}. ArXiv:1908.02815. * Kai Yi, Yi Guo, Yanan Fan, Jan Hamann, Yu Guang Wang, {{ archive:appds:2001.11651-cosmo_vae.pdf |CosmoVAE: Variational Autoencoder for CMB Image Inpainting}}. ArXiv:2001.11651. * Rich Ormiston, et. al. {{ archive:appds:2005.06534-gwave.pdf |Noise Reduction in Gravitational-wave Data via Deep Learning}}. ArXiv:2005.06534. * Masato Shirasaki, et. al. {{ archive:appds:1911.12890-gan.pdf |Decoding Cosmological Information in Weak-Lensing Mass Maps with Generative Adversarial Networks}}. ArXiv:1911.12890. * Aryeh Brill, et. al. {{ archive:appds:2001.03602-multiple_iact.pdf |Investigating a Deep Learning Method to Analyze Images from Multiple Gamma-ray Telescopes}}. ArXiv:2001.03602. See also: [[https://arxiv.org/search/advanced?advanced=&terms-0-operator=AND&terms-0-term=noise&terms-0-field=all&terms-1-operator=AND&terms-1-term=cnn&terms-1-field=abstract&classification-computer_science=y&classification-physics_archives=quant-ph&classification-include_cross_list=include&date-filter_by=past_12&date-year=&date-from_date=&date-to_date=&date-date_type=submitted_date&abstracts=show&size=200&order=-announced_date_first| ArXiv: CNN and Noise in last year]] ===== CNN ===== * [[https://arxiv.org/pdf/2001.03602.pdf|ArXiv:2001.03602]]. Investigating a Deep Learning Method to AnalyzeImages from Multiple Gamma-ray Telescopes. Aryeh Brill, at al. * [[https://arxiv.org/pdf/1912.09898.pdf|Arxiv:1912.09898]]. Studying deep convolutional neural networks with hexagonal lattices for imaging atmospheric Cherenkov telescope event reconstruction. D. Nieto, A. Brill, Q. Feng, M. Jacquemont, B. Kim, T. Miener, T. Vuillaume * [[https://arxiv.org/pdf/1803.10698.pdf|ArXiv:1803.10698]] Application of Deep Learning methods to analysis of ImagingAtmospheric Cherenkov Telescopes data. I. Shilona, et. al. * [[https://arxiv.org/pdf/1907.02428.pdf|Arxiv:1907.02428]] Lyard E, Walter R, Sliusar V and Produit N, for the CTAConsortium, Probing Neural Networks for the Gamma/HadronSeparation of the Cherenkov Telescope Array ===== Machine learning ===== * [[https://arxiv.org/pdf/1911.11717| DeepRICH: Learning Deeply Cherenkov Detectors]], Authors: Cristiano Fanelli, Jary Pomponi * **[[https://arxiv.org/pdf/1910.09435|Background Rejection in Atmospheric Cherenkov Telescopes using Recurrent Convolutional Neural Networks]]**, Authors: R. D. Parsons, S. Ohm * [[https://arxiv.org/pdf/1907.02428| Probing Neural Networks for the Gamma/Hadron Separation of the Cherenkov Telescope Array]], Authors: Etienne Lyard, Roland Walter, Vitalii Sliusar, Nicolas Produit * **[[https://arxiv.org/pdf/1905.11825| Fast Data-Driven Simulation of Cherenkov Detectors Using Generative Adversarial Networks]]**, Authors: Artem Maevskiy, Denis Derkach, Nikita Kazeev, Andrey Ustyuzhanin, Maksim Artemev, Lucio Anderlini * [[https://arxiv.org/pdf/1903.11788| Cherenkov Detectors Fast Simulation Using Neural Networks]], Authors: Denis Derkach, Nikita Kazeev, Fedor Ratnikov, Andrey Ustyuzhanin, Alexandra Volokhova * [[https://arxiv.org/pdf/1807.09024| Deep learning techniques applied to the physics of extensive air showers]], Authors: A. Guillen, A. Bueno, J. M. Carceller, J. C. Martinez-Velazquez, G. Rubio, C. J. Todero Peixoto, P. Sanchez-Lucas * [[http://www.opennet.ru/opennews/art.shtml?num=50127|NVIDIA открыла код StyleGAN]], генератора лиц на основе машинного обучения. [[https://medium.com/syncedreview/nvidia-open-sources-hyper-realistic-face-generator-stylegan-f346e1a73826|NVIDIA Open-Sources Hyper-Realistic Face Generator StyleGAN]] * [[https://www.tensorflow.org/|TensorFlow]]. TensorFlow™ is an open source software library for numerical computation using data flow graphs. * [[http://pytorch.org/|PyTourch]]. PyTorch is a python package that provides two high-level features: * Tensor computation (like numpy) with strong GPU acceleration * Deep Neural Networks built on a tape-based autograd system * [[https://caffe2.ai/|Cafe2]]. Caffe2 aims to provide an easy and straightforward way for you to experiment with deep learning and leverage community contributions of new models and algorithms. ===== Noise reduction ===== * [[https://arxiv.org/pdf/1909.11403| Prototype Schwarzschild-Couder Telescope for the Cherenkov Telescope Array: Commissioning Status of the Optical System]], Authors: C. Adams, et al. * [[https://arxiv.org/pdf/1908.04620| Sensitivity Improvements of Very-High-Energy Gamma-Ray Detection with the Upgraded H.E.S.S. I Cameras using Full Waveform Processing]], Authors: Justus Zorn, H. E. S. S. Collaboration * [[https://arxiv.org/pdf/1907.08617| Detection of extensive cosmic ray air showers by measuring radio emission]], Authors: Yoshitaka Kawashima * [[https://arxiv.org/pdf/1904.06324| Direct measurement of stellar angular diameters by the VERITAS Cherenkov Telescopes]], Authors: W. Benbow, et al. * [[https://arxiv.org/pdf/1906.00646| A photomultiplier tube test stand and on-site measurements to characterise the performance of Photonis XP3062 photomultiplier tubes at increased background light conditions and lower gain]], Authors: J. Zorn, K. Daumiller, R. Engel, H. -J. Mathes, M. Riegel, R. Smida, F. Werner ===== Метаданные ===== * [[https://projects.iq.harvard.edu/provenance-at-harvard|Provenance@Harvard]] * [[http://camflow.org/|CamFlow]] * [[https://www.w3.org/TR/prov-dm/|W3C PROV-DM]]. Data model for provenance ===== Методы машинного обучения ===== * [[http://hal.univ-smb.fr/hal-01841581v2/document|GAMMALEARN: DEEP LEARNING APPLIED TOTHE CHERENKOV TELESCOPE ARRAY (CTA)]]. Mikaël Jacquemont * [[https://arxiv.org/pdf/1611.09832|The analysis of VERITAS muon images using convolutional neural networks]] \\ Применение нейросети типа convolutional neural network из библиотеки Keras (с TensorFlow в качестве интерфейса) в задаче идентификации частиц гамма-адроны в гамма-телескопе VERITAS. * [[http://www.ipp.ca/pdfs/Tomalty_IPP_report-summer2016.pdf|Particle Identification in Cherenkov Detectors using Convolutional Neural Networks]] \\ Применение нейросети такого же типа тоже в TensorFlow к идентификации частиц электрон-мюон, правда, не в гамма-телескопах, а в другом черенковском астрофизическом проекте Super-Kamiokande. * [[https://pos.sissa.it/301/809/pdf|Exploring deep learning as an event classification method for the Cherenkov Telescope Array]] \\ Применение нейросети такого же типа к идентификации гамма-адроны в будущих гамма-телескопах CTA. Правда, в качестве интерфейса к библиотеке Keras вместо TensorFlow ипользуется другая аналогичная среда Theano. * [[https://www.sciencedirect.com/science/article/pii/S0927650517302219|A deep learning-based reconstruction of cosmic ray-induced air showers]] * [[https://habrahabr.ru/post/309508/|Что такое свёрточная нейронная сеть]] * [[http://neuralnetworksanddeeplearning.com/chap6.html|Deep learning]] ==== Инструменты ==== * [[https://ai.intel.com/intel-ngraph/|Intel nGraph]]: An open source library for developing frameworks that can efficiently run deep learning computations on a variety of compute platforms ===== Форматы данных ===== ==== Инструменты описания бинарных форматов данных ==== === Kaitai Struct === [[http://kaitai.io|Kaitai Struct]] Kaitai Struct is a declarative language used for describe various binary data structures, laid out in files or in memory: i.e. binary file formats, network stream packet formats, etc. The main idea is that a particular format is described in Kaitai Struct language (.ksy file) and then can be compiled with ksc into source files in one of the supported programming languages. These modules will include a generated code for a parser that can read described data structure from a file / stream and give access to it in a nice, easy-to-comprehend API. [[https://2016.zeronights.ru/wp-content/uploads/2016/12/ZN2016-Kaitai-Struct.pdf|Обратная разработка бинарных форматов с помощью Kaitai Struct]] === DFDL === [[https://www.ogf.org/ogf/doku.php/standards/dfdl/dfdl|Data Format Description Language (DFDL)]] Data Format Description Language (DFDL) is a language for describing text and binary data formats. A DFDL description allows any text or binary data to be read from its native format and to be presented as an instance of an information set. DFDL also allows data to be taken from an instance of an information set and written out to its native format. DFDL achieves this by leveraging W3C XML Schema Definition Language (XSDL) 1.0. It is therefore very easy to use DFDL to convert text and binary data to a corresponding XML document. === FlexT === [[http://hmelnov.icc.ru/FlexT/index.ru.html|FlexT: язык спецификаций бинарных форматов данных]] ===== Методы агрегации ===== ===== Критерии функционирования системы ===== ==== Best Practices in Research Data Curation ==== === Resources === [[http://www.dcc.ac.uk|Digital Curation Centre (DCC)]] The Digital Curation Centre (DCC) is an internationally-recognised centre of expertise in digital curation with a focus on building capability and skills for research data management. The DCC provides expert advice and practical help to research organisations wanting to store, manage, protect and share digital research data. [[https://www.dataone.org/best-practices|The DataONE Best Practices database]] The DataONE Best Practices database provides individuals with recommendations on how to effectively work with their data through all stages of the data lifecycle. === Papers === * [[http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005510|Good enough practices in scientific computing]] * [[http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001745|Best Practices for Scientific Computing]] * [[http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005097|Ten Simple Rules for Digital Data Storage]] * [[http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003285|Ten Simple Rules for Reproducible Computational Research]] * [[https://arxiv.org/ftp/arxiv/papers/1512/1512.00988.pdf|Towards a Model for Computing in European Astroparticle Physics]] * [[http://www.data-archive.ac.uk/media/2894/managingsharing.pdf|MANAGING AND SHARING DATA]] ===== Гамма астрономия ===== * [[https://taiga-experiment.info/taiga-iact|Сайт проекта TAIGA/TUNKA]] * [[http://x4u.lebedev.ru/che2019/f/talks/02.Kuzmichev.pdf|Экспериментальный комплекс TAIGA: статус, результаты, перспективы]]. * {{ archive:appds:the_future_of_gamma-ray_astronomy.pdf |The future of gamma-ray astronomy}} * {{ archive:appds:mephi_lecture2_gamma_rays.pdf |Gamma-ray Asrophysics}} * [[https://indico.cern.ch/event/568904/contributions/2646096/attachments/1486401/2308455/ISAPP_Lecture_I.pdf|Gamma-ray astronomyLecture I: History, instruments & detection methods]] Markus Ackermann * [[https://arxiv.org/pdf/1508.05190.pdf|Space- and Ground-BasedGamma-Ray Astrophysics]]. Stefan Funk * [[https://iopscience.iop.org/article/10.1088/1742-6596/718/2/022010/pdf|Gamma-Ray Astronomy from the Ground]]. Dieter Horns ===== Прочее ===== ==== BigchainDB ==== [[https://www.bigchaindb.com/|BigchainDB]] \\ BigchainDB is for developers and organizations looking for a scalable, queryable database with blockchain characteristics such as decentralization, immutability and the ability to treat anything stored in the database as an asset. ==== PlantUML ====