- Arecibo Celebrates National Engineers Week 06 Apr, 2022
- The Arecibo Observatory at the Upcoming 240th American Astronomical Society Meeting06 Apr, 2022
- The Arecibo Observatory Survey Salvage Committee Report06 Apr, 2022
- Facilities and Operations Update06 Apr, 2022
- PRISMA Meteor Radar Arrives at AO04 Apr, 2022
- The Grand Reopening of the Angel Ramos Science and Visitor Center at the Arecibo Observatory01 Apr, 2022
- Orbital stability analysis and photometric characterization of the second Earth Trojan asteroid 2020 XL531 Mar, 2022
- Arecibo Celebrates International Women’s Day31 Mar, 2022
- A Letter from the Director Eng. Francisco Cordova31 Mar, 2022
- The History of Arecibo’s Legacy Telescope to Impact the Future, Thanks to the AO Salvage Survey Committee31 Mar, 2022
- Announcing AO/GBT Single Dish Summer School May 16th - 20th, 2022 30 Mar, 2022
- NSF REU program at Arecibo receives funding for next 3 years23 Mar, 2022
- A Parkes "Murriyang" Search for Pulsars and Transients in the Large Magellanic Cloud23 Mar, 2022
- Noise analysis in the European Pulsar Timing Array data release 2 and its implications on the gravitational-wave background search23 Mar, 2022
- Arecibo S-band Radar Characterization of Local-scale Heterogeneities within Mercury's North Polar Deposits23 Mar, 2022
- Arecibo’s Eye on the Sun21 Mar, 2022
Byadmin15 December 2021 #AOScienceNow
The Arecibo Observatory was part of a coalition of institutions to win HPCWire’s 2021 Reader’s Choice Award for Best High-Performance Computing Collaboration for their efforts to “transfer petabytes of irreplaceable observation data to a safe place in proximity to capability-class computing to foster analysis” within weeks following the collapse of the 305-meter radio telescope at the Arecibo Observatory.
The Arecibo Observatory was well-situated to facilitate this critical task thanks to the creation of its Big Data Program in 2019 through a supplemental grant funded by the National Science Foundation. The program was created with the goal of improving data management and analysis by taking advantage of new cloud storage and computing capabilities.
Although the data storage project had already been in operation for several months, it became urgent to quickly move the data following the failure of the first of the telescope’s cables in August 2020, since the staff were not sure which areas of the facility would be impacted in the event of a total collapse.
That is when several institutes, including the Texas Advanced Computing Center, the Engagement and Performance Operations Center, the NSF Cyberinfrastructure Center of Excellence Pilot, and Globus at the University of Chicago, offered to assist Arecibo and the University of Central Florida. The coalition was able to accelerate the storing process and successfully secure decades of collected datasets.
“For us it’s an honor to be part of this great community that worked tirelessly to salvage decades of historical data,” says Mr. Francisco Córdova, director of the Arecibo Observatory. “We are incredibly thankful to everyone that helped us throughout this process.”
Mr. Córdova notes that the telescope’s ultimate collapse reinforces the need for programs like AO’s Big Data program. “As computing power and tools have become more accessible, a more structured approach to data management and analysis - especially at large facilities that are continuously accumulating large amounts of observational data - is needed.”
In addition to storing the data and making it accessible to all Arecibo users, the Big Data program was designed to search for newly discovered phenomena in datasets acquired over the telescope’s 50+ years of observations by improving the data processing time, incorporating real-time analysis of some of the recurring data sets, and implementing advanced machine learning techniques.
“There is great potential in the AO dataset,” says Mr. Córdova. “At the time much of these data were taken, we still did not know what an exoplanet was, or what a Fast Radio Burst detection would look like, so there is a good probability that signals of these phenomena are hidden in the data but we just weren’t able to recognize them with the technology we had at the time.”
As for the future of Arecibo’s Big Data program, Mr. Francisco Torres, a software engineer on the Big Data team says, “We have several other projects in the works. There is still so much we plan to do to help the scientific community access and utilize the wealth of data contained in the Arecibo archives so that the data can continue to be used to make significant contributions to the exploration of our universe.”
“We have several other projects in the works. There is still so much we plan to do to help the scientific community access and utilize the wealth of data contained in the Arecibo archives so that the data can continue to be used to make significant contributions to the exploration of our universe.” - Eng. Francisco Torres, Software Eng, team member of the Big Data Program
Keywords: big,data, program, arecibo, observatory, cordova, torres