GWOSC and Auxiliary Data – Combining LIGO style detectors with all earth-based sensor networks

Correlating LIGO style strain and auxiliary data with all data from earth-based sensor networks

I followed many of the pathways from this email looking for downloadable data on the Internet accessible from the browser. All I found was your groups favorite software ways to access things you are interested in. And nowhere did I find a simple list of dates, channels, sizes, data formats, and “click to download”. Just find all the auxiliary data, put lists in HTML/Javascript accessible form. Do NOT put terabytes of downloads, but tell people how big it is, what is contained, show samples of the output – in human and machine readable formats.

If you put things on the Internet that only people with very specific tools can use, then it is not really “open”.

Some time ago, you released 3 hours of auxiliary data and I wrote to you about ways to improve your methods. That link at https://www.gw-openscience.org/auxiliary/GW170814/ is better than what you just wrote by far, but much less than useful for any serious questions.

I have been writing to you since May 2019 asking about basic data. What I want to do is simple. I could have been done years ago. But every time you “share” it is only on your terms and in your favorite ways, and not Internet-wide common formats accessible to the 5 billion people using the Internet now.

I need data from times when three detectors sites are working, and I need ALL the auxiliary data for those times – in browsable form NOT as giant lumps that are impossible to work with, but as a nice complete list. If something is large, you take it bit by bit. If there is really TeraBytes, then show the details first. It is a database, not a data dump. We should say “share the databases” not “share giant lumps of useless to most people data”. You should give lists of tables, fields in each database, and queries. And that all has to work for people with only a browser HTML/Javascript. And in both human and machine readable format. The ONLY Internet-wide format now is utf-8. I spent the last 25 years checking what people and groups use on the Internet.

The LIGO style detectors CAN be used to scan the earth and solar system. But it won’t happen if you take years to learn how to share in truly accessible forms. For EVERY second of every strain data file, there should be a parallel set of auxiliary data streams. I looked very closely at the strain data and it has a LOT of noise from earth in it, that can be identified, correlated with earth-based networks and carefully separated and used. And that can only be sorted out with ALL the auxiliary data.

Are there any people there older than 30 or 50? I am 74 now and I do not have a lot more years to wait for someone who understands databases and simple programming. All the “auxiliary data” really needs to be in a department or group that cares about earth and solar system “noise”, not treated as something to be eliminated. I wanted to combine the LIGO sensors into earth-based gravitational and geophysics and other sensor networks, but I think I am not going to live long enough to see that even get started. A simple demo to track the moon, or scan the interior of the earth, or track earthquake seismic waves would be a good start. It is NOT hard, just tedious, requiring care and close attention to details.

https://www.researchgate.net/project/Solar-System-Gravimetry-and-Gravitational-Engineering

Richard Collins, The Internet Foundation

> On 03/10/2023 5:23 PM CST gwoscworkshop@igwn.org wrote:
>
>
> Dear GWOSC User Group,
>
> GWOSC has recently released a new type of data set!
>
> The LIGO instrument at each site uses a large number of sensors to track the state of both the instrument and the surrounding environment. A subset of those sensors (channels) are used as part of astrophysical analyses by the LIGO-Virgo-KAGRA collaboration. The particular subset of channels provided here has been used to remove additional noise from LIGO data or to improve the quality of the astrophysical analyses.
>
> This data set includes 40 channels from LIGO Hanford and 46 channels from LIGO Livingston, that were used in O3 astrophysical analyses. The data set has a total size of 13 TB. This data set includes channels that were used to create data quality flags, to subtract instrumental noise from the strain data, and to measure the amount of correlated noise between the gravitational-wave detectors.
>
> Learn more at: gwosc.org/O3/auxiliary/
>
> Best Regards,
> The GWOSC Team
>
> Contact Email: gwosc@igwn.org