{"id":1690,"date":"2021-06-30T03:57:53","date_gmt":"2021-06-30T03:57:53","guid":{"rendered":"\/?p=1690"},"modified":"2021-06-30T04:16:06","modified_gmt":"2021-06-30T04:16:06","slug":"nanograv-raw-observing-data-merging-all-global-sensor-networks-on-the-internet","status":"publish","type":"post","link":"\/?p=1690","title":{"rendered":"NANOGrav raw observing data – merging all global sensor networks on the Internet"},"content":{"rendered":"
Hello,<\/div>\n
<\/div>\n
I was visiting https:\/\/data.nanograv.org\/<\/a> to see what kind of data you are sharing online.\u00a0 I am tracking most all the sensor networks on the Internet, especially the emerging ones.<\/div>\n
<\/div>\n
If I understand this talk by Steve Taylor at https:\/\/www.youtube.com\/watch?v=_UptA7pkARo<\/a> you are only observing at widely separated times for about 30 minutes?<\/div>\n
<\/div>\n
He talks about the correlations and the common motions of the earth.\u00a0 It occurred to me that if there were simultaneous observations in all directions simultaneously one could solve for the movement of the earth.\u00a0 Since the nanoHertz signals cover 1E9\/365.25*86400 = 31.69 years, all the solar system observations won’t help much, except for time of arrival.<\/div>\n
<\/div>\n
Anyway, I would like to know what your “raw observations” look like, how many records, frame rate, size, format.<\/div>\n
<\/div>\n
I can’t deal with more than about 20 Terabytes at a time\u00a0 I just have a small machine.\u00a0 I can download Gaia 3, or Wikipedia, or Google NGram, or hundreds of major GitHub projects.\u00a0 I look at literally everything on the Internet, figure how it fits into the whole, and what it takes to standardize it for global communities of tens or hundreds of thousands of specialists to work together, or hundreds of millions or billions for many economic and social and environmental processes.<\/div>\n
<\/div>\n
Have you used the LIGO strain data too?\u00a0 I am working on that right now for a simple scan of the interior of the earth and moon.\u00a0 It is hard working with 16384 sps data from three really eclectic instruments, but I try to work with any dataset, no matter how odd or troublesome.<\/div>\n
<\/div>\n
If I can see a sample of one observing session, I can tell a lot.\u00a0 \u00a0Three stations all looking at pulsars in different directions would be better. I can see what would happen if all the sites ran continuously.\u00a0 And it can be the variable stars as well.\u00a0 If those are tracked for years, their models will improve and precise times and properties available.<\/div>\n
<\/div>\n
There are many electromagnetic sensor arrays new on earth that, and many potential gravitational networks.\u00a0 I used the superconducting gravimeter array almost 20 years ago now to measure the speed of gravity.\u00a0 That uses the JPL solar system ephemeris as reference. It is sensitive enough to put bounds on atmospheric models.\u00a0 Those are only single axis instruments, so I went through all the IRIS.edu seismometers to find that the broadband seismometers as gravimeters are sensitive enough to track the sun and moon vector tidal signal on all three axes with just a linear regression – offset and scale factor. There is no phase, which is why those three axis instruments can be used for gravitational GPS. Some of them are transportable.\u00a0 But you can solve for orientation of the axes, since that much data and only six parameters allows for continuous calibration.\u00a0 I like it because it works underground or undersea and is insensitive to ionospheric delays or satellite problems.\u00a0 The SGs cost about quarter million and their offset changes with the helium level.\u00a0 You can track the level of the helium at the station based on a simple regression to the sun and moon tidal signal.\u00a0 One line vector calculations.\u00a0 There are LOTS of gravimeters designs and groups now.\u00a0 \u00a0I have been encouraging all of them to increase the sensitivity and go to time of flight methods and frequencies.\u00a0 You know it well, once they have a signal, then you can make faster improvements.\u00a0 Hard to tell young people that, but it is always true.<\/div>\n
<\/div>\n
MEMS gravimeters, Bose Einstein gravimeters, atom interferometer gravimeters, electron interferometer gravimeters, electrochemical gravimeters. The best of\u00a0 them are just able to detect the seismic waves from magnitude 8 earthquakes anywhere in the world.\u00a0 Of course they should work well close by for volcanoes and such.\u00a0 The Japan earthquake registered on the SG network and also on exactly those broadband seismometers I had been calibrating as gravimeters.\u00a0 I want to use them all for constraining the global climate models.\u00a0 That is taking me a while to learn to download and use the data from the simulations.\u00a0 I have worked on global climate change off and on since about 1990 when I was working for Phillips Petroleum in their Business Intelligence group. I was helping with the models and negotiations.\u00a0 I am a world expert on global economic and social models and good enough to follow and improve the climate part.<\/div>\n
<\/div>\n
I am never sure what people might be interested in.\u00a0 I did go through the software defined radio groups a couple of years ago.\u00a0 There are quite a few amateur efforts to use that for regional and global monitoring of the spectrum.\u00a0 I think they might be slightly ahead of radio telescope groups on price and sensitivity.\u00a0 If a method is so cumbersome only a few people can get involved, it tends to die.\u00a0 I only have 23 years of looking at global groups on the Internet. But I would say most groups that are closed die, and part of that is pricing things so no one can get involved. That is why sharing of data from all these different networks can be so important to the future of the human species.<\/div>\n
<\/div>\n
If I were setting up a network of pulsar monitoring stations, I would not use the big telescopes, but rather widely dispersed arrays of different kinds of sensors – gravimeters, infrasound, magnetotelluric, electromagnetic interference, ionosphere, GPS dual frequency time monitors, Mossbauer detectors (these directly sample the change in the gravitational potential), and any atomic clock detectors (also time dilation, direct gravitational potential change detectors).<\/div>\n
<\/div>\n
Continuous, open, auditable, shared models, share algorithm development, all countries, all schools, any individuals or groups. The Internet is for everyone – no exceptions.\u00a0 And when you get hundreds of millions of people synchronized and working on common models, not hundreds of millions of separate copies of partial models – things can happen in days that otherwise would take decades or centuries.\u00a0 The rules for optimizing human use of the Internet are simple, you just have to use a lot of unique data.<\/div>\n
<\/div>\n
Richard Collins, Director, The Internet Foundation<\/div>\n","protected":false},"excerpt":{"rendered":"

Hello, I was visiting https:\/\/data.nanograv.org\/ to see what kind of data you are sharing online.\u00a0 I am tracking most all the sensor networks on the Internet, especially the emerging ones. If I understand this talk by Steve Taylor at https:\/\/www.youtube.com\/watch?v=_UptA7pkARo you are only observing at widely separated times for about 30 minutes? He talks about
Read More »<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13,19,23,7,6,12,16,21],"tags":[],"class_list":["post-1690","post","type-post","status-publish","format-standard","hentry","category-all-sky-cameras","category-collaborative-global-model-of-the-sun","category-experimenting","category-gravitational-engineering","category-gravitynotes","category-intelligent-algorithms","category-internet-best-practices","category-schools-universities-learning-and-working"],"_links":{"self":[{"href":"\/index.php?rest_route=\/wp\/v2\/posts\/1690","targetHints":{"allow":["GET"]}}],"collection":[{"href":"\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1690"}],"version-history":[{"count":4,"href":"\/index.php?rest_route=\/wp\/v2\/posts\/1690\/revisions"}],"predecessor-version":[{"id":1694,"href":"\/index.php?rest_route=\/wp\/v2\/posts\/1690\/revisions\/1694"}],"wp:attachment":[{"href":"\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1690"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1690"},{"taxonomy":"post_tag","embeddable":true,"href":"\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1690"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}