{"id":1279,"date":"2020-12-11T22:44:57","date_gmt":"2020-12-11T22:44:57","guid":{"rendered":"https:\/\/aurorasaurudev.wpengine.com\/?p=1279"},"modified":"2024-03-18T21:33:24","modified_gmt":"2024-03-18T21:33:24","slug":"eyes-on-the-aurora-part-3-exploring-over-a-thousand-nights-of-aurora-on-your-phone","status":"publish","type":"post","link":"https:\/\/blog.aurorasaurus.org\/?p=1279","title":{"rendered":"Eyes on the Aurora, Part 3: Exploring Over a Thousand Nights of Aurora on Your Phone"},"content":{"rendered":"

Guest post by Jeremy Kuzub<\/em><\/p>\n

Attending AGU 20?\u00a0Jeremy will be presenting\u00a0Keogramist<\/a>\u00a0as\u00a0a poster in\u00a0The MacGyver Session<\/a>: The Place for Novel, Exciting, Self-Made, Hacked, or Improved Sensors and Software Solutions to Understand Space Weather eLightning on December 15, 2020 at 6:00 AKT\/7:00 PT\/8:00 MT\/9:00 CT\/10:00 ET\/15:00 UTC. There will be a Q&A element, so bring any questions for Jeremy!<\/strong><\/p>\n

Not attending this year? The information on his poster is publicly available here<\/a>.<\/strong><\/p>\n

 <\/p>\n

\"An<\/a>
Figure 1. Exploring all-sky camera video from AuroraMAX using keograms and the immersive first person viewer at Keogramist.com<\/figcaption><\/figure>\n

This is it, the final part of our three-part tour of how researchers and citizen scientists observe and catalog the aurora!<\/p>\n

In Part 1 we looked at All Sky Cameras<\/span><\/a>, which are specially designed to capture the entire sky in every frame and record thousands of images a night for months and years on end.<\/span><\/p>\n

\"An<\/a>
Figure 2. An all-sky camera\u2019s view of the night sky. Its \u201cfish-eye\u201d lens sees the entire sky from horizon-to-horizon. The northern horizon is at the top of the image, southern at the bottom. (Illustration by author, data from AuroraMAX played faster than real-time)<\/figcaption><\/figure>\n

\u00a0<\/span><\/p>\n

In part 2, we looked at \u201ckeograms\u201d<\/span><\/a>, a special data visualization which compresses an entire night\u2019s all-sky camera recording into a single image, like a seismograph of the sky so that specific events can be identified at a glance.<\/span><\/p>\n

\"An<\/a>
Figure 3. Creating a keogram from an all-sky camera timelapse video. The center column of pixels is sampled from each frame, then stacked left-to-right to make a keogram image of the entire night\u2019s activity. (Animation by author, data from AuroraMAX)<\/figcaption><\/figure>\n

In Part 3, we\u2019ll look at how a project created by a citizen scientist used these data sources to make a web app that enhances the explorablability, immersiveness of all-sky camera imagery.<\/span><\/p>\n

Aurora observation by citizen scientists<\/span><\/h2>\n

Researchers are not the only ones looking at all-sky cameras and keograms. Aurora reseasearch has long been a collaborative effort with citizen scientists, for example the worldwide Visual Observer volunteers during the International Geophysical Year of 1957-8 [1], and the members of the Alberta Aurora Chasers that first identified the STEVE phenomenon in 2016 [2]. As a continued part of this collaboration, many all-sky cameras are accessible online.<\/span><\/p>\n

The AuroraMAX all-sky camera and archive<\/span><\/h2>\n

One such camera is the <\/span>AuroraMAX camera<\/span><\/a> in Yellowknife, NWT. Maintained by the University of Calgary\u2019s <\/span>Auroral Imaging Group<\/span><\/a> and in cooperation with Astronomy North and the Canadian Space Agency. This is an uncalibrated full-color all-sky camera that provides\u00a0 <\/span>real-time<\/span><\/a> images approximately every 5 seconds. The AuroraMAX website also features a <\/span>deep archive<\/span><\/a> of nightly time-lapse videos. For example, <\/span>here is the night of March 1, 2019<\/span><\/a> which had active aurora conditions and a clear sky:<\/span><\/p>\n

\"A<\/a>
Figure 4. A single frame from one night\u2019s timelapse video from the AuroraMAX all-sky camera video archive, the night of March 1, 2019<\/figcaption><\/figure>\n

The challenge: lots of nights, lots of data<\/span><\/h2>\n

With such a wealth of time-lapse imagery, a big challenge is simply finding your way around the archive. If you don\u2019t know what night you are specifically looking for, how do you find moments of aurora activity and filter out nights of cloudy weather? How do you find aurora overhead, or on the horizon, recurring patterns throughout the night, or rarer events like red aurora? I spent a lot of time searching through this video archive and felt that there was an opportunity to help navigate and experience the years of timelapse videos in a new way.\u00a0<\/span><\/p>\n

A\u00a0project created by a citizen scientist: Keogramist<\/span><\/h2>\n
\"A<\/a>
Figure 5. The Keogramist web app landing page introduces users to the concept of keograms, Kp Index, and the first-person perspective.<\/figcaption><\/figure>\n

As a citizen scientist, <\/span>using my background in computer vision and software development, I envisioned a web app that would help users explore years of AuroraMAX all-sky camera video using keograms, and watch any night\u2019s aurora activity from a first-person view, as if they were there in Yellowknife. In other words, I wanted to take hundreds of hours of video and provide a big-picture view and a very personal experience for the viewer using the same data set.<\/span><\/p>\n

Why on the Web and on Phones?<\/span><\/h2>\n

I wanted this project reach as many people as possible – this meant it had to work on the devices they use most often to access the internet, which is often their smartphones [4]. The app had to work on desktops, laptops, tablets, and smartphones, so had to adapt to whatever screen was available.\u00a0<\/span><\/p>\n

On smartphones, apps often need to be installed from an app store. However modern web browsers are very powerful, and a standard called \u201cProgressive Web Applications\u201d[5] allows a website to have similar functionality to a downloaded app, without the commitment and hassle of installation.<\/span><\/p>\n

Seeing hundreds of nights at a glance using a keogram bookshelf<\/span><\/h2>\n

The first task was to make over a thousand nights of video easy to triage at a glance. I wanted users to be able to find the most interesting nights throughout the year and the best aurora moments during those nights, which ebb and flow on the scale of minutes. They should be able to see if any given night was cloudy, had low contrast due to moonlight, or if the all-sky camera dome was obscured by snow. This was a perfect job for keograms, arranged chronologically and split by year:<\/span><\/p>\n

\"Keogram<\/a>
Figure 6. A stack of keograms as an index of aurora activity. The interface is like a bookshelf, where each shelf is a single night from dusk till dawn. These are aligned by start and end time of the associated timelapse video so consistent patterns can be seen comparing night and morning activity.<\/figcaption><\/figure>\n

The keograms were created with a <\/span>Python<\/span><\/a> program. It processed every frame of the archived timelapse video for each night to create the keogram image files. Another program extracted the corresponding timestamp from the corner of the video frame using machine vision and stored descriptions of each night\u2019s observations in a \u201cmetadata\u201d file that could be used to sort the keograms and timelapse videos.<\/span><\/p>\n

Now that we had keogram images and context for them,\u00a0 it was time to organize them into an intuitive interface.<\/span><\/p>\n

Correlating geomagnetic activity with visual aurora<\/span><\/h2>\n

Increased auroral activity is correlated with increased geomagnetic activity, as measured by magnetometers across the globe. Forecasts of this activity include a scale called \u201cKp index\u201d [3] \u2014 analogous to the Richter scale for seismic activity (See this <\/span>Aurorasaurus blog post<\/span><\/a>). I wanted to align this data with the keogram images using a \u201cheat-map\u201d visualization strip that runs along the bottom of each keogram. This way, the visual aurora can be correlated with the magnetic activity in a more intuitive way.<\/span>
\n<\/span><\/p>\n

\"A<\/a>
Figure 7. Three broad ranges of activity type, each with strength levels. Blues for \u2018quiet\u2019 periods, purples for \u2018unsettled\u2019 and oranges for \u2018storm\u2019 conditions.<\/figcaption><\/figure>\n

This Kp data is available from <\/span>NOAA\u2019s space weather archive<\/span><\/a>, and a Python program combined this data with the keogram description data for use in the app interface.\u00a0 Citizen science often involves combining and processing data from multiple places, and programming with Python opens up a wide range of software tools to help in these tasks.<\/span><\/p>\n

Making an Immersive Video Experience<\/span><\/h1>\n

When a user finds an intriguing a keogram, they probably want to dive into the data to find out more. I wanted the app to make them feel like they were transported to Yellowknife under that night\u2019s aurora. Practically speaking, this meant showing them a \u2018first person viewpoint\u2019 of the timelapse video from the AuroraMAX camera – turning the circular fish-eye perspective into a 360 degree virtual sky so they could look up and around with a mouse or with a finger-tip.<\/span><\/p>\n

The approach I chose was a \u2018virtual planetarium\u2019 model. A planetarium is essentially a movie screen, but instead of a flat rectangle, the screen is the inside of a dome. Projecting video onto a dome creates a fully immersive first-person viewpoint because it surrounds the viewers just like the night sky. The projector system (one or more projectors) has to cover the whole dome with imagery to make for a convincing illusion.<\/span><\/p>\n

\"Diagram<\/a>
Figure\u00a0 8. A planetarium works by reversing the path of light from the sky to the camera, so that the image is projected back through a fish-eye lens onto a dome which acts as the sky. (modern planetariums don\u2019t just use one camera but an array of them, however the final effect is the same as a single central camera)<\/figcaption><\/figure>\n

In the app, I created a virtual dome surface in 3D space and \u201cprojected\u201d the AuroraMAX timelapse video onto it. Every 3D environment has a virtual observer\u2019s point of view, and I put this observer right in the center, so that the dome surrounds the viewer. This allows the user to look around in any direction and see what they would have seen had they actually been standing in the same spot as the AuroraMAX all-sky camera that night. There is some fine tuning needed as well: the dome\u2019s geometry has to precisely \u201cundo\u201d the distortion AuroraMAX fish-eye camera lens. This level of graphics performance is possible because modern web browsers use the hardware-based graphics accelerators on their host devices via a standard called \u201c<\/span>WebGL<\/span><\/a>\u201d.<\/span><\/p>\n

\"A<\/a>
Figure 9. Early prototype of projecting the all-sky camera video onto a virtual dome to create an immersive experience using JavaScript 3D code in a web browser. Note that the timelapse video is running at about 100 times real time speed (1 frame every 5 seconds), so it is slowed down in the final app<\/figcaption><\/figure>\n

Navigating the night<\/span><\/h2>\n

These timelapse videos projected overhead can look convincing, but I wanted the user to be able to jump to any part of the night to see highlights and interesting features. I decided to put the keogram right at the bottom of the view, like a dashboard, along with the local time. Clicking or tapping on any part of the keogram fast-forwarded to that point in time. The video also had to be slowed down a bit, and other fine-tuning controls were needed, so these controls were added as \u2018url parameters\u2019 which advanced users could tweak.<\/span><\/p>\n

\"An<\/a>
Figure 9. Virtual trees hide the edge of the video and a tent and campfire illumination give a little sense of story and presence.<\/figcaption><\/figure>\n
\"Animated<\/a>
Looking straight up at the aurora overhead. Clicking and dragging or touching and dragging allows the user to look all around them as the night plays out. This is sped up a bit from the actual app<\/figcaption><\/figure>\n

A big part of understanding the rhythms and patterns of aurora is not just time, but direction. <\/span>Substorms<\/span><\/a> and bursts of activity cause the main body of auroral activity to move north and south over a viewer, so I felt it was important to add an unobtrusive compass to the interface. The approach I chose was an unobtrusive \u2018hula-hoop\u2019 that surrounded the user on the snow around them. All-sky cameras are usually aligned with magnetic north, so both true and magnetic directions are indicated.<\/span><\/p>\n

\"An<\/a>
Looking down at the compass ‘hula-hoop\u2019. Both magnetic and true north directions are indicated. Hey look – there is your virtual tent, a reminder of the \u2018virtual dome\u2019 technique of projecting the aurora video overhead.<\/figcaption><\/figure>\n

\u201cCozy\u201d details<\/span><\/h2>\n

Finally A line of trees and snow, and a tent glowing warmly make the experience more immersive, like the viewer was out camping for the night. This is not just for looks – the edges of the dome have artifacts that need to be concealed like buildings and lights,\u00a0 and the user needs something to stand on! The tent acts as a reference point when the user gets a bit dizzy from looking up and around.<\/span><\/p>\n

Future work<\/span><\/h2>\n

There is still so much to explore with this wealth of data. For example, machine learning and pattern detection could be used to classify and rank auroral activity with users participating in the ranking process. A virtual camera with different lenses and controls could be added to allow users to take \u2018photos\u2019 of what they are seeing and understand aurora photography technique. Finally, more all-sky cameras could be added from other locations. Someone might even find a pattern or auroral behaviour that has never been classified before!<\/span><\/p>\n

Challenge<\/span><\/h2>\n

For part 3, we want to link you to a few specific nights of aurora activity in the app, so you can see the variety of auroral patterns and activity:<\/span><\/p>\n

    \n
  1. March 17, 2015<\/span><\/a> – The St. Patrick\u2019s Day Storm – a Kp7\/8 night with rare strong red aurora emissions from oxygen higher in the atmosphere<\/span><\/li>\n
  2. August 22, 2017<\/span><\/a> – A moonless clear night of almost constant late-summer aurora activity from dusk till dawn, with the aurora overhead and violet emissions from nitrogen<\/span><\/li>\n
  3. April 20, 2018<\/span><\/a> – the sky clears in time for a strong substorm that fills the sky overhead with corona aurora.\u00a0 Just before dawn, another substorm creates some amazing coronal aurora patterns overhead<\/span><\/li>\n
  4. January 5, 2017<\/span><\/a> – pulsating aurora fill the sky after local midnight<\/span><\/li>\n
  5. October 30, 2016<\/span><\/a> – the bottom edge of the aurora curtain is bright magenta from nitrogen emissions. The playback speed is slowed down to really focus on this fast-moving feature<\/span><\/li>\n<\/ol>\n

    Jeremy\u2019s Bio<\/span><\/h2>\n

    Jeremy Kuzub is an interactive software simulation developer and aurora photographer originally from Alberta and now based in Ottawa, Canada. He has created an aurora visualization web app to explore years of <\/span>AuroraMAX<\/span><\/a> public outreach video of Yellowknife aurora, which can be viewed at <\/span>keogramist.com<\/span><\/a> and write articles about aurora photography, equipment, and history at <\/span>capturenorth.com<\/span><\/a>.<\/span><\/p>\n

    More to Explore<\/span><\/h2>\n

    Try the Keogramist web app right here<\/span><\/a><\/p>\n

    You can read more about all-sky cameras in part 1 of this article series: <\/span>Eyes on the Aurora, Part 1: What is an All-Sky Camera?<\/span><\/a>.<\/span><\/p>\n

    You can read more about keograms in part 2 of this article series: <\/span>Eyes on the Aurora, Part 2: What is a Keogram?<\/span><\/a><\/p>\n

    The famous paper \u201cThe Development of the Auroral Substorm\u201d by Syun\u2010Ichi Akasofu<\/span>
    \n<\/span>    http:\/\/www.ss.ncu.edu.tw\/~lyu\/lecture_files_en\/Lyu_Aurora\/Ref_Papers_AuroraSubstorm\/Akasofu_1964.pdf<\/span><\/a><\/p>\n

    The AuroraMAX website, an amazing resource with years of all-sky camera imagery<\/span>
    \n<\/span>     https:\/\/www.asc-csa.gc.ca\/eng\/astronomy\/auroramax\/default.asp<\/span><\/a><\/p>\n

    Astronomy North is a northern sky outreach and education website with aurora forecasts<\/span>
    \n<\/span>    http:\/\/astronomynorth.com\/<\/span><\/a><\/p>\n

    References<\/span><\/p>\n

      \n
    1. \u201cRockets, Radar, and Computers: The International Geophysical Year\u201d, NOAA<\/span>
      \n<\/span>      https:\/\/celebrating200years.noaa.gov\/magazine\/igy\/welcome.html<\/span><\/a><\/li>\n
    2. \u201cIntroducing Steve – A Newly Discovered Astronomical Phenomenon\u201d, <\/span>https:\/\/www.sciencealert.com\/introducing-steve-a-newly-discovered-light-in-the-sky<\/span><\/a>, retrieved November 2, 2020<\/span><\/li>\n
    3. \u201cWhat is Kp index?\u201d, <\/span>https:\/\/blog.aurorasaurus.org\/?p=187<\/span><\/a>, retrieved November 2, 2020<\/span><\/li>\n
    4. \u201cMobile vs. Desktop Usage in 2019\u201d, <\/span>https:\/\/www.perficient.com\/insights\/research-hub\/mobile-vs-desktop-usage-study<\/span><\/a>, retrieved November 2, 2020<\/span><\/li>\n
    5. \u201cIntroduction to Progressive Web Apps\u201d,\u00a0 <\/span>https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/Progressive_web_apps\/Introduction<\/span><\/a>, retrieved November 2, 2020<\/span><\/li>\n<\/ol>\n

      Acknowledgements<\/span><\/h2>\n

      Special Thanks to Dr. Liz MacDonald and Laura Brandt at <\/span>Aurorasaurus<\/span><\/a>, Dr. Eric Donovan, Dr. Emma Spanswick, and Darren Chaddock at the <\/span>University of Calgary Auroral Imaging Group<\/span><\/a>, Dr. Don Hampton at the <\/span>University of Alaska, Fairbanks Geophysical Institute<\/span><\/a>, the <\/span>Royal Astronomical Society of Canada<\/span><\/a>, and the <\/span>Alberta Aurora Chasers Group<\/span><\/a>.<\/span><\/p>\n

       <\/p>\n","protected":false},"excerpt":{"rendered":"

      Guest post by Jeremy Kuzub Attending AGU 20?\u00a0Jeremy will be presenting\u00a0Keogramist\u00a0as\u00a0a poster in\u00a0The MacGyver Session: The Place for Novel, Exciting, Self-Made, Hacked, or Improved Sensors and Software Solutions to Understand Space Weather eLightning on December 15, 2020 at 6:00 AKT\/7:00 PT\/8:00 MT\/9:00 CT\/10:00 ET\/15:00 UTC. There will be a Q&A element, so bring any questions… Read More »Eyes on the Aurora, Part 3: Exploring Over a Thousand Nights of Aurora on Your Phone<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":1290,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rop_custom_images_group":[],"rop_custom_messages_group":[],"rop_publish_now":"initial","rop_publish_now_accounts":[],"rop_publish_now_history":[],"rop_publish_now_status":"pending","neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"off","neve_meta_content_width":70,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","footnotes":""},"categories":[24,7,29,17,1,12],"tags":[],"class_list":["post-1279","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-citizen-science","category-education","category-hands-on-activities","category-other-resources-for-aurora-hunters","category-uncategorized","category-understanding-the-aurora"],"_links":{"self":[{"href":"https:\/\/blog.aurorasaurus.org\/index.php?rest_route=\/wp\/v2\/posts\/1279","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.aurorasaurus.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.aurorasaurus.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.aurorasaurus.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.aurorasaurus.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1279"}],"version-history":[{"count":0,"href":"https:\/\/blog.aurorasaurus.org\/index.php?rest_route=\/wp\/v2\/posts\/1279\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.aurorasaurus.org\/index.php?rest_route=\/wp\/v2\/media\/1290"}],"wp:attachment":[{"href":"https:\/\/blog.aurorasaurus.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1279"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.aurorasaurus.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1279"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.aurorasaurus.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1279"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}