CPD Final Project (WCS): Data Visualization – Interactive Map – Jingyi Zhu

Project: Data Visualization – Interactive Map

Date: 05/22/2019

Background

WCS is working on a research about juvenile sand tiger sharks in Great South Bay. Great South Bay is an ideal nursery ground for sand tiger sharks and juvenile sand tiger sharks migrates to Great South Bay as their summer location. Sand tiger sharks are vulnerable species according to the conservation status. They usually stay in shallow waters (<20 m) during the day.

WCS would like to better understand sand tiger sharks and Great South Bay as their habitat from the research. Their goal is to make the marine world around New York City visible to the citizens.

Idea and Concept

My final project idea is to visualize the sand tiger shark detection data that WCS provided on an interactive digital map. In this way, more information can be extracted from the data and the data can be more easily understood.

The datasets I received are of two juvenile sand tiger sharks in Great South Bay, New York. The sharks are tagged with signal emitters and the detection are done by signal receivers in fixed locations. Therefore the datasets are only about shark occurrences at each receiver.

Since WCS also would like to see what dataset overlapping can tell us about Great South Bay, I came up with an idea of overlapping human activity data with shark detection data to see how humans and sharks coexist in Great South Bay. I did some research and found that potential threats to sharks in general are by-catching and noise pollution which are related to vessel activities in addition to commercial fishing. Therefore I decided to incorporate vessel position data into the map.

I based my data visualization ideas on the datasets and WCS’s objectives. My goal of the data visualization project is to make the huge datasets visible and understandable for non-professional users, thus helping them to learn about sand tiger sharks in Great South Bay.

Process

1> Data Scraping and Processing

I first scraped data from Robots4Whales with Python Beautifulsoup module. But after studying the scraped data, I found that all the data points are at the same location, thus it is not suitable for visualization on a map. 

I downloaded vessel position data from marinecadastre.gov. Then I cleaned the vessel data and the shark data and converted them into json files with python.

As I prototype with the josn files, I continued to reprocess the data into structures that better suit my algorithms for data visualization.

2> Map Making

I used HTML, CSS and JavaScript to make the map. I first drew squares on the map to represent receiver locations.

i) Animated Map

I first made an animated map to look at the movement of every data point, so as to briefly understand the relationship between shark movements, vessel movements and time. One unexpected effect is that when animated, the map made obvious the entrance point of the sharks and the speed of each movement, and clearly show that sharks stay around one place for a long time.

ii) Static Map

My ideas of making a static map are to draw circles of different sizes to represent the cumulative occurrences and to draw translucent lines to represent the sharks’ simplified movement from one receiver to another so that the lines can layer upon one another to create the effect of a heat map to show the frequency of sharks taking each path and staying in each area. When the users choose to show the circles, they can hover on the receivers and read the exact number of occurrences.

I chose to visualize the vessel positions as dots instead of drawing vessel routes because the data points are dense enough to show the routes and drawing lines make the whole picture unreadable.

iii) User Interface

For the user interface, my idea is to make it as intuitive as possible. I made a side bar for users to show or hide the chosen datasets. I also made a timeline where users can choose the time range by month. Both control palettes only appear when the users hover on either side of the window. The background color of each choice are set to the corresponding color on the map and I used gray scale to distinguish chosen from unchosen.

Both animated and static maps have a user guide button and an explore button. The user guide automatically appears when users first open the map. When the explore button is clicked, screenshots of my findings about the datasets will appear, which is intended to show users some examples of the useful information hidden in the map and to encourage them to explore more.

In the animated status, users can use a drag box to adjust the speed of animation or to pause it.

iv) Introduction and Credits Page

In order to provide users with the basic background knowledge, I made an introduction page which briefly introduces my data visualization ideas, Great South Bay, sand tiger sharks and WCS’s shark detection methods, accompanied with links to more resources. I also put links to the two maps on the introduction page. All the credits are listed on the credits page.

Challenges and Further Developments

The available datasets are limited and many of the vessel datasets are incomplete. Thus, it took me more time than expected to extract the useful data. The shark datasets are not the best ones for visualization on a map since the receivers don’t follow the sharks but only wait for the sharks to come.

I didn’t receive photos or videos of the sharks from WCS, but I would like to include more media into the maps if available so that the maps can be more informative and educational.

The algorithms and data structures that I used to build the maps are not efficient enough, so it results in heavy computation. I would like to solve this issue in the future.

Week 11: Progress Report – Jingyi Zhu

Project: Interactive Digital Map – Data Visualization

Date: 05/07/2019

Static map of sharks (2017):

Overlapping vessel data and shark data (2017):

After the progress presentation, I improved my project based on comments and suggestions I received. In order to reduce the burden of computation and to visualize the data more clearly, I made a new map for shark data only that shows the amount of time for which the sharks stay around each receiver as well as the graph of sharks moving from one receiver to another.  I also built a simple homepage that links to the two maps.

Major functions that need to be implemented are the timeline and hover-to-show-information.

My idea for overlapping vessel data and shark data is to show the intersection of human activities and shark activities in the sand tiger sharks’ nursery. I did some research and found that potential threats to sharks related to vessels are by-catching and underwater noise. I am trying to visualize the datasets in a less messy way.

Week10: Progress Report – Jingyi Zhu

Project: Interactive Digital Map – Data Visualization

Date: 05/01/2019

Last week, I mainly dealt with data collection and processing. I learnt more about data scraping and file converting with Python modules. I now have three  sets of cleaned data in json files  – the whale data from Robots4Whales, the shark data from WCS and the vessel from NOAA. After studying the whale data,  I decided not to visualize it on the map because all the data points are  fixed in one position and thus visualizing them on a map is ineffective.

I spent a lot of time searching for vessel position data. Vessel position datasets are usually very large and not free of charge. I was lucky to find vessel data on NOAA.

I created simple dictionaries for the both the shark data and the vessel data. But as I start to make the map, I find that I need to process the vessel data into nested dictionaries to easily draw points for the vessels.

My data visualization idea is to first make static maps for each dataset with a week or a month as a unit. Then, I plan to animate the maps by interacting shark data and vessel data. The map will have basic functions such as switching layers. I also want to include user interactions such as clicking on data points to read information.

WCS Progress Report – Jingyi Zhu

Project: Interactive Digital Map – Data Visualization

Date: 04/24/2019

My project idea is to visualize marine animal related data and to raise awareness of marine animals around New York City by making an informative interactive map.

So far, I have identified two datasets that I would like to work with. One is the detection data of two tagged sand tiger sharks in Great South Bay provided by WCS, and the other is the detection data of four kinds of whales outside New York Harbor found on Robots4Whales website.

I am at the stage of understanding and analyzing data, trying to form an idea to visualize the data. Both datasets are about the existence of the target animals detected within the range of the signal receivers. I think a practical way of visualization is to show the frequency or length of existence. My inspiration comes from Nine Point Five Earthquake Visualization, a 3D visualization of earthquakes happened each year. 

Information that I currently need to help me understand the datasets are the age of the tagged sharks, the choice of the locations of the receivers, the detection range of the receivers and the range of transmission the sounds of whales.

I am also planning to add a layer of human activities that might affect sand tiger sharks and whales. After some research, I find that by-catch and noise from marine traffic are two major threats in addition to commercial fishing. I am currently looking for available datasets about marine traffic and fisheries. However, it is rather difficult to find recent data about fisheries.

My plan for this week is to finish data scraping and cleaning of the whale data and data mining of marine traffic and fishery data. My plan for next week is to finish information gathering on sand tiger shark and whales and make a prototype based on my current idea.