Final
Scatterplotting
I rearranged the data file so that it would show only the 2023 final grade for each country, later I determined a color palette for the 4 levels.
I figured I might want to get rid of the units and make labels for each country but then would need a scale system that would fit with the country’s size perhaps, so I cleaned the data so it would only show the 2023 grade, after that I switched the values in the country’s name to the relative population size in the world.
I then translated the csv code to json using this website and added to the code but all the circles were gathered in the corner together because the numbers are way too small for pixel space, so I multiplied the d.x and d.y and that helped a little. After a few more changes I received a more pleasing result.
My result after multiplying by 10 – China is obviously far but I’m still missing half the data
Obviously having it organized this way doesn’t make much sense because the x value is relative and shouldn’t have it’s own axis. I decided that instead of trying to add another component (location, years etc) I would just have the circles scattered around the page in a random order or in one line, the main interaction should be when each circle is clicked and not through some filtering or table of content.
Design Decisions:
Layout: I would have continued with the full page layout but I’m not sure I’ll be able to use all the space considering that there are only two factors in this visualization. I think I might center everything in the end and just put the index in the bottom.
Font: I like font “Lato” because it somewhat resembles “Proxima Novo” which is always very useful. I added the code to the <head> and created a new class there that would later be used for the main title and descriptive paragraph.
Color: If I won’t be using colors than I would try to add opacity to the circle so that they could all be seen but I am leaning toward my initial intention which was to create four different colors for the stages. I also think there’s something interesting about maybe not knowing in advance what is the situation or what country it makes you think about your assumptions. I think it could also be funny to maybe make the circles animated, like people or faces, because the anonymity of the circles makes it harsh, predetermined, as if it’s a constant state that can’t be changed.
Labels: I would like them to be minimal and appear when the circle is clicked and not hovered otherwise it would be a mess. Not sure where the data goes so I’ll wait a little with that.
Colors and labels:
I wanted to work with Category 10 or Ordinal Scale that is part of the d3 library, to make 4 different levels of the “democracy state”. I found interesting schemes online but they are not in the colors I wanted so I’ll customize them later.
let color = d3.scaleOrdinal()
.domain(["New York", "San Francisco", "Austin"])
.range(["#FF0000", "#009933" , "#0000FF"]);1- CSS Customizing is an option but it doesn’t take data in consideration.
2- d3.scale.category10 in older versions of the library, now it’s called d3.scaleOrdinal(d3.schemeCategory10), or any other scheme category.
3- creating var color
4- I need to get the max and min to determine scale but my data is all in json and doesn’t have text so I’ll just use the x and y and see what happen.
CSV and JSON
CSV- Comma Separated Values
JSON- Javascript Object Notation
I have worked with JSON in the past for “live web” class so I thought it would be a good idea to start with it for my data cleaning but then the data became too complicated and messy so I switched to a previously arranged CSV file I found online.
I later realized that copying everything straight into a json file would have been much easier but I’ll return to this option later after I manage to bind the CSV first.
Error in the Console!
I made a shorter list just for Afghanistan, which is the first country in the list but the browser just wouldn’t upload the data, constantly giving me this error:
So I decided it’s time to use a server.
I went to my DigitalOcean droplet just to find out it wasn’t working so had to make it work through the itp.io domain but that wasn’t the end of it. I tried accessing my files through the Fetch directory and it just won’t work so I just uploaded everything into a folder in my own private domain (that I keep for my portfolio) and then ran it again.
This time it worked and I finally got all my data logged into the console.
Binding the data
There are a few stages in binding the data using D3 in JS, I will try to list them all here in hopes of better understanding them myself:
1- Scale- in previous versions of D3 it was called d3.scale.linear but in the latest version of the library it’s called d3.scaleLinear. It takes a set of numbers and defines a scale to them, it uses two components: the domain and the range, so that the domain is the data space and the range is the pixel space. You can later calculate different ratios to better represent your data or possibly make it look better, what’s important is that it lets you transform the numbers into pixels that you see on screen.
There is no need for scaling the data if it’s already in pixel space, but most times it is not and would require some translation to fit the screen.
2- var svg- Appends a new element to the HTML DOM element (body in this case) into. The way .append works is that it sort of holds on to an object so that it could be changed, The .attr methods that come after it are meant to create those changes (width, height, etc) and they work only on the specific element that is selected.
3- svg.selectAll– This method is the most important since it finally connects all our elements to the data itself, up until now we had all this elements created separately on the page but there was no assignment of them together.
.enter and .exit are crucial but will be explained separately
You can also put all of the elements in a separate variable to avoid complication, so var svgs (rects, circles etc.) could be defined before all the attributes are added.
This allows to handle changes in data without complicating the code.
4- function render– Takes in two arguments (data, color) and can alter the color of specific elements connected to certain data. If we use this function on the rects variable (or any variable defined to select all the svg elements) then we can play around with colors.
In cases where data is dynamic and constantly changes perhaps without our direct knowledge we can set 4 stages for the code: Bind, Enter, Update and Exit.
5- Connecting the data in an csv file- up until now I used data that was listed in the HTML file directly, now I will add the x and y values from an existing csv file. To do that I create a new variable in the end of my code var myArrayOfObjects which would later go into the d3.csv method along with the function type that is meant to give us the x and y properties of our data.
Here is an example using circles instead of rectangles:
Setting the Environment
I wanted to work locally in the beginning so didn’t try the Ajax or npm options that were mentioned on he d3.js website. I do have a server running on digital Ocean so I might try them later on, but for now I have it all set offline. I downloaded the libraries from this link and kept uploaded my repository to this GitHub page.
Data:
I found some interesting data on this website and decided to use it for the rest of the examples as well. This EIU (Economist Intelligence) report discusses the state of democracies around the world by creating an index system for measuring each country’s different components. These factors are later being calculated into a final grade which fits into one of the four “levels” of democracy: Full Democracy, Flawed Democracy, Hybrid Regimes and Authoritarian Regimes.
I found that information to be interesting, especially in light of recent events around the world, our post covid-19 era and social media’s effect on information distribution and understanding of truth. I also feel that this crisis in democracy is related to other phenomenons that are happening here in the USA and would like to maybe emphasize that later on, but for now I will use the content as it is presented in the report.
You can view the whole report here:
Democracy-Index-2023-Final-report
Files
I set up all my files in a local folder and added the libraries to those folders. In cases where I could avoid opening a new Js or css file I did so and used the <script> and <style> tag where I thought it’s needed.
Server , Terminal and Libraries
I eventually installed node and npm in my directory and then added the d3.js library through the apt npm install command. I also downloaded the 2 versions on the website and another one I found in the book just to make sure I won’t have any problems when I add it to my website. I’m attempting to maybe run the whole thing through the terminal because it’s only the html that needs replacing and the data is pretty easy to manage, but not sure yet.
I got the Fetch directory to work so now I can upload the files there, but still think that working with the Terminal only might have been faster.
The Refugee Project
This is a project I’ve been following for a while now.
I feel like it’s content has a lot to do with the chaotic feeling of our times and is connected to many events that are currently happening around the world. It is not just war and financial situations that are being discussed here but also many other topics that are not always covered by the media.
I feel like it’s important for us as human beings to understand what’s happening around us even is it’s not directly related to our daily lives. Although the peak is mentioned as 2022 this continuous mobilization of people has been happening for quite some time now and is rarely being discussed in the global context this map suggests. The information is presented with great detail and is being updated regularly, something which I find important for future discovery.
In terms of the visual elements and overall design and feeling:
Although it looks like a pretty standard layout that is structured in a rather traditional way (maybe resembling that of a news website or other formal source of information), I think it has some benefits that other websites don’t offer. The strong visual identity and original interactions makes this for me a unique experience not only for learning about the subject but also for further investigating into the design process .
The opening slide:
I think having this opening slide with a sort of an introductory paragraph really makes a difference in the way you approach this experience. The dark background with just a hint of color from the logo makes you understand that this is not a funny topic and that this information should be approached in a more serious manner.
I like how the color in the logo is the only hint to the information later presented, the gradient feels to me like a good way to visualize transition, which is the very essence of the life of a refugee here. The location and size of this logo is for me another sophisticated remark on how this phenomenon is so often missed or being hidden because it involves the otherwise marginalized groups is our society.
The Main map:
The main page presents the map with two shifting states: one for the outgoing movement and one for the incoming, when the escaped countries are the first to be revealed in a bright red.
I think this small choice of creating this navigation bar, where the entire map turns from it’s initial alarming red to the more soothing blue, is an interesting metaphor for maybe the feeling of relief upon arrival to the safety of an asylum country. I think this interaction in it’s smoothness is also a good reminder to us the viewers that in this quick shift between these situations isn’t really possible in real life. As we navigate through these maps we soon understand that this process of immigration, perhaps running away from your home country, is not something that can be done in a click of a mouse, it is not a moment and not always a possibility for things to suddenly turn blue, it is much more complicated than that.
Although the data does speak about refugees seeking an asylum, in most cases the process of going from “red” to “blue” can be extremely dangerous and complicated, so I think that this choice of a rather simple and clean aesthetics gives room to the real stories that are presented on the side. I don’t think it’s obvious that people would want to read so many details about events that are so tragic in other people’s life, but here we are presented with a very clear vision and thoughtful presentation that really invites us to dive in.
Graph Charts:
On the side menu you can choose between reading the articles on the lower part (through the “read more” tag attached to each image) or between filtering the existing graph chart by country or year (you can also run it as a sequence through the play button). These options are expected to be found in almost any GIS visualization so that wasn’t something I was overly impressed with, but I did find the whole combination of the components and the general proportions of this structure to work in very effective way.
i think this layout is exceptionally successful because the interaction through the bars on the upper part is also a sort of gallery scroll through the images in the bottom, making it all become one cohesively themed experience.
The choice to make this side menu dynamic is also something I thought was very smart. I think having the experience of just wondering through this map without certainty of what you might see or looking for is crucial to the understanding of the feeling of immigration.
The choice to make this side menu dynamic is also something I thought was very smart. I think having the experience of just wondering through this map without certainty of what you might see or looking for is crucial to the understanding of the feeling of immigration.
Animation and informative labels:
. 
Each country when clicked presents a label with number of outgoing or incoming refugees. This labeling system is not new or original but the vector animation that accompanies these labels is an interesting addition. The vector is colored in the same scheme as the rest of the website but is has a gentle gradient that helps you understand the direction of the refugees as they flee their country. The line is purposefully thin and elegant so that it doesn’t force it’s present on the page and by that in my opinion allows for greater legibility and bigger attraction. I think that this choice is again a very successful way to bring awareness into harsh realities that otherwise we would try to avoid, This transition in the animation is done is such a smooth way that again it only puts more emphasize on the difficulty and hardships of this situation. When people imagine immigration or escape I think they normally think there’s a route or some target that people seek, but this visualization shows how much uncertainty and anxiety is involved in this process, it shows how this reality can really shatter your feeling of belonging and break communities apart.
Overall I think this website is a good example of how small details in interaction and layout design can have a big effect when visualizing information. I think that although a map might seem like the obvious choice for this sort of presentation it could have easily become tedious and irrelevant to the viewers, so the way that this experience unfolds itself is also very helpful and successful in my opinion. The project uses complicated and rather big quantities of data so I feel that in this aspect the research and analysis for this project are also very impressive.
Other interesting examples:
Who We Are: Simulated Dendrochronology of Immigration to New York City, 1840–2017
Visualize NYC 2021 / MIT Civic Data Design Lab
You’ve Got Mail: Part 6 of a Year-Long Personal Data Project
NYC Health Rat Information Portal
311 SERVICE REQUESTS IN NEW YORK CITY
All of Inflation’s Little Parts (2008)
How Fake News Takes Flight on Twitter/ ACCURAT
p5 Examples:
https://vda-lab.github.io/2015/10/hands-on-data-visualization-using-p5
Scatterplot Examples:
Observable Exercises
Exercise 1- Draw a Smiley
Exercise 2- Draw a flower petal
First Graph Chart
Creating a graph with a dataset
I created my first chart using a small dataset that was found in Murray’s book, in the beginning it only showed div blocks that had no specific measurements so were hard to identify. I used his explanations from the book to change the style() method using (d) + “px” to return the hight value and after that creating a variable barHeight to enable different data sets including a one with randomly generated values.
. 
Creating a graph with SVG
There are 2 ways to generate svg :
1- Through an <svg> element created inside the HTML <body> tag.
2- Through manipulation of the <body> DOM element in the Javascript code (through the <script> tag but also could be done in a separate file)
Here is a screenshot of squares generated in both ways, as you can see in both cases they look identical, but the JS version can be manipulated with data, it can be binded to the objects in CSV, JSON or other formats of text based files.
It’s important to notice that the .enter method is designed to create the DOM elements in JS, as if those elements are entering the page. This means that objects that would be created after the initial .enter will be added and not replace the previous elements.
Starting out
Books:
I started my D3.js journey with a book I bought a while ago called “Interactive Data Visualization for the Web” by Scott Murray. I didn’t have a chance to complete all the assignment in it so I was happy this class could be an opportunity to do so.
I was pretty shocked to find the whole thing was just randomly scanned into a pdf online but I used the printed book for my learning process.
Syllabus:
I also used exercises from our class found in this syllabus.
I followed the daily exercises on “Observablehq.com” following this timeline.
I posted my work there in a separate blog post called “Observable Exercises”.
Tutorials:
I found interesting tutorials online which I highly recommend:
Then I found there’s a more advanced version of it here
Turtle evolution on a remote island
Usu scripta evertitur scribentur cu. Eum te vivendo albucius. Probo dicit ut his, consul tempor nusquam est ea. Ut cum altera dictas, qui et expetendis incorrupte. Tibique sensibus rationibus qui ut.
Nam ut mucius conclusionemque, et eligendi ullamcorper nec. Te novum dicam vix, sit minim equidem ei, id tantas graeci nostro pro. Erant ubique mediocritatem sit te. Decore signiferumque te sit, docendi commune omittantur at pri. Timeam denique adipiscing sea no, vide sapientem ocurreret sea ex, unum quando vix id. Ne pro vero elit iriure.
Vim alii sumo ad, his ei eius dolores phaedrum. Mucius impedit suscipit cu mea. Putant fuisset accusata his in, id qui altera possim consectetuer. Novum equidem inimicus pri et, quo ea sonet facilis. Id qui quis everti vivendo. Probo maiorum cu vel, sit et affert perpetua honestatis. At ferri animal aliquid eam, putant persius vel ne.
Accusata gubergren incorrupte eu vis. In vel summo mucius, et corpora noluisse voluptua vel. Autem molestie pro ut, habeo posse omittam no qui, vix nostrum constituto at. Pri phaedrum adolescens ei, recusabo explicari ad qui. An eum omnium vituperatoribus, sea eruditi pertinax et, at diam mollis accusata pri.