WEEK 6 – Midterm Due

Assignments:

  • Document your midterm, link to it in your blog but also make sure the documentation link is up to date.

P5+ml5 Canvas

I started working on my ml5 poseNet canvas that could possibly send the location data to the server, but ran into some issues with the objects inside the ml5.

I went through a bunch of tutorials (+help) only to find out that the ml5 objects were actually changed and renamed their latest version so I’m not completely out of it. I will now continue on to try and connect the canvas to the server using socket.io and see how it goes from there. I might try using Flash animation to create the movement of the shapes? we will see.

Here is my progress so far:

https://editor.p5js.org/el1841/sketches/uLmbt6qGK

I think this could be useful later in understanding the skeleton object:

 

 

Adding Sockets and Sending JSON data
I created a new folder in the Fetch directory titled “ml5” and put all my code there. I changes the variables to fit with my new “nose” function.
It seems to work because the data is logged in the console but the camera doesn’t turn on so nothing moves. I suspect this has to do with my lack of knowledge about video sharing, I will complete the lessons I missed and see from there.


I downloaded the new cetificate+key files that Shawn sent us and updated them in the Fetch main directory. I added the code for the https server to the previous “server_sockets_example” server file to make it secure and named it “https_sockets.js”. It all works on port 443 but the circle is not showing on the canvas (the camera works though which is an improvement). I will now try to establish peer-to-peer connection with WebRTC and see if that helps.

WEEK 5 – Midterm Ideas

Midterm Project Ideas
Review – Extra Help – Misc Topics

Assignments:

Doodle Dancing

I was going to to call this “Doodle Party” but Shawn seemed really excited with “Doodle Dancing” so let’s go for that. This project is essentially a shared canvas in which people can move their doodles around together. I would like the users’ drawing to be the input but if that won’t be possible then I will prepare some custom illustrations that could fit with the PoseNet model.
The users would then be able to place their doodles on the canvas and animate them with their bodies and cameras.
I think I should start with some basic lines and see how it goes from there.

I’m assuming I will be using:
1- WebRTC 
2- ml5 PoseNet model and neural network training
3- p5 canvas

I came up with this drawing for my idea, but obviously I have no idea what I’m doing and should be prepared for changes and LOWER ALL EXPECTATIONS IMMEDIATELY!

 

Here are resources I collected to start my ml5 learning:

ml5js.org

A Beginner’s Guide
Image Classification with MobileNet
Webcam Image Classification
Train Your Own Neural Network
Pose Estimation with PoseNet
Pose Classification with PoseNet and ml5.neuralNetwork
An hour of code- p5 and poseNet

Shared Canvas with P5 and Sockets

The Coding Train ==> ml5

Other Resources:

Real-Time Pose Estimation…. / Dan Oved

 

 

WEEK 4 – Live AV Web

The Live AV Web: HTML5 Audio and Video with WebRTC and HTTPS
Code from class

Assignments:

  • Get up and running with HTTPS
  • Enhance one of your last 2 assignments (or create a new one) with the ability to share camera images.
  • Come up with an idea for a midterm project, blog about it, prepare to discuss in class.

I downloaded the https code and uploaded to the Fetch directory after changing the “key” and “cert” variables to the right file names.

I uploaded the js file (httpsserver.js) to the fetch directory along with the two files of the certificate and key.
I saw the lock sign next to the address on the browser confirming it was a secure connection, it all worked and I was happy.

 

WEEK 3- Transmitting Other Data Types

Transmitting Other Data Types:

Collaborative Drawing, Mouse Events, Other data:

Notes

The Canvas

JavaScript 103: Objects and JSON

p5.js with Sockets

Assignments:

  • Improve on the example, send a different type of event to the server and to the clients. This could be a collaborative drawing program or a means to control other people’s experience.
  • Email me your NetID and Digital Ocean Server IP and I’ll setup YOURNETID.itp.io to point to your server. We’ll need this done for next class so the sooner the better!

Express Module and Forever App

I took the example code from class that listens to port 3000 to respond with the
“Hello World” text when a client requests the path “/”.
I saved it as “httpserver.js” and uploaded to the main directory on Fetch, then ran it through the terminal –> node httpserver.js

That failed completely and annoyed me so I killed all processes and started again.

This time I was a little more suspicious because I thought there might be some node processes running in the background, I wasn’t sure if I installed “forever” (because I’m sure I did at some point) and also wasn’t sure if I installed the Express module, which I’m pretty convinced I did, So I did things as follows:
1- I ran the command that would list forever activity–> forever list 
2- Received the surprising message that no such app exists so went ahead and installed it in the main directory (not the “public” one in which I previously ran my commands) –> npm -g install forver. That yielded some nice results (or maybe it’s just me that likes pink loading bars), anyway I continued with a joyous feeling. 
3- I ran the new httpserver.js file again through the forever app–>
forever start httpserver.js
4- I was happy to discover that my feeling was right as I opened port 3000  (“http://el1841.itp.io:3000/”), to finally witness the magical appearance of “Hello World”  in my browser window.
5- I had to reconnect to the server for some reason because it didn’t allow me to run the the forever list command. This kept happening on various occasions so think i should check it later. The process showed lcAM as my process id.
6- I shut the process down–> forever stop lcAM (uid)

The amazing port 3000 in action

A colorful forever shut down 

Sockets

Socket.io is a node module that makes working with WebSockets easier. It also transparently supports older methods such as AJAX in the event that the browser doesn’t support Web Sockets.

1- I installed the socket.io module–> npm install socket.io
2- I saved the example code as “sockets_example.js”
3- I created a new directory called “sockets” and uploaded the script there.
4- I created a new HTML file with the example code and changed SERVER_NAME–>
var socket = io.connect(‘http://147.182.222.126:8080/’);
5- Saved files and uploaded to Fetch
6- ran–> node sockets_example.js and everything worked well  

Other Data / P5 Canvas

1- I downloaded the files from the example and named them “canvas_socket”
2- Opened p5 editor and wrote the code for the drawing application
3- Added
<script type=”text/javascript” src=”/socket.io/socket.io.js”></script>
To the head of the HTML to connect to the socket server
4- Added:
var socket = io.connect(); socket.on(‘connect’, function() { console.log(“Connected”); }); To the top of JS file
5- I uploaded the p5 folder to the public directory in Fetch (server file remains the same as it has the public directory connected from it)
6- The drawing was visible at http://el1841.itp.io/p5/

6- I rewrote the drawing function in the same way we did in class so that it will include different variables for the mouse location and a data object that could be sent. That sketch worked but it wasn’t collaborative yet.
7- I added the socket.on function as we wrote it in class but still couldn’t get the drawing to be collaborative. I got this error message in the terminal:

I tried going through the whole process again and reinstalled socket.io but that didn’t work either. I also realized that maybe I should have worked with the previous code example and not the server file from the chat example (although it’s pretty much the same), it also worked but wasn’t collaborative, this time there were no errors though.
I will try debugging this later because it seems like an issue with the socket.io module or one of the functions missing a variable and I feel could be easily fixed.

I managed to debug it and yes it was something with socket.io but not what I thought. I wrote “socket.io.emit” instead of “io.emit” in the function, I fixed it an it all worked!
(I think it would be good to go over the functions in socket.io library, because it might allow interesting experimentations.. but maybe later)

WEEK 2- JavaScript 102, AJAX and WebSockets

JavaScript 102: Becoming Live AJAX, and WebSockets
Serverside JavaScript: Node.js
The foundation: Chat
Forever – (Will make your life better!)
Virtual Private Servers with Digital Ocean
Unix Essential Training: Logging in and using the command prompt

Assignments:

  • Get up and running with Node.js on Digital Ocean and the example Chat Application. Try to extend it a bit, make it better, more fun.
  • Put your self-portrait up on the server as well.
    Link to it from a blog post so we can all look at them in class.

I downloaded this version of Node.js and created an account on Digital Ocean to create my own private virtual server.
I ran node through the terminal to get access to the server and then opened files according to the requirements: index.html, style.css and script.js.
I put all the files in a folder called “Live Web” and put it in the “Fetch” directory.
I also created a separate “server.js” file and put it outside of the folder in the main directory so it would connect all the files to the server.

I ran some commands in the terminal to get things working:
1- First had to connect to the DigitalOcean server through the IP and password –>
ssh root@147.182.222.126
2- Then put in the password to get a “Welcome” prompt in the Terminal
3- I installed node.js before but wanted to check just in case, so ran –>
node -v to discover the known fact that I’m using the 8.9.0 version
4-  I also installed npm –>  apt install npm
5- installed express.js and socket.io through the “npm install” command
6- After everything was set I ran the server.js file –> node server.js
7- Inserted the IP number into the address line in the browser
8- Received the “Hello World” from my HTML file on the browser
9- I checked that it also works with my el1841.itp.io address

final result:

 * It’s important to notice that when connecting to Fetch you need to provide the following details:  “SFTP”, under “Hostname” or “Server” use the IP address for your droplet Digital Ocean. The “username” is “root” and the “password” is what you just created in Terminal.

I took the code from the class syllabus and created HTML, CSS and JS files accordingly. I uploaded them to the “public” directory on my “Fetch” folder and updated the “server.js” file. I ran the “node server.js” command in the terminal and managed to get this result on my el1841.ito.io website.

 

I asked a few people to write me messages on my new chat application so I could check if it actually works. I was happy to see that the messages kept on updating as each user added their input and there were no errors in the console or the terminal.

WEEK 1 – What is The “Live” Web?

Introductions

What is the “live” web? (Synchronous vs. Asynchronous)
HTML5 and Client Side JavaScript: JavaScript 101 and Media Elements

Videos:

HTML Basics

CSS Basics

CSS Basics – Better Organization

JavaScript DOM Basics

Assignments:

  • Add your name, preferred email address, and blog URL to this Google Doc. (Note: You must be using Google Docs with your NYU login in order to edit this document).
  • If you want more basic JavaScript, go through lynda.com’s JavaScript Essential Training
  • Using HTML5’s video or audio tags along with JavaScript, create an interactive self portrait. Link to it on your blog.
  • Write a blog post detailing the last “live” media you consumed. Answer these questions: What was it? Why did you consume it “live” or in real time? How was your relationship to the media or others affected through the “live” consumption of it? What was missing in your experience?