Human of AI
COCO is the database behind these three artworks that links to “91 object types that would be easily recognizable by a 4 year old”. It’s also a special database since the labeled objects in the database are not shown in the normal circumstances, but rather in some coincidental environments which also include other labeled objects. Therefore, COCO uses a machine rather than a human perspective. These images may be bizarre for a human, but they make sense for a machine because it reflects the connections and relationships between different labels.
Because it chooses words that are familiar to 4-year-olds, it gets me thinking about how this is different from teaching children. If we also set up a learning environment like this for a child (a very inhumane experiment, for sure), will the child come up with the same logic as the machine algorithm, such as that it is normal for a cat to be in a toilet?
This goes back to the topic we explored before. Machines don’t ask why, but humans do. A human would ask why a cat would stay in the toilet, so a human would not come up with the same logic. But for a machine, as long as there are enough samples about the cat and the toilet, the machine will assume that this is a normal combination of the two labels. COCO database reveals this in a very amazing way.
Declassifier reveals the classifying process of the model by showing the database behind the classifier. Most of the time, the data behind models is invisible. Machines just magically detect objects as if they “see” the objects. Declassifier attaches the pictures from the COCO database upon the detect boxes which reveals the actual process of machine learning –“prediction is based on others’ past experience”.
Humans of AI is the credit process of the database since it “awards 34,248 certificates of appreciation to all known contributors”. This might also be an expression of resistance. Because most of the time the data used by Big Data is taken without permission and does not give the data providers the respect they deserve.
All of COCO is the database itself but not exactly. It is the slide that contains all the data in the database, but revealing the context of the image, it gives life back to the data. The same picture is recognized as “two dogs” by machine, but it means “Stella, frisbee, Gryfe” for the data provider Rebecca Siegel. It touches me that in the era of big data, it is easy to also bring in the machine way of thinking, where we can only see data, or more commonly, numbers. But forget that there is often a vivid story behind each of these numbers. How’s machine learning and data change our perception of memory is worth thinking about.
Sketch
Here’s the link: https://editor.p5js.org/AislynnLi/sketches/OkfMbDo87
I first thought about what would happen if the detected things swap positions.
Here’s the code:
After trying it out I found that because it would always detect people, there will always be one object that is super big. When there is more than 1 object, it will not be stable. When there’s only one object, there will be a feeling that the object is the master, while the person is held in control by the object.
I find this very interesting. I tried to do more experiments on it. For example, I tried to make the transformed part transparent and attach the original camera to the back to emphasize this inversion more.
Here’s the link: https://editor.p5js.org/AislynnLi/sketches/Vg9gBqa0J
Well…here’s the effect:
I find the effect is not very stable and the concept is not that clear anymore.
I also tried to let the transformed part light the background.
Here’s the link: https://editor.p5js.org/AislynnLi/sketches/w8UsR2ZGK
Here’s the code and the effect:
Where’s the transform effect? As you can guess, I ditched that effect in the process of trying. But seeing a result like that, I asked myself again, why not just show the identified object? It was as if the machine could only see these objects.
And then I thought since I already only show the objects that have been detected, and I like my transform effect, how to combine them? I came up with the idea to try to accomplish the idea in the assignment, that is, the transform effect will only appear when the detect-box of my object overlaps with the generated box.
Here’s the additional code:
Here’s the final effect:
It may not be that creative, but I hope it brings enough fun.