Emily Wang: Midterm Portfolio – #5 Audio Descriptions of Captions

Project Description:

For the REMIX assignment, I chose to use Pika, the AI video generator tool, to create the visual for my captions and voice over. I brought the captions in assignment 1 and broke them down into pieces and remixed. This is like an interesting experiment to see how you can play with the different combinations of the words and sentences. A change in order may result in a big difference in the meaning you wanna transfer to your audience.

Documentation:

Transcript:

I would like to grant you

a wish.

And she just watches TV?

She hasn’t said a word since the earthquake.

It’s as though I don’t exist.

IS THAT CLEAR ENOUGH?

If she had a lover, she wouldn’t stay at home watching buildings crumble all night long.

Right.

“I’m never coming back.”

No, no, no, why don’t you take a short trip or something?

What’s the most painful thing that’s ever happened to you?

“Living with you…is like living with a chunk of air.”

I can’t remember.

No matter what you wish for, you can never be anything but yourself.

Reflection questions: 

When I used Pika to generate the visual, I tried to feed various keywords to make the visual look exactly what I wanted. However, Pika is not doing great with facial expressions. One thing that I think could enhance the accessibility of the piece is to add more audio descriptions. The tricky thing is that since I remixed the captions, the visual happens really fast to match the audio. There’s already some overlapping between the captions, and I was unable to add audio descriptions in between.

What is the theme of the work? What is it you aim to express?

The theme of the work is to remix the original scene’s captions and audio descriptions and make it something ‘new’: to let the visual follow the audio. I aim to express the inanition inside the female character and the disconnection between the male and the female character.

How is that theme particularly expressed through the modality of the week?

When remixing the audio and generating the new visual, I first edited the captions and aligned them on the timeline and then used audio descriptions as keywords to generate the visual in Pika. The visuals were 100% triggered by the audio.

Which elements of the work are beautifully/wonderfully/perfectly expressed through the modality?

The dialogues between the characters. I used distinguishable voices to present the captions. And I adjust the speed and the pitch based on the emotion and the plot. Also, I think the visual part are well expressed. When generating the visual, I added the keyword ‘1990s grainy film’ for all scenes. And to make the frame and facial expression perfect, I tried as many keywords as I can.

Which elements are lost or inexpressible through the modality of the week?

The audio description. The tricky thing is that since I remixed the captions, the visual happens really fast to match the audio. There’s already some overlapping between the captions, and I was unable to add audio descriptions in between.

Who does this project exclude? Who would not be able to interact with this work ? Who is this modality not accessible for?

The people with visual impairment. This modality is not accessible for people with visual impairment because I did not include much audio description in my work which results in a missing of translation of key visual elements.

Now that you’ve identified who is excluded, what is one way you could remix this piece to include another population? (You don’t have to make this part, but think about it and write about it)

To include audio description of key visual elements.

For the remixes: what is lost and what is gained in this remix? What did you have to leave behind and what could you take with you?For this remix assignment, I would say a new meaning is gained. Assignment 5 is the most struggle one for me compared to the previous four, however, I’m glad that I was able to jump out of the original plot and impart new meaning to the old captions. One thing I could take with me is to always keep accessibility in mind when creating a work. I would definitely add some pauses to include audio descriptions in my work in the future!