Progress
For the past few days, I added more videos and more visual effects to them. My final video is a remixed MV allowing real-time control. You may download my patcher from here.
Background videos
Previously, I added the “Colorblind” MV to my patcher. Since it has a wide range of black area, I would like to fill it with other patterns or videos. Therefore, I went online and downloaded two videos: one vintage-style animation, and one jellyfish video.
“Colorblind” MV
- As introduced in the previous post, I used jit.unpack and jit.pack to swap the color planes of the original MV. The song has lyrics containing many colors, but they don’t match with the colors in the modified MV, which recalls the idea of “colorblind”.
- The MV allows manual change of its scale and bias, which is smoothened by line objects.
- To make the MV play at its original FPS, I used “getfps” message to get it and route it back to the metro object.
- I also changed the size of the MV so that it doesn’t occupy too much space in the patcher.
- The volume is set to 0.5 upon loading this patcher so it plays the song. Other videos in the patcher are muted.
- To achieve background replacing, a suckah object is placed on the jit.pwindow of the MV.
Vintage-style animation
- I chose this animation because it has many colors rotating around the bottom center. It is a great material for plane mapping experiments.
- Since the video has 4 planes, I decided to do plane mapping with the RGB planes. In total, there are 9 plane mapping options, which I have included in the planemap.txt extracted by a coll object.
- The plane mapping option changes once per 3 seconds. It is sent to the jit.matrix to achieve the plane mapping.
- To emphasize the theme of “colorblind”, I lowered the contrast of the vintage-style video.
Jellyfish video: “scissors and glue”
- I used jit.scissors and jit.glue to reorganize the video parts because it makes the jellyfish ‘teleport’ on the screen, which is interesting.
- Only the top-left, top-right, center, bottom-left, and bottom-right sections have video parts playing in the modified jellyfish video.
- I created a randomGlueAbs to randomize the process of replacing the video parts.
Live video: jit.grab
To add more real-time elements to the video, I added a jit.grab object to get real-time video from my camera to the patcher. The contrast is also set to 0.5 so that it looks pale compared to the foreground MV.
Chromakeying
Now that the background is all set, I went on to modify the foreground. When tweaking the attributes of jit.chromakey, I discovered by chance that sometimes the dark content in the MV video gets cut out, too. This can create beautiful patterns like below:
Then, I saved the set of attribute values using a message. It will automatically load to the jit.chromakey upon the patcher opening.
During this step, I initially copy-pasted the chromakeying structure in the tutorial. However, I soon found that my modifications to the objects didn’t work. The online forum said that it may be because the original structure has its abstraction locked. Therefore, I created a new abstraction called “colorplaneRemap” and made modifications there, and they finally worked fine.
How to make background transitions among 3 videos?
Now I have 3 video sources for the background: the vintage-style video, the jellyfish video, and the live video from my camera. Next, I would like to allow manual background transitions among them. However, in the tutorial examples, there are only two videos. Therefore, I did calculations and found that the logic is just 3 piecewise functions.
After calculation, I implemented the functions using if objects.
At first, I thought I could include operators in the if statement. But I soon found that if objects cannot understand operators, meaning that I need to break down each function into several steps of calculations. It is interesting to think about how to use operators to achieve each function.
Smoothen the background transitions
I used a slider to control the background transition. However, if I jump from one value to another in the slider, the background will also change immediately without a smooth transition.
To solve that problem, I added a series of objects similar to the ones connecting with jit.scalebias for the MV foreground. With this structure, the background transition is always smooth.
Control the background with keyboard input
To make it easier to control the background, I implemented keyboard control over the slider.
Remaining Questions
- Is there a way to define jit.matrix dimension using a message?
- Why is the scale and bias transition from preset 1 to 3 (or the reverse) not smooth?
- I haven’t figured out the logic for color thresholding. I tried to implement it on the camera video but couldn’t figure out the logic. It is different from the tutorial example, in which every pixel’s value is based on its position. However, the color thresholding requires the current ARGB/grayscale color of a pixel. Nevertheless, I have included the unfinished rainbowAbs.maxpat in my folder, which was supposed to do the color thresholding.