IMA Documentation

For this weeks assignment – our midterm – I built off the idea of what I created last week. I wanted to create an experience that would mimic a farm dog herding/leading its clan of chickens. Of course, during the process of creating this app, I ran into various challenges.

First, I had difficulty adjusting the constraint layouts of the buttons. On Xcode, I wanted the buttons to be laid out this way (shown below) to allow the user to have a larger screen space for the animals. But for some reason, I could not figure out how to get the constraints to be organized in the way shown on the screenshot below. I would change the constraints of each individual button, but it would always end up becoming even more messy and disorganized so I left it in the plane detect storyboard format that was created during class. This layout of buttons still worked though, as there was still enough space to see the animals. In the future, I will need to go back an adjust the constraints to allow for maximum screen space. 

 

I also ran into a problem where I could not get the chickens to be added on as a child. After speaking to Nien, we figured out that the problem was because I set the default scale of the entities in Reality Composer as something very small, so that when I pressed “add a chick”, there was a chick being added but it was so small that you couldn’t see it by eye. I had originally set the scale small because in the past when I imported models into RC and put them into the AR app, the scale/size of the entity in real time was huge (as seen for my sheep in my previous week’s assignment). However, I realize now this is something you can and should alter through code via Xcode, not through RC scalability. 

In addition, I tried to include the alteration of scalability of the entities that were added. I tried to write code that would create a relationship between the slider and the size of the chicks (so it would mimic if the chick added were a baby or an adult) but the slider didn’t end up working. I’m not sure if it were the code I wrote that had incorrect logic or if the relationship created on Xcode was faulty.

Here is the link to the screen recording!

In the future, it would have been fun to experiment with having the entities jiggle or wobble back and forth in real time to resemble “walking” when the user pressed on the “move to cursor” or “move right” buttons . I didn’t know if this were possible/would cause a crash because I knew you could trigger a notification that would cause an entity to jiggle through RC functions, but I didn’t know how to link up two actions – with one being on the Swift side and the other being on the RC side. When I tried writing the code for it and wiring it up, the build failed or just caused my screen to stay frozen. 

This week we were given the task to create an interactive AR app. Using Xcode’s storyboard, I attempted to design a very basic user interface. I continued on from my previous week’s farm animal theme and wanted to create an interface where the user could add more animals to the farm – particularly sheep and chickens. Initially, I wanted to have a page swipe component so that the user could add sheep to one page and chicken to another page, but it seemed really complicated and I just ended up going into the deep web of stack overflow and confusing myself more. So, instead, I just put two buttons that call the entity of the animals. However, as I was laying out the components, I found a lot of difficulty trying to break the links created from the storyboard to code and assemble the constraints so that what is seen on the storyboard is reflected on my device. 

*old version

 

Here’s the link to the video!

This week we were given the task to create 2 programs – Fibonacci and Lyrics – on Swift. Below are the screenshots of the programs and the app interface when you tap on the screen.

The second part is the creation of our own One Button AR app. For this portion, I wanted to continue my farm animal streak from the weeks before. I imported a grass field mesh and a bunny mesh into Reality Composer.  I added the grass field and the bunnies to the origin anchor, so when you enter AR this view would pose as the world.

For user interactivity, each tap would cause a carrot to appear, as if you were feeding the bunny. I wanted to try and make the bunny to scale up in size to indicate that the bunny “ate” the carrot, but I was having trouble figuring out the code for it. So that would be the next step for this project. Also, it would be really cool if I could target the bunny that the carrot drops on scale up and have the carrot be removed after a set period of time. 

Here’s the link!

Model Credit: Grass: Somersby, Bunny: yellowkab

This week,  I had a lot of troubles getting Xcode to work with my RC project, and displaying the model in AR on my phone. Building off my previous model from last week – the pig. I imported the pig into Substance Painter, where I found the mesh of the model was a bit confusing to work with, but eventually got the hang of it. I added a pink calf skin color as a base color, then added some leaves (b/c fall is upon us) and darker splatter paint on the bottom to resemble mud. 

Next, I put the model into RC and created 3 notification triggers to be accessed in Xcode. This is where I found the majority of my troubles to be. Initially, the pig model wouldn’t show on my phone, but the triggers were able to be seen. In particular, the third trigger of sound didn’t work and I wasn’t sure how to make it sound on my phone. I will need to go back and try to fix this problem. But for now, I was able to get the Flip and Spin triggers to work. 

Here is a link to the screen recording! Xcode was kind of confusing to me, but more on the side of all the weird functionalities and getting things to work. 

For this week’s assignments, we were given the task to create 3 custom 3D models and put them into Reality Composer. Starting off, I had a really difficult time figuring my way around and creating models in Blender. There was just a lot going on everywhere and there were so many hot keys to remember, even after watching the introduction tutorial (multiple times).

I decided to build off my idea from last week – starting off with creating a sheep! To get some inspiration, I literally just searched “beginners tutorial blender sheep” and found this one tutorial to follow. I’m giving credit to Grant here because I followed his model pretty closely. However, after modeling the sheep and adding color via the shading tab, I soon ran into a problem that only affected my sheep model. As I was exporting the model (to glb and fdx format) and putting it into Reality Converter, some parts of my model did not get exported! You can see this in my final AR experience – the main head part and eyes weren’t able to transfer over so the sheep just looks a bit scary. The first screenshot shows what my sheep is supposed to look like, but then when I put it into Reality converter to be converted into usdz format, it lost some key components. I even tried to delete the original eyes and head portion and replace it with a new duplicate, but for some reason it just wouldn’t hold. I’m not sure why this happened for my sheep because for my other farm animals, everything was fine! I was able to export the model as a glb file and convert it via Reality Converter to a usdz so that it could be read in Reality Composer.

Now, onto my next two models. Recently, my brother has been raving about his Minecraft worlds and so I was inspired to create Minecraft-like models so I could brag to him about MY models. For my second model (pig), I ran into another inconvenience! I didn’t center the body portions of the pig nicely/parallel to the anchor, so I had to do extra alignment work so that the head portions and leg lined up nicely. You can see in the second screenshot below how the pig is slightly sideways. Other screenshots show the model and color. 

Finally, I made my chicken. However, the coloring in Blender turned out different when I converted it to usdz so I had to go back and redo the color shades multiple times before I was satisfied.

Now, it was time to put my models into Reality Composer. I wanted to create a farm animal experience so, whenever you tap on the animal, its associated sound would play. I chose a horizontal anchor. Below is a screenshot of what the experience looks like in RC. 

 Here is the link to the screen recording.

In the future, I would have loved to develop the models a bit further but I didn’t feel like I had gotten the hang of Blender quite enough yet. I would have loved to create a swirly pig tail for my pig and even looked online for ways to do it, but there wasn’t too much information on it and if there was, it was in the older Blender versions. It also would have been really cool to add some flappy wing animations for my chicken. Even though the process was a bit overwhelming at first, I’d say this was a good start and am looking forward to really getting to know the ins and outs of Blender for future models.