Project Wonder
For this project, we must create an abstract interpretation of the human form using motion capture data we collect from the studio. This can be as abstract as we want, where we can do any type of movement, from dance and theoretical performance to combat. This blog will showcase the progress of this journey.
Research Stage
Before starting the project, the first step is to research what type of motion capture to help me with inspiration and ideas. It is also essential to investigate how movement can be captured/interpreted into motion capture and other aspects such as film, animation, etc., as it will give a broader understanding of the subject.
Research #1
The first example that caught my eye was from a music video called Official 2016 AICP Sponsor Reel Director's Cut by Method Studios. This video is an excellent way of showcasing how movement can be captured with different effects. An effect that caught my eye was at the 1:46 mark, which used ribbons to show off the movement within the dance was an interesting interpretation. Also, the combination of making the ribbons very colourful and how the ribbons bonces and move with the movement makes it very playful. Here is a screenshot of the section.
Here is the original video
Research #2
The second research is also from the same video as the first one; however, I chose this one for the effect and how the effect interpreted movement. At this stage, I had an idea to create something with the lava lamp effect, as I thought I could turn it into a fun performance instead of just a dance. And this effect at 2:36, this effect uses balls to create the human form. This was another enjoyable way to interpret the human form. During the whole section, the balls changed colours and moved with the movements and music. This example is similar to creating a lava lamp effect to create an abstract human form.
Here is the original video
Research #3
The third is slightly different from the previous two motion capture examples. However, this is an example of how motion can be interpreted from an animated movie Fantasia 2000, especially the section Firebird. I love how movement is interpreted in this film, especially the spring spirit. This is also a great example of how theatrical performance can be used to showcase movement. The spring spirit uses many over-extracted moves to show the audience. Researching and breaking down this section also inspired another idea from my project of using the combination of nature, mainly flower petals and water, to showcase movement within a performance. Both Fantasia films are also great examples of how music and performance can be used abstractly to tell the audience a story without words.
Here are some photos from the section.
Here is the original video where the photos are from:
Research #4
For the fourth research, I was looking into a music video for a rock band called Home and Drys. Where for this video, they used a mixture of motion capture and 2d motion vector and effects combined with 3d lighting to create this abstract form of the human form. Throughout the video, the form gets more and more discord and abstract. At the end of the video, the form is mainly just geometric shapes. I chose this video as an example to showcase the use of paint effects and colour to create interesting abstract forms with the human form. Here are some screenshots from the video to show examples:
Here is the original music video:
Research #5
For the final part of research into abstract movement, the last example is the most abstract out of the 5 examples. For the research section, I wanted to get a variety of styles and types of motion movement. As a result, this final one solely uses motion capture to create abstract movements, unlike the rest, which use effects to create an abstract form of the human form. For this example, they use a particle effect combined with motion capture data to create an abstract interpretation of movement. Through movement, the particles change colours.
Here are some photos from the video:
Here is the original video:
MoodBoard Stage
Once I completed the reach stage of the project, I moved on to creating a mood board to help visualise my project's main ideas, themes, moods, and aesthetics. The two ideas I have at this stage are the lava lamp/paint neon light idea and the nature/petals and water effect. Both ideas are very different. However, both use performance to tell the story. The mood/tone for both ideas is also the same, and I want to create something fun and joyful.
Board #1
This board is for the first idea of using a mixture of a lava lamp and paint effect to create abstract forms of the human form. As for the colours, I plan to have them bright with glow effects. As a result, the background and room with be black to make the effects the main focus.
Board #2
The second board is for my second idea of using nature/petal effects to create human forms and showcase movement. This idea is much more performance-based than the first idea. For the background, I planned to have the ground be water to create another interesting element, especially with how petals create with water.
Board #3
The third board is inspiration/ideas for the background and lighting ideas for the project. The plan is to have a water floor reflection effect for both ideas. The first idea was planning to have black as the background so the effects are focused more than the background. The second idea also planned to have a simple background of the blue sky, with the water floor effect.
Story Board
I want to go with a simple approach to the idea for the storyboard, where the main focus is the performer and the effects. As a result, the beginning starts with a zoom-in into a particle effect of flower petals blowing in the wind. Then when the camera zooms back out into a wide shot, the flowers turn into the main character. From there, the idea is the main character exploring the new world they woke up to and trying to play and follow the wind of flowers. I wanted to play around with the shot angles and placement throughout the storyboard. To show the movement within the performance. Some of the shots I have are front shots and bird's eye view shots. Wide and medium-range shots etc. Also, plan to try and make the camera feel like it's moving with the wind and the performer. For the story's ending, I wanted to have the main character trying to touch the wind but getting scared, and in the end, the main character turns into a flower and blows away with the wind.
Mocap Session
The next step was to book the mocap lab to capture our data. For this, I was in a group of three. The session is for about 2 hours long.
Cortex data clean up
Once we have the mocap data, the next step is to book the cortex captures to begin the post-production process of cleaning up the mocap data and then re-targeting the data to the chosen mesh. For the cortex stage, this process went well; the main problem in the data cleanup was in the hands and feet markers. There was a lot of missing marker or markers swapping. These problems were fixable, just a bit more time-consuming, as these happened multiple times throughout the data. However, that was the only main issue.
Retargeting
Once the data has been cleaned up, the next step is to re-target the data to the character mesh. As for my project, I plan to use the mesh; I need to retarget the data onto the chosen mesh. To re targeted the data, we used the program Motion Builder. First, you import the data into the program, create an actor and connect the data to the actor; from there, you merge the related data to the chosen character mesh.
Animating Feet and Hands
Once the data has been retargeted onto the mesh, the last step before importing it into Maya. It is to add animation on the hands and feet, as the hands and feet have no movement or animation, as there are no markers on these points during recording at the motion capture studio. As a result, you have to add animation by hand. For this step, you can either use motion builder or Maya. I prefer to do this step-in motion builder. I add slight movements in the hands and feet to make the movements more elegant, instead of the stiff hand and feet the original data has. Once done, I send the data to Maya and start the next process, using the time editor to edit the animations.
Experimenting with effects in Maya
BiFrost liquid
I experimented with Bifrost Fluids in Maya for the water floor idea to create the water floor effect. However, through experiments, this method did not work very well. I created a container for the water to sit in to make the water floor; however, when I tried to see what the water looks like inside Arnold'sstory's render view, only the first few frames worked, and then the effect stopped working and was not rendering. The water effect was also too rough feeling even after I played around with the effect's settings.
Fluids
The second method I experimented with was the fluids in Maya, mainly the pond effect. This did create a nice water floor effect; however, the main issue with this method compared to Bifrost fluids is that the water of the pond effect doesn't get affected by the movement of the animation. As a result, the water has no movement and is stagnant.
Mirror Floor idea
After experimenting to try and get a water effect, I try a different approach. The next idea was to have a mirror-reflected surface instead of a water floor. For this, I modelled a narrow rectangle; then UV unwrapped it and went into substance painter to create a reflective material. Once I was happy with the material, I exported the materials and placed them into the final scene. Maya file added a Skydome for simple lighting a view this Arnold. I was pleased with the results of the mirror-reflective surface combined with the sky dome. It looks very similar to the water effect I originally wanted, without the ripple effect.
nParticles
For creating the particles, I did a lot of experimenting to get the right effect. First, I experimented with nParticles however, when I changed the particle shape with the petals through Instunce, it did not work very well, the particles were moving jumpy, and when I went into the render view, they were not rendering correctly.
MASH
Next, I experimented with MASH in Maya. These trials were more successful than the first method. However, after playing around with the settings and experimenting with the different nodes, there are within MESH. I didn't like how the petals looked on the Mesh. So, I started experimenting with different shapes and seeing what looked appealing and what didn't. In the end, I liked how spheres looked on the model, plus with the emission on, the spheres looked similar to stars.
Change of Direction
Povit Idea
After experimenting with Maya, I decided to change the concept slightly. The main aspects I'm changing are the idea's effects and aesthetics. The storyline and emotion within the project are the same. I am planning to change the flowers to stars instead, as through the experimenting, I wasn't happy with how the flower effects were turning out, so I pivoted the idea to a galaxy night sky theme instead of a floral daytime sky theme. Here is the new mood board for the change of aesthetics/theme.
Time Editor
During the time editor stage, I had many problems. Where when I tried to create the clip from the mocap data. The clip would either not be created, or there would be no keyframes within the clips. As a result, I was not able to edit the mocap animation. This problem was fixed with the help of the lecturer. The only way to fix the problem was to merge all the animation layers and then make the animation onto the skeleton; after doing this, we retried to create the clip, and at first, it still wasn't working. However, after a few tries, we got it to work. Once the problem was fixed, I started editing the clips and experimenting with the animations I had from the mocap data. I edited the clips and then exported them as clips to import them into the science I had created for the final render.
Creating Final Background
For the background, I went very simple. For the lighting, I used the Arnold Physical Sky and played around with the settings in the colour section, where I experimented with the sun placement, tone, etc., to create a night sky effect. I used the mirror floor for the floor, which I created through substance paint. To make the stars in the night sky. I did a lot of experimenting and found the MESH tool was the most effective method. Where I created a sphere, added the aistandardsurface material added the emission settings. Next, I made a plane, placed it where I wanted the starts then I selected the sphere, created a MESH, then changed the distribution type from liner to Mesh, and dragged the pane into the input Mesh to make it the Mesh instead the amount to hundred and hide the original plane, and this created an interesting starry sky and created to different levels to add more depth.
Creating Final Effects
When creating the final effects, I first tried creating an emitter inside the Mesh as the idea was to have stars inside the Mesh moving as the character moves with the elegant movements. To create an emitter inside the Mesh, I first needed to create a sphere and parent to one of the bones in the skeleton. Next, I selected the sphere and created an emitter from an object. This made the sphere the emitter. This method worked. Next, I made the Mesh a passive collider through the nCloth tool. However, this is when things suddenly stop working, the particles stop appearing, and even when I checked the render view, the particles were not there. I restarted and did everything again, but this did not work.
As a result, I went with the second method. I have been experimenting with MASH, where I created a sphere with emission effects; however, I created a material from Substance Painter for the final. Once I created a MASH, I added a signal node to create movement within the Mesh. Next, I still wanted to have a type of trail to show movement. I ended up creating two emitters in the hands of the model, using the same technique of creating an emitter in the Mesh. I played around with the settings until I was happy with the result of having a pixel-like trail effect connecting to the star/galaxy theme.
Final Test Render
Once happy with the effects, I do a test render for one frame to see how everything looks once rendered through Arnold. After the first render test, I changed some settings for the particles emitting from the hands as initially, it was spheres; however, after rendering the frame, the spheres felt too heavy for the emotion I wanted to portray through the movement. As a result, I change it to sprites. I also changed where the stars are in the background, where I bring them further down, as it felt like they were too far away in the shot. Then I do another render, and once I am happy, I create the cameras.
Ending Effects
For the ending, I have the spheres inside the mesh going large with the timing of the last large gesture in the performance. To do this, I went into the signal node in MASH, increased the scale, and then keyframed it. This results in the sphere slowly going larger at that keyframe. Then I moved on to the next effect for the ending, which was that after the sphere grew bigger, they slowly started disappearing until there were no more spheres. To this effect, I went into the distribution node in the Mash, and the first key framed the normal amount in time with the big gesture. Then I moved to the last keyframe and made the distribution zero, then keyframed it. As a result, the sphere slowly disappears after the big gesture.
Cameras Stage
For the cameras, I experimented with how the camera can be used to portray movement and make the viewer feel the movement through the camera and the visual movement. To achieve this idea, I made the camera move with the movement, where the camera rarely stayed in one position. I also experimented with the focus lens of the camera, where I changed the focus lenses to between 30, 35, and 40.
Music Credits
Making of video music:
Wet Weather by SergeQuadrado:
Chetvertnykh, S. (2021, December 26). Download free media from SergeQuadrado | Pixabay. Pixabay. https://pixabay.com/users/sergequadrado-24990007/
Wonder Music:
Intergalactic Communication Space Music by JuliusH
H, J. (2016, December 4). Download free media from JuliusH | Pixabay. Pixabay. https://pixabay.com/users/juliush-3921568/
Comments