Season2 Project: Scene Breaker

Background: VFX Design Once Wrong, Hundreds of Billions Disappear?!

A Blockbuster VFX Project Requiring Collaboration of 3000 People

With the increasing utilization of VFX in recent film production, it is not an exaggeration to say that the CG field is now receiving the most attention, surpassing the production stage, which had the largest capital investment. The highly successful <Avengers: Endgame> grossed 2.8 billion dollars, completing nearly 2500 VFX shots through collaborative work of over 3000 personnel at 14 VFX studios, with the majority of the production budget being allocated to VFX.

Wrongly Designed VFX Resources Following Soaring Production Costs

In the film industry, where the proportion of VFX is gradually increasing, the role of the VFX supervisor responsible for the visual aspects of the film is becoming increasingly significant in managing and overseeing this process.

Exploring more efficient production methods within limited VFX resources (capital, time, personnel) is a direct way to improve the quality of VFX and enhance the visual aspects of the film. To do so, the VFX supervisor must actively plan and design VFX from the pre-production stage, and consider how to utilize the limited VFX resources.

Due to erroneous designs, the cost of restoring poorly shot VFX shots is entirely borne by the companies involved. Incorrect VFX designs leading to soaring production costs also affect other processes.

The First Button for Efficient VFX Design, Scene Breakdown

During the production stage, the scenario and storyboard are analyzed scene by scene, and the elements to be filmed and produced in CG are determined through discussions between the VFX supervisor and other main steps. The VFX studio then conducts research and development on these elements, materializing the visuals imagined by the director.

Reference - Jung, Kim (2021) "A Study on the Role of VFX Supervisor for Efficient CG Production: Focused on the Film <Exit>" Journal of Visual Culture Contents 23 pp.117-142: 117.

Introducing Scene Breaker, an AI Assistant that breaks down 3000 scenes in 20 minutes!

Try Scene Breaker

Issues Discovered by Scene Breaker are as Follows

• During the pre-production stage, a significant amount of manpower and time is required to determine which parts of the movie require VFX effects.

• CG supervisors at VFX studios must conduct meetings to decide on VFX processing for each line of the script from beginning to end. This is why.

• During this process, the need for an AI assistant that can read the entire script, identify the parts requiring VFX processing, and make suggestions was discovered.

Key features of Scene Breaker

1. Simply upload the PDF script, and the AI will analyze the entire content.

2. For every scene in the script, the AI independently evaluates the necessity of VFX and suggests it with evidence.

3. You can receive the entire analysis results in a well-organized CSV file.

How to Use Scene Breaker

1. Prepare a PDF script containing the entire scenario.

2. Create a new project by clicking the + button on the left panel.

3. Enter the project name and description, then upload the prepared PDF script.

4. Click the Create Project button on the top right to start the script analysis using Scene Breaker.

5. Once the analysis process is complete, click the result.csv button at the bottom of the status bar to download the result file.

Development Process

1. Enhancing Understanding of the Scene Breakdown Process

• The idea for Scene Breaker was proposed by JungBTC, who worked as a VFX coordinator in the film industry. However, except for JungBTC, other team members lacked background knowledge of the film production process, so it was necessary to initially understand this process.

• JungBTC held a small seminar on VFX production process to raise the understanding of the team members.

Seminar materials on VFX production process

2. Finding the Best Prompt for Optimal Results

• We needed to find prompts that could produce the following results:

  • Analyze each sentence (or dialogue) in the script to determine if VFX is needed

  • Produce results that closely resemble the judgments of actual VFX supervisors

  • Include logical reasoning for each judgment

  • Judge the necessity of VFX in a way that is suitable for the production team's composition

  • Consistently produce responses in a uniform format

• However, it was very difficult to find a prompt that met all of these conditions.

• In particular, it was very difficult to "div... hottest tag...").

  • ChatGPT was not well-versed in understanding "scenes" and was very vulnerable in showing a "consistent format" of results.

  • Furthermore, even with the same prompt, we found that using the ChatGPT website and using the OpenAPI resulted in very different outcomes.

  • Most of the development period was invested in prompt research to find the best prompt.

• Recognizing that using ChatGPT website and the API produced different results with the same prompt, we decided to directly set up the environment required for prompt research and use it.

Environment for prompt testing set up by Ryan

• During this process, we ended up using the Completion mode of the OpenAPI, and you can see some of this research process on the page below.

  
  

  Go to Prompt Research Process

3. Making Final UX Decisions and Frontend Development

• Since the results of the prompt research were continuously being updated, flexible UX/UI changes were required to match this process.

• We implemented the website with a modularized UI structure to handle the development situation flexibly.

4. Testing the Service

• Since this service can directly address issues experienced in the field, we would like to meet with VFX supervisors in the field and request them to test the service.

• Those of you reading this can try the service yourselves! Feel free to use it and share your diverse opinions :)

What We Learned from the Project

• It was great to be able to clearly define valid problems based on the expertise of JungBTC in the field and attempt to solve them with AI integration.

• When needing to split and request a large amount of text due to token limitations for API calls, the responses did not bring back the previous context, resulting in different formats and qualities of each response.

• It seemed that the quality of the response obtained by requesting through the API was lower compared to the ChatGPT website, and the reason for this was unclear (although this issue was raised in the community, it is still uncertain).

• It would be ideal to achieve the following additional requirements:

  • Indicate which team should handle the VFX effect.

  • Provide information on how much budget should be allocated for the CG processing.

  • Provide a 3-line summary of each scene to facilitate understanding of the script.

Feedback from Team Members

• Jerry (PM)

  • It was satisfying to work with great people and attempt to solve valid problems that exist in the field by integrating AI.

• Ryan (Developer)

  • It was great to build a service from end-to-end with new people. It was also a time of significant learning as I tried to use as many new tools and frameworks as possible.

• JungBTC (Developer)

  • It was great to be able to create a product in demand! However, I regret that due to my ambition, the project progress was slower than expected because I couldn't compromise on the service quality.

• Chan (Designer)

  • It seems that my understanding of generative AI has definitely increased through this project. However, I still feel the limitations of generative AI quite a bit. It is still difficult to handle detailed responses, and I feel the level of detail is unsatisfactory, making it still challenging for practical use in the industry.