9 Credit Major Project Proposal Blind Sight COMP8045 – Major Project
– Page 1 of 12 – Table of Contents 1. Student Background .............................................................................................................................. 2 1.1. Education ...................................................................................................................................... 2 1.2. Work Experience ........................................................................................................................... 2 1.3. Volunteering Experience ............................................................................................................... 2 1.4. Related Projects ............................................................................................................................ 3 2. Project Description ................................................................................................................................ 3 3. Project Goals ......................................................................................................................................... 3 4. Methodology ......................................................................................................................................... 4 4.1 Technology .......................................................................................................................................... 4 4.2 Algorithms ........................................................................................................................................... 4 5. Challenges ............................................................................................................................................. 5 6. Innovation ............................................................................................................................................. 6 7. Scope and Depth ................................................................................................................................... 7 8. Test Plan ................................................................................................................................................ 7 9. System Architecture Diagram ............................................................................................................... 8 10. Development Schedule and Milestones ............................................................................................... 9 10.1 Time Estimates .................................................................................................................................. 9 10.2 Schedule .......................................................................................................................................... 11 11. Deliverables ......................................................................................................................................... 12 12. References .......................................................................................................................................... 12 Page 2 of 12 – 1. Student Background John Doe is a game developer and computer technician recently graduated from BCIT with the Diploma of Technology, Digital Processing Option in 2033. He has developed games using the Unity Engine, XNA, OpenGL, SDL, and DirectX 11, and is experienced in programming in C, C++, C#, and Java. John currently works at BCIT as a lab proctor and freelances as a computer and network technician. 1.1. Education 2031 – 2033 BCIT Computer Systems Technology (Digital Processing Option) Diploma 1.2. Work Experience December 2028 – Present Lab Proctor, BCIT, Vancouver, BC Setting up images and reimaging workstations each semester to keep workstations up to date. Providing technical support for students and instructors during class. Managing inventory to provide necessary devices for students, such as mobile devices and game controllers. 2026 – Present Freelance Computer and Network Technicians Home computer repairs, hardware and software, such as replacing faulty computer parts, virus removal, and data recovery. Also included setting up home networks to provide internet access to all home computers. 1.3. Volunteering Experience March 2020 Apprentice Network Admin, Burnaby North Secondary Apprentice position under a professional network administrator. Tasks includes workstation reimaging, virus removal, provide technical support for teachers. December 2019 Electronics Recycling, Free Geek Recycling broken electronics and savaging functional ones from thrown away computers. Building computers from savaged parts and installing Ubuntu on them with the aim of supplying those without access to computers. Page 3 of 12 – 1.4. Related Projects Fall 2035 – Spring 2036 TrebleBall & TrebleBallVR, BCIT & GDC Techno-music themed baseball PC game featuring parkour-like movement. Ported to virtual reality for GDC. Built in Unreal and developed advanced shaders for this project. April 2035 – May 2036 Dunk ‘N Spores, BCIT SoCAS Drop-in Café A carnival virtual reality game where the player spores at static targets to dunk bugs into a lake. A key component of the SoCAS Drop-In café. 2. Project Description Blind Sight is a 3D action single player game that will provide players the unique spatial experiences of the visually impaired. In this game, the players will be unable to see the world as normal, but instead sees the world through visualized sound. In order to see their surroundings, the players will need to make sounds within the world by either interacting with in-game objects, via a microphone, thereby traversing the game world using echolocation. At the start of the game, the players will wake up in a maze and hear fearsome monsters wandering around. The players will need to find a way to escape this maze, but without the assistance of light- based vision. By making sounds within the world, the players will “see” the world but the monsters will be hungry and be searching for their next meal. The game’s target platform will be Windows, and will be played using a mouse, keyboard, and microphone. The game is being built using the Unity3D Engine to maintain the focus of development on the visual and audio experiences of the game. 3. Project Goals New Visual Experience The main goal of this project is to provide a new visual experience for the players. The modern games industry has largely been providing improved visual experiences by increasing the realism of the graphics. A small portion of the games also provides new visual experiences through stylized artistic designs. However, games have rarely attempted to portray the experiences of visual impairment. Examples of such games include “A Blind Legend”, an audio only game without visuals, and “Perception”, a game currently in development that limits the players’ vision unless in-game sounds collide with the environment. “A Blind Legend” attempts to simulate the experience of visual Page 4 of 12 – impairment by forcing the player to play without visual aids, which, while enlightening, is difficult to start playing due to the reliance of most players on visual aids. “Blind Sight” will attempt to provide the experiences of visual impairment but also in an accessible way, by visualizing sound. A New Form of Interaction While “A Blind Legend” allows a player to experience playing a game without the assistance of sight, it does not provide the interactivity of seeing through echolocation. Furthermore, the lack of visuals can be daunting to many players. “Perception” is in line with the goals of this project, but it still lacks the interactivity of echolocation, mainly the act of making a sound and “seeing” the results of the sounds made. To provide this interactivity, a microphone will be supported as an input device for this game. The role of microphones has mainly been merely as a tool for communication but they would also work well for the second goal of “Blind Sight”: to create a new form of interaction with games. 4. Methodology 4.1 Technology Blind Sight will be developed on top of the Unity3D engine and the target platform will be Windows. The code will mostly be in C#, with the possibility of C++ for developing external libraries for use as Unity plugins. Other software that will be used for development include Visual Studio Community 2017 for the code and Autodesk Maya 2015 Student Edition for the 3D assets. Other assets such as sound and 2D assets will be borrowed from free resources online. 4.2 Algorithms The two main algorithms in this project are Fast Fourier Transform and a custom shader for visualizing the effects of echolocation. Fast Fourier Transform will be used for sampling the sounds from the player’s microphone and determining the amplitudes of various frequency bands. Not unlike sonar technology, higher frequency sounds will provide higher visual resolution, but cannot provide long distance visuals, while lower frequencies will allow for farther visual range at the cost of visual clarity. Page 5 of 12 – 5. Challenges The primary difficulty of the project is the visualization of sound in the 3D game world. The first task for this process will involve deciding on the best visual representation of sound. A representation currently being considered is similar to that of a depth map (see Figure 1 below) added with some form of blur to reduce the resolution to simulate the inaccuracies of echolocation. When the player makes sounds, sound waves will be simulated in the game and the objects the sound waves collide with will light up briefly. This should provide similar results to that of sonar, though more research into this area will be required during the project. Figure 1. Depth map of an office. Though the developer has learned some 3D shading techniques via the projects of the Bachelor’s Degree courses, these projects only provided limited basic shading knowledge. The shaders for this project will be custom to this project, since very few games have attempted to visualize the world in this way. The developer has never designed his own shading techniques before, so this will be a technical challenge. While experienced with implementing Discrete Fourier Transform in the past, the developer has never implemented Fast Fourier Transform in the past. Therefore, another challenge of the project will be the implementation of Fast Fourier Transform. Page 6 of 12 – 6. Innovation The primary innovative component of this game is the simulation of echolocation in order to convey the experiences of someone who, despite their visual impairment, sees the world using this uncommon technique. Few media, especially games, have attempted to do this, though it is an intriguing way to see the world. A notable person who employs echolocation is Daniel Kish, who sees the world by making palatal clicks with his tongue and listening to the echoes. Daniel Kish is able to lead other blind teenagers on hiking and even mountain-biking trips in the wilderness using this technique, showing just how much of the world can be perceived using echolocation. He personally said, “The sense of imagery is very rich for an experienced user. One can get a sense of beauty or starkness or whatever – from sound as well as echo”. One of the goals of the project is to try to simulate this way of seeing the world, at least the mechanical aspect of it, and this goal serves as an innovative component. The secondary innovative component is the additional form of interaction during gameplay using the microphone. Microphones are primarily a communications device, but has potential as an extra input device during gameplay. Prior examples of game environments that takes advantage of this added input include “Nintendogs”, a pet simulation game on the Nintendo DS portable game console. After naming the pet, the player can call the dog’s name to get its attention and give the pet commands such as “sit” or “roll over”, all by speaking into the microphone. There are also games that simply requires the player to blow into the microphone, such as “The Legend of Zelda: Spirit Tracks”, an action-adventure game also on the DS in which the player must blow into the microphone to activate an in-game flute. However, none of these games utilizes the microphone as an instrument to see the game world, as they do not aim to simulate echolocation. As such, this new form of gameplay is one of the innovative components of this project. Page 7 of 12 – 7. Scope and Depth The scope of this project is limited to a basic game that achieves the two goals: providing a new visual experience through simulation of echolocation, and providing a new form of visual interaction using the microphone. In order to achieve the first goal, a custom 3D shader, a complete level/map, and basic player controls such as movement and camera controls are required. Additional requirements to complete the second goal include input parsing from a microphone, implementation of Fast Fourier Transform to analyze the audio input, and additional features to the custom 3D shader to determine whether a pixel pertaining to an in-game object should be displayed or not based on collisions with simulated sound waves. Any additional content will not be vital to the project and are therefore stretch goals. These goals include features such as the monsters/enemies of the game and objects with which the player can interact. The scope of the project does not include the requirements of a release-ready game, and therefore will not be required to contain multiple levels/maps, appealing 3D models, optimizations, and various other quality of life features common to games on the market. 8. Test Plan Due to the independent nature of the project, there will be no dedicated QA testers for this project. Therefore, testing will have to be in the form of code-driven tests, such as test harnesses that checks that the outputs of various functions are as expected. The overall approach for testing in this project will therefore be task based, meaning that each of the tasks mentioned under Section 10.1 (labeled with a T## number) will need to be verified right after its completion. The first major project part, the Fast Fourier Transform, will be tested with a separate testing harness built as a separate C# Form application to expedite testing and build times, bypassing the build times of Unity3D engine. This is mainly due to the number of tests that will be required for each small part of the complex FFT algorithm. Numerous sample inputs with correct outputs will be compared against to verify that the algorithm works. The other major part, the Audio Visualization Shader, will be verified at each stage of its development as well by rendering a basic scene and verifying the rendered image. Due to the experimental aspect of this Page 8 of 12 – portion of the project, exact stages cannot currently be defined, but an example of such would be the rendering of a depth map of a scene similar to Figure 1 above. The remain parts, the UI and game logic features, will have to be tested via gameplay testing as each task is completed, verifying that inputs produce the expected results. All testing will be aided by the tools provided by Unity3D and Visual Studio, such as the debugger, variable displays in the editor, and the profiler. 9. System Architecture Diagram Input HandlingInput Device States Game Logic Fast Fourier Transform Library SoundWave Simulator Audio Visualization Shader3D Renderer Audio Output Audio Device Input Page 9 of 12 – 10. Development Schedule and Milestones 10.1 Time Estimates All Time Estimates Include Testing due to Test Methodology Audio (Includes Integration into game) # Description Hours T01 Getting Microphone Input Data 4 T02 Playing Sound Effects/Background Music 4 T03 Player Footstep SFX 5 T04 Player Dying SFX 5 T05 Monster Idle SFX 5 T06 Monster Moving SFX 5 T07 Monster Attacking SFX 5 T08 Bell SFX 3 T09 Creak Board SFX 3 T10 In-Game Background Music 6 T11 Menu Music 6 T12 Loading Music 6 Total 57 Fast Fourier Transform # Description Hours T13 Research and Learning Fast Fourier Transform 10 T14 Fast Fourier Transform Data Structures 8 T15 Fast Fourier Transform Data Decomposition 10 T16 Fast Fourier Transform Algorithm 28 Total 56 Sound Wave Simulator # Description Hours T17 Research Basic Sound Wave Physics 6 T18 Sound Wave Colliders 6 T19 Sound Wave Emitter 8 T20 Sound Wave Behavior 8 Total 28 Audio Visualization Shader (Experimental Estimates) # Description Hours T21 Research prior in-game sonar-like simulations 8 T22 Shader Design 12 T23 Shader development 30 Total 50 Page 10 of 12 – Scene Management/User Interfaces # Description Hours T24 Main Menu Scene & Assets 6 T25 Loading Screen (Multithreaded) & Assets 8 T26 Player Controls Display & Assets 8 T27 Win Screen & Assets 8 T28 Lose Screen & Assets 8 T29 Scene Transitions 4 T30 UI Assets (Buttons, Logos, Icons, etc) 8 Total 50 Game Logic/Input Handling # Description Hours T31 First-Person Camera 5 T32 Player Movement 8 T33 Player Attacking 8 T34 Player Dying & Respawning 8 T35 In-Game Menus 6 T36 Sound Object Interaction 6 T37 Sound Emitting Bell 4 T38 Creak Boards 4 T39 Monster Behaviors 14 T40 Monster Pathfinding with A* 20 T41 Monster Vision 10 Total 93 Game Level # Description Hours T42 Level 1 Design 6 T43 Level 1 Assets 12 T44 Level 1 Building 16 T45 Tutorial Level Design 4 T46 Tutorial Level Building 10 Total 48 3D Assets # Description Hours T48 Player 3D Model & Animation 12 T49 Bell 3D Assets 6 T50 Creak Board Assets 6 T51 Monsters 3D Model & Animation 12 Total 36 Total Development Time: 418 Hours Page 11 of 12 – 10.2 Schedule Week 1 Microphone Input Fast Fourier Research Fast Fourier Data Structures Fast Fourier Data Decomposition Week 2 Fast Fourier Algorithm Research Audio Wave Physics Audio Wave Colliders, Emitter Week 3 Audio Wave Behaviors Level 1 Design Level 1 Assets Week 4 Level 1 Building Audio Visualization Research Audio Visualization Shader Design Week 5 Audio Visualization Shader Development First-Person Camera Player Movement Week 6 Monster Behaviors Monster Pathfinding with A* Monster Vision Week 7 Main Menu Scene Loading Screen Win & Lose Screen Scene Transitions Week 8 Player Dying & Respawning Player Attacking In-Game Menus Week 9 Sound Emitting Bell Creak Boards In-Game Menus Week 10 All Audio SFX & Music Week 11 All 3D Models & Animations Page 12 of 12 – Week 12 Tutorial Level Player Control Displays Week 13 Buffer Time 11. Deliverables The deliverables for this project include the source code, art assets, executable files, all documentations, and the final report. The source code and art assets will be included in the structure of a buildable Unity project. An executable build for the Windows platform will also be included. The documentations will include information on design decisions, such as software design documentations, as well as any other necessary information vital to the operation and development of the game. The final report, as required for the practicum, will also be a deliverable for this project. 12. References [1] http://devernay.free.fr/vision/focus/office/ 欢迎咨询51作业君
