Playing Along with Video Games: Investigating the Role of Procedural Music
AbstractHave you ever wondered about the various types of music in a video game you've played? You may not have paid much attention to the music, but its job was to enhance your gaming experience. In fact, the wrong kind of music can detract from the atmosphere of the game. Can you imagine the music in Mario KartTM playing in Street Fighter®? In a game, music can indicate many different things, such as a special or new event, shift of mood, or the arrival of a character. This kind of music is called procedural music, and it changes according to what happens in the game. In this video and computer game, you will design a game that uses procedural music to enhance the player's gaming experience.
Michelle Maranowski, PhD, Science Buddies
Use music to indicate a change of event, mood, or character presence in a video game.
Music can make us feel a variety of emotions. For example, when you hear scary music in a movie, you know that something frightening may happen. As the tempo of the scary music increases, you start to feel more and more excited until finally something happens. A similar musical process can be used in video games, where music creates atmosphere. In addition, music can signal to the player a change in event or experience level, or the arrival of a new character. This type of music is called procedural music or procedural audio. Procedural music or audio occurs when certain events within the game cause the music to evolve in real time in some form. Unlike the synthesized music and sound effects in early video games like the original Pac-ManTM and Mario Bros.TM from the 1980s, the procedural music in games of the 21st century can switch from one tune to another, the tempo can vary, or different tunes can be layered on top of each other. This gives the player a dynamic experience, where the music will always fit the gaming atmosphere. An example of procedural music occurs in Super Mario Galaxy 2TM, when Mario grabs the grand star and special music signals this event, giving the player a sense of accomplishment.
Procedural music in video games is created algorithmically. This means that there are rules and logic that dictate how music is generated. The procedural music in video games can be divided into two categories: interactive and adaptive. Interactive music refers to music events that are directly triggered by the player or the input device, like footsteps when the player moves or shooting sounds when the player fires a gun. The player has control of the timing, although the game's program or engine controls how the music is played back. Adaptive music refers to musical events that are affected indirectly by the player. Adaptive music is cued or controlled by the game's engine and is usually not immediately repeatable.
Now that we know the different ways music can be triggered in a video game, the next question is how the programming in the game (the game's engine or brains) actually creates the music. Procedural music can be generated using either transformational algorithms or generative algorithms. Transformational algorithms don't affect the data size (the amount of storage space the music takes in memory), but they do affect the structure of the music. The pitch of certain notes can be varied, and instrumental parts can be added or dropped within a phrase of the overall song. (A musical phrase is a group of lines of music that has its own musical theme or logic.) Generative algorithms increase the data size because the music is created rather than just modified during game play.
In this video and computer games science project, you will create a video game with procedural music, using a programming language of your choice.
Terms and Concepts
- Procedural music
- Procedural audio
- Interactive music
- Adaptive music
- Transformational algorithms
- Generative algorithms
- Musical phrase
- Conditional statements
- Flow chart
- In a review of video games that you have played, how do the games use music or some type of audio to add to the atmosphere?
- Can you find some examples of procedural music or audio in the video games that you play?
- What is the difference between interactive and adaptive music?
- How do transformative algorithms differ from generative algorithms?
The following website lists several interesting articles on procedural audio:
- Fournel, N. (2010, January 1). Procedural Audio. Retrieved October 26, 2011.
This YouTube video demonstrates procedural audio:
- lykkesM. (2009, November 16). Procedural audio in computer games (English). Retrieved October 26, 2011.
Materials and Equipment
- Computer with Internet connection
- Programming language of your choice. See the Kid-Friendly Programming Languages and Resources page for ideas.
- Optional: Audacity or other sound and audio recording software; you can download Audacity free of charge from https://www.audacityteam.org/
Planning Your Game
- In this video and computer games science project, you will create a simple game that uses procedural music to indicate changes happening during the game and to enhance the player's experience.
- First, you need to decide what programming language you will use. The Kid-Friendly Programming Languages and Resources page lists several options.
- Before you start programming your game, do some tutorials to get familiar with your chosen programming language.
- As noted at the beginning of this procedure, this project follows the Engineering Design Process. Remember, if you run into trouble making your game, or if you feel you want more practice before starting this project, the home page for your programming language probably contains many tutorials, help documents, and forums where you can ask questions.
- Define the problem. In this case, you will create a fun video game that uses procedural music to enhance the gaming experience for the player. Refer to the Science Buddies Define the Problem page to help you set the boundaries for the project.
- Do background research. Read the references in the bibliography to develop an understanding of procedural music. You should also start thinking about the goals of building a successful video game.
- Develop the project requirements. The project requirements are the characteristics that your video game must have to be a successful video game. Refer to the Science Buddies Specify Requirements section for tips on how to formulate your game's design requirements. Here are some ideas to consider when formulating the requirements:
- What kind of game do you want to play? If you find the idea of the game interesting, chances are other people will, too. Do you want the game to be a maze or a game where the player picks things up?
- How much time you can spend on writing the game. If you only have two weeks, you need to design a game that can be written in a week to leave time for testing it. Simple games can be fun to play, too.
- What kinds of sprites (that is, images or animation) will you use? Where will you get them?
- Where in the game should the music change? Will the music change as an indication of a reward, when the player does something right? For example, will there be a trumpet flourish when a goal is reached? Will the pace of the music change as the player advances through the game, getting louder and faster? Will the music change depending on the position of the object?
- How many rooms or areas will your game have? Will the music change depending on the room or area that the player is in?
- What kinds of music will you use and where will you get the music? Do you plan to use built-in music files that came with your programming language, or will you put together your own music or sounds? If you plan to make your own sounds, you can use audio editing and recording software like Audacity, available for download from https://www.audacityteam.org/.
- How long should the game be and how will it end?
Building Your Game
- Create and analyze solutions. Keeping your project requirements in mind, think about different ways that you could build your game. Take a look at the Science Buddies document Create Alternative Solutions to guide your efforts. Once you have developed a few solutions, analyze the solutions by making rough sketches and flow charts for each one. Refer to the Science Buddies Choose the Best Solution document to help you pick a working solution.
- Build and test a sample video game. Once you have created a set of requirements and a possible solution, it is time to open your programming environment and start working on building a sample video game. Build character and have them operate in a simple version of your game. Remember to review your requirements so that you keep yourself focused on the task. Review the Science Buddies Prototyping document.
- Program your video game. Keep testing the game as you work. When you have fulfilled a requirement or task, run the game and test it out.
- Break the programming for the game up into smaller tasks so that the project is not overwhelming.
- Test the game along the way so that you can fix small issues as they come up. This will prevent your having a long set of events at the end that don't work.
- Once you have finished your game, check to see that all of the project requirements are fulfilled.
- Test and redesign. Review the Science Buddies Test and Redesign document to help organize your work. Test your game out on your family, your friends, and yourself. Take notes on what your players enjoyed and didn't enjoy about the game. Use the feedback to improve your game.
The Final Product: Presenting Your Game
- When presenting your game at your science fair, try to bring in a computer. If you are not able to do so, take screenshots of your work, print them out, and mount them to a poster board. If you need help taking screenshots, ask a teacher or someone else familiar with the computer for help.
- You should include the following items in your presentation:
- A list of your project requirements that guided your building of the video game.
- The rough sketches or flow chart that describes how the game works.
- An explanation of what you learned from your research and from creating the video game and its procedural music.
- If you would like to publish your game for a wider audience to play, check the home page for your programming language. Many of them have ways for you to upload and share your game online.
Ask an Expert
- Is procedural music really necessary? Test a game sound-free and a full version with music on a group of volunteers. Measure the players' satisfaction and interest in playing each game. Is there a difference? Because you will be testing on a group of volunteers, make sure to get permission from their parents or guardians if they are minors.
Working with Human Test Subjects
There are special considerations when designing an experiment involving human subjects. Fairs affiliated with Intel International Science and Engineering Fair (ISEF) often require an Informed Consent Form (permission sheet) for every participant who is questioned. Consult the rules and regulations of the science fair that you are entering, prior to performing experiments or surveys. Please refer to the Science Buddies documents Projects Involving Human Subjects and Scientific Review Committee for additional important requirements. If you are working with minors, you must get advance permission from the children's parents or guardians (and teachers if you are performing the test while they are in school) to make sure that it is all right for the children to participate in the science fair project. Here are suggested guidelines for obtaining permission for working with minors:
- Write a clear description of your science fair project, what you are studying, and what you hope to learn. Include how the child will be tested. Include a paragraph where you get a parent's or guardian's and/or teacher's signature.
- Print out as many copies as you need for each child you will be surveying.
- Pass out the permission sheet to the children or to the teachers of the children to give to the parents. You must have permission for all the children in order to be able to use them as test subjects.
If you like this project, you might enjoy exploring these related careers: