Abstract
How fast are computers? How quickly do you think they can do 100 million arithmetic problems? Try this science project to discover an easy way to measure how fast computers have become and whether or not your computer is a math whiz.
Important: this project was originally written in the mid-2000's, and computers have gotten a lot faster since then! The project will probably not work as intended if you run the code on a modern computer.
Summary
Andrew Olson, Ph.D., Science Buddies
Sandra Slutz, Ph.D., Science Buddies
Updated by Howard Eglowstein, Science Buddies
- Excel® and Windows® are registered trademarks of Microsoft Corporation
- Linux® is the registered trademark of Linus Torvalds
- OS X is a registered trademark of Apple Inc.
- POSIX is a registered trademark of the IEEE
- Python is a registered trademark of the Python Software Foundation
- Raspberry Pi is a registered trademark of the Raspberry Pi Foundation
Objective
Measure how long it takes a computer to perform mathematical operations (addition, subtraction, multiplication, and division) with different types of numerical data.
Introduction
Have you ever sat down at an old computer, turned it on, tried to open a program, and found yourself waiting... and waiting... and waiting... until finally the program opens? It probably does not even take that long! It takes maybe a couple of seconds or a minute, certainly not long enough to even eat an ice cream cone or listen to a new song, yet it feels like forever. Computer technology continues to advance at an amazing pace, and today's computers run a lot faster than computers from 10 or even 5 years ago. We have gotten used to the newer, faster computers, and it is hard to go back.
How much faster are newer computers? Is your computer faster than your friend's? Is there a way to measure how fast a computer is? The answer is yes! Just like a sprint can tell you who is the fastest short-distance runner, a mathematical computer race can show you which computer is the fastest. In fact, manufacturers race computers all the time to test their computers. They call this testing benchmarking. You can test the speed of a computer, but before you do, it is a good idea to know a little more about how a computer works and what it takes to make one faster.
Computers understand what you want to do when you, the user, do something through clicks of the mouse, taps on the keyboard, or some less common input, like writing on a tablet, issuing voice commands, or using a joystick. These instructions, called inputs, tell the software (computer program) to do something. The exact instructions were written by a programmer and processed (understood and acted on) by the central processing unit (CPU) which is part of the computer's hardware. The CPU, the "brain" of the computer, does the actual tasks and then stores the results in the computer's memory. For example, the software on your computer changes the task from something you understand in words, like "color this square green," to a logical or mathematical operation for the CPU to perform, like "increase the color by 50 units and store the new color value in the memory." Increasing the processing speed of the CPU or the amount of available memory are two common ways to make a computer faster.
Modern computers can also multitask, meaning that the operating system (the software that runs the basics operations of the computer) splits the CPU's attention between many programs running at once. Just like you can eat a cookie, talk to a friend, reply to a text message, listen to music, and window shop all at the same time, a computer can do several things at the same time. The more things you ask it to do, however, the slower it might seem.
In this science project, you will use a benchmarking technique for timing operations to measure how long it takes a computer to perform four different mathematical operations: addition, subtraction, multiplication, and division. You will test the speed in each of these operations for four different ways of storing numbers, called data types: 8, 16, 32, and 64 bit integers, floating-point numbers, and double-precision floating-point numbers. The integer data types only store integers (counting numbers such as 1, 2, 3; no fractions) and can be either positive (+) or negative (-). The difference is that the integers with more bits can store larger numbers but they take up more memory. Floating-point numbers and double-precision floating-point numbers are both data types that store non-integers, such as numbers with decimals. Both of these data types can also store positive numbers (like 1.55) or negative numbers (like -1.55). The floating-point data type has a smaller range of numbers because it only uses 32 bits of memory. Because the double-precision floating point number data type uses 64 bits of memory, it has a larger range. Table 1 gives examples of all four data types.
| Data Type | Type of Number | Amount of Memory Used | Range of Values That Can Be Stored |
|---|---|---|---|
| 8-bit Integer | Integer | 8 bits | -128 to 127 |
| 16-bit Integer | Integer | 16 bits | -32768 to 32767 |
| 32-bit Integer | Integer | 32 bits | -2,147,483,648 to 2,147,483,647 |
| Long (64-bit) integer | Integer | 64 bits | -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 |
| Floating point | Non-integer | 32 bits | ±1.4E-45 to ±3.4028235E+38 |
| Double-precision floating point | Non-integer | 64 bits | ±4.9E-324 to ±1.7976931348623157E+308 |
Measuring how long it takes your computer to solve different math problems and store answers provides useful information. Programmers use measurements to develop programs that will run quickly. Computer manufacturers use measurements as a benchmark comparison between two computers. However, you must keep in mind that with today's multitasking operating systems, measuring the execution time of any single process is difficult. The operating system splits CPU time between all of the programs that are running. No program has exclusive access to the CPU. Generally, your computer works so fast that you do not notice that it is doing multiple things at one time, especially if programs appear to respond instantly. Behind the scenes though, each program that is open or running is getting a slice of the computer's attention (CPU time), then waiting for its next turn before it can run again.
In this project, instead of using your own computer, you will run tests on a server, or a special computer that runs websites and stores lots of information. The file server running the web-based tests is doing hundreds or thousands of other things, too, like running the rest of the Science Buddies website. In order to get the most accurate results, try each test a few times, and try again at a different time of day when the server might be less busy.
Terms and Concepts
To do this project, you should do research that enables you to understand the following terms and concepts:
- Benchmarking
- Inputs
- Software
- Central processing unit (CPU)
- Hardware
- Memory
- Multitasking
- Operating system
- Data types
- Integers
- Long integers
- Floating points
- Double-precision floating points
- Bits
Questions
- Which data type do you predict the test computer will process most quickly? Which do you predict the computer will process most slowly?
- Computers used to work with 8-bit numbers, or bytes for calculations. Modern computers use 32- or 64-bit numbers for simple calculations. How do you think that might affect the results you saw? If you could run these tests on a really old computer that worked with 8-bit numbers (and had to do four operations to perform a 32-bit addition), how might your results differ?
- How do you measure the speed of computer operations?Hint: Even if you do not run the programs on your own computer, look at the code provided in the materials list. The server's code is similar. See the Technical Note #1 in the Experimental Procedure and Figure 2.
- Why are older computers slower than newer ones?
- When manufacturers sell a computer, they list the "processor speed." What does this mean?
- Why does it matter how much memory a computer has?
- How does a computer do multiplication and division? If it has special hardware to do the task it can do it a lot faster than if it has to do it long way like you learned in school. Does the test server you ran the code on have hardware multiplication and division? Do you think it has hardware to handle floating point math? What about your own computer if you tried running from the source code?
Bibliography
- Evslin, T. (2008, April). Why Old Computers Get Slow. Retrieved May 24, 2011.
- Fay-Wolfe, V. (2005, December). How the Computer Works: The CPU and Memory. Retrieved May 25, 2011.
- Flanagan, D. (2001). Primitive Data Types. Java In a Nutshell. (Ch. 2.4). Retrieved May 25, 2011.
- Wikipedia Contributors. (2011, May 20). Benchmarking (Computers). Wikipedia: The Free Encyclopedia. Retrieved May 25, 2011.
Materials and Equipment
- Computer with web browser and Internet access
- Calculator or spreadsheet program, such as Excel®, for analyzing data
- Lab notebook
- Graph paper
- Optional: Python 3 with IDLE and Python 3 timing test code
- Optional: C compiler and C language timing test code (2 files)
Experimental Procedure
- First, do your background research, and make sure that you understand the terms and concepts and can answer the questions from the Background section.
- Create a data table, like Table 2, in your lab notebook or spreadsheet program. Note: If you use a spreadsheet program, remember to print out your table when you are done and paste it into your lab notebook.
| Computer | Data Type | Mathematical Operation | Trial # | # of Calculations | Total Time To Perform Calculations, Milliseconds (ms) |
|---|---|---|---|---|---|
Image Credit: Howard Eglowstein, Science Buddies / Science Buddies
Figure 1. A timing test web page running on one of Science Buddies' servers.
In the basic form of this experiment you will be using one of Science Buddies' servers to run some experiments. A web interface, shown in Figure 1, lets you select an operation, a data type, and the number of times (iterations) that you want to run the operation (See Technical Note #2).
The materials list includes source code for programs in Python and C. The Python program is written to run in Python 3.x, available on Windows, OS X or Linux (try it on a Raspberry Pi!). The C program is written for standard POSIX C, so with minor modifications it should run easily on any machine with a reasonably standard C compiler. If you work with C or C++, then you know what that means.
Even if you do not program, download these source code files and look through them. Do not worry about the language details, but read them as a description of how the tests work. You will be surprised how easy it is to read some code, even if you do not know how to write it.
Image Credit: Howard Eglowstein, Science Buddies / Science BuddiesDiagram showing a timing algorith. The process begins when the start time is recorded. The program then does a single math operation, and evaluates if the operation has been done a preset number of times. If no, the program does another math operation, and re-evaluates the test condition. If yes, then the program terminates, and the start time is subtracted from the end time to calculate the total processing time.
Figure 2. A flowchart of a timing algorithm. The program will see what time it is, run an operation some number of times, look at the time again, and subtract to see how long the operations took.
- Load the program by clicking the testing page.
- Use the web page to measure the execution time for each of the mathematical operations (addition, subtraction, multiplication, and division) using each of the data types (integer, long integer, floating point, and double-precision floating point).
- Choose 32-bit integer (int) from the Set Data Type options.
- Choose addition (+) from the Set Operation to Perform options.
- Set Number of Iterations to 100,000.
- Run the test at least 20 times. Each time you run the test it will return the amount of processing time in milliseconds (ms) that it takes your computer to run all 100,000 calculations using that data type and mathematical operation. See the Technical Note to learn more about how the program works.
- Record the time results of all 20, or more, trials in your data table.
- Repeat steps a–e for all the possible combinations of data types and mathematical operations. In the end there should be 24 combinations, each with 20 trials.
The web application and the source code that you can run on your own computer works through a series of commands:
- First, it tells the computer to record the start time.
- Then, it gives the computer two commands, we'll call those a and b.
- Command a, tells the computer to perform an operation on a number. This number can be one of several types of integers, floating points, or double-precision floating points, depending upon which option you chose. The operation (addition, subtraction, multiplication, or division, depending upon which option you chose) is performed on the number stored in memory. We want to do the minimum amount of work in addition to the actual operation, so the examples use small constants like adding or subtracting 1.
- The next command, b, tells the computer to repeat command a a certain number of times. You can choose the number of repeats (or iterations) by changing the Set Number of Iterations. If you select 100,000, this means that the computer is doing 100,000 arithmetic calculations each time you click 'Run'.
- When command b is done, the next command records the end time.
- The final command is to subtract the start time from the end time and display that on the screen. The time, in milliseconds (ms), that it takes to do the calculations.
Is it always better, when running benchmark tests, to run a large number of iterations or not? Well, there are quite a few things that can affect your results.
The most evident problem is that most current computers will be running multitasking operating systems. That means that even if you think the computer is not doing anything, it is probably running dozens, if not hundreds, of small tasks. Your calculating 100,000 or 1 million additions might take a fraction of a second, but during that time, the computer might have also checked the mouse for motion, looked at the keyboard, talked on the internet, etcetera. The longer you run (the more iterations), the more likely it is that the program was briefly interrupted. On the other hand, if you only run it for a few iterations, those will happen so fast that the computer's clock may not be able to time them accurately.
To minimize the effect of this multitasking, you will want to run with a variety of settings and look for an average speed. You can also try running the tests on your own computer using the source code provided for Python or C (or better yet, write your own in your favorite computer language!). If you are a purist, see if you can write a test like this on a computer that does not multitask.
Running the Python Code
The supplied Python code will run on most computers using Python 3. We recommend you start by running the Python interactive editor, IDLE, and load the code into the editor.
At the very top of the program, look for lines like:
testDataOperation = "no op" # could be "plus", "minus", "multiply", "divide", "no op" or "assign"
testDataNumOps = 10000000 # The number of operations
Change the initialization data on these lines and then run the program. Record your results and then re-run, or change the data and try something else.
You might prefer to use this code as the basis for your own and write a procedure that runs several tests in a row without your having to do it manually.
Running the C code
The supplied C code was written for and tested on GCC Linux compilers but will also work with any C compiler that is close to POSIX compliant. The 'makefile' tells the GCC compiler what to build. From a command line, type 'make' to build the default executable program. If you're not a fan of 'make', you can compile it with the gcc compiler directly with the command 'gcc -o sbtest sbtest.c'.
The program, as supplied, will use some preconfigured settings if you simply run './sbtest'. You can also specify arguments on the command line to set the operation, number of iterations and the data type. './sbtest add 1000000 bit32 ' will execute one million 32-bit integer additions.
Analyzing Your Data
- Use a calculator or spreadsheet program, such as Excel, to calculate the average processing time for each combination.
- Make four bar graphs to examine if the amount of time the computer needed to do a single type of mathematical operation (like addition) changes with different data types.
- Make one bar graph for each type of mathematical operation. There will be a total of four graphs: addition, subtraction, multiplication, and division.
- Each graph should have four bars on the x-axis, one for each data type.
- The height of each bar, on the y-axis, should be equal the average time for that combination.
- Look at all four bar graphs. Are any of the data types slower than the others or faster than the others? How do your predictions about processing time and data type compare with the actual results? Were you surprised? If so, can you think of an explanation for what is going on?
Ask an Expert
Variations
- Try using the Python or C programs and running these on different computers whose processor type and speed you know, and compare the results. Keep conditions as similar as possible by minimizing the number of additional programs running. How do the times compare for computers that have the same processor types, but different speeds? How do the times compare among computers with different processor types?
- Do your results change if you have other programs running while you make measurements? Does it matter if they are actually doing something or not?
- Here is an idea for analyzing your data further: How many clock cycles does each operation take? If you know the "speed" of your processor (in MHz or GHz), you can calculate this by multiplying the time per operation (in seconds) by the processor frequency (in hertz).
- For another look at this problem, use JavaScript to write your own program to investigate execution time for a function call or writing data to the screen. For example: how long does it take to generate a random number using Math.random()? How does this compare with calculating the sine of an angle using Math.sin()? How does this compare to simple arithmetic operations in any computer language you might know? Can you estimate how many arithmetic operations are performed to generate a random number? How long does it take to write a line of text to the screen? How does this compare with changing the background color?
Careers
If you like this project, you might enjoy exploring these related careers:
