SimPandemic™ FAQ

What is SimPandemic?

SimPandemic is an easy-to-use, yet powerful simulation tool to learn how a pandemic evolves, enabling you to choose, explore, and witness the outcome of different scenarios. Ask, "What if?" and see what happens in the Sandbox.

Uniquely, SimPandemic looks at both the disease and the economy. What can you do that helps? Or, hurts? What policies are plausible? Or, ridiculous? Can you stop the spread without destroying the economy or overwhelming the healthcare system? We believe that a democracy functions at its best when the citizenry is enabled to ask intelligent questions.

Like a bike with training wheels or learning to swim with a kickboard, SimPandemic is designed to let you hop on for a ride or dive right in for a swim. You can run your first simulation in under one minute, but once you get a feel for things you have approximately 100 parameters that you can customize to explore your own ideas and scenarios.

Who is SimPandemic for?

SimPandemic is a valuable tool for:

How does SimPandemic work?

What were the design objectives for SimPandemic?

The most important objective of any model is to enable the user to ask good questions, questions that they wouldn't have asked without the aid of the model. Other objectives for SimPandemic included the need to be:

What technology does SimPandemic use?

To meet our objectives, we chose to use a classic computer science technology, agent-based modeling. Probably the best-known example of agent-based modeling is the game SimCity™, first introduced in 1989. Agent-based modeling is ideal for populations of people or things where:

In addition, agent-based modeling requires fewer variables to be defined in advance than some other techniques. In fact, agent-based models display emergent behaviors, where some variables can be observed from the behavior of the model itself. This is especially important in the case of COVID-19, because as a novel coronavirus the existing data about the disease is still very uncertain and noisy. The fewer variables that we would need to define up front, the better. Overall, it was clear to us that agent-based modeling was the right tool for the job.

SimPandemic creates a virtual world of 100,000 simulants whose sole functions in life are to remain healthy or become infected, and if they are adults, to perform work. Adult simulants create economic output with their work, so long as they are not symptomatic, quarantined, or out of work because of social distancing business closure. (More on economic output below.) If infected, simulants are able to infect others. When they are no longer infectious and the disease runs its course, they become immune or die. Simulants are divided into different buckets based on age, because some viral diseases such as COVID-19 have outcomes that strongly depend on the individual's age. For every day of the simulation (up to two years or 730 days), SimPandemic updates the status of every simulant based on approximately 100 parameters, including population statistics, disease characteristics, and interventions to fight the pandemic. A single simulation for 365 days involves well over a billion calculations.

Because the simulation is stochastic (it involves chance and probability, just like the real world), you may see the output graphs "wiggle" a bit if you run the same scenario multiple times. See also, Why does the output change when I do multiple runs with the same scenario or parameters?

Why don't you model more than 100,000 simulants?

We would like to, but having more simulants consumes more computer resources. To increase the number, we need to spin up more servers and/or increase the efficiency of the code. We will be tracking usage diligently and will increase the maximum number of simulants if possible.

What assumptions does SimPandemic make?

As mentioned above, an agent-based model requires very few assumptions. SimPandemic assumes that infection gives a simulant immunity for the duration of the simulation; however, it seems like a reasonable assumption because those who recovered from the SARS virus had immunity lasting an average of three years. Additionally, we assume that once a simulant recovers, it can no longer transmit an infection. If our knowledge of the virus changes, we will relax these assumptions.

There are a number of simplifications explained in the FAQ entries on Economic Output.

The vast majority of parameters defining the simulation can be changed by the user for any reason whatsoever, including new data or just to see what happens.

How does the infection process work in SimPandemic?

The Latent Period is the number of days from the initial infection date until an individual is first able to infect others. It is defined in the Disease Characteristics to have a range from a minimum to a maximum number of days. For each infected simulant, SimPandemic chooses a start date for a simulant's period of infectiousness using a uniform distribution between the minimum and maximum Latent Period. (A normal distribution was much slower and didn't change the results much. Given all the uncertainties in the underlying disease data, we went for speed.) Similarly, the Disease Characteristics define a minimum and a maximum Duration of Infectiousness. Using a uniform distribution, SimPandemic chooses dates within the Duration of Infectiousness to infect a number of additional simulants with an expected value equal to the Reproduction Rate. (The algorithm is more complicated, but that's what it is equivalent to.)

The Daily Reproduction Rate = Basic Reproduction Number × HealthySimulants/TotalSimulants, reduced by any interventions that are in effect. Quarantined simulants are off the table, being defined as not Healthy, and Vaccinated simulants (for whom the vaccine worked) are off the table because they are Immune. In other words, Healthy means able to be infected.

How does SimPandemic calculate Economic Output?

The simulator calculates an economic indicator, Economic Output, that ranges from 0 to 100%. The default starting value is 100%, meaning that the economy shows no impact from the pandemic at that time. The economy may or may not be at full output for other reasons.

This is how Economic Output is calculated, including many important simplifications:

NOTE: Currently, the Economic Output is a linear function. There is no attempt to estimate cascading, nonlinear effects or multipliers. Also, there is currently no adjustment for behavioral effects, such as workers declining to return to work after a closure because they are fearful.

How do interventions to fight the virus work?

Interventions to Fight Pandemic Mechanism for the Intervention How Is the Effectiveness Parameter Applied?
Quarantine Quarantine starts when the latent period ends, and all quarantined simulants are infected by definition. Quarantined simulants are taken "off the board." Effectiveness is a measure of the percentage of simulants observing their quarantine.
Improved testing Testing moves infected, asymptomatic simulants into quarantine. It has no effect if a quarantine is not in place. Effectiveness is the percentage of how many of the tested people who are infected actually go into quarantine. It does not take into account false positives.
Improved therapy Directly reduces the fatality rate by the percentage specified. NA
Vaccine A vaccination makes the simulant immune. Because it takes time to develop an immunity after receiving a vaccination, a simulant can still get sick if infected before vaccine effectiveness begins. There is a direct relationship between whether a simulant receives immunity from a vaccination and the Vaccine Effectiveness.
Social distancing (Rounds 1 and 2) for Healthy simulants (protection against contracting infection) Reduces the chance of a simulant getting infected. Reduces the chance of being infected in proportion to the Effectiveness.
Social distancing (Rounds 1 and 2) for Infected simulants (protection against transmitting to others) Changes the Reproduction Rate. (1 - Effectiveness) × Reproduction Rate
Business closures Only effects Economic Output. NA
PPE for Healthy simulants (protection against contracting infection) Reduces the chance of a simulant getting infected. Reduces the chance of being infected in proportion to the Effectiveness.
PPE for Infected simulants (protection against transmitting to others) Changes the Reproduction Rate. (1 - Effectiveness) × Reproduction Rate
Travel In Adds simulants to the population. NA
Travel Out Deletes simulants from the population. Any simulant who is not dead can travel out. NA

How does SimPandemic combine the Effectiveness of different interventions?

In many cases, interventions interact with each other. For example, PPE and social distancing are not independent. If you are social distancing in your apartment all day without seeing anyone, you are not safer if you also put on a mask! The data to precisely calculate these dependencies simply does not exist. Yet we know for sure that most interventions are not independent of one another.

To resolve this SimPandemic applies a naive dependency algorithm that approximates the overlap between multiple interventions being applied at the same time to the Reproduction Rate, reducing their combined effectiveness. We call it naive because it is both a simplification and an approximation, but we feel that it gives reasonable results. When there is only one intervention, its full effectiveness works as specified in the parameters.

How accurate is SimPandemic?

Any simulation has simplifications; a model is not the same as the real thing. This simulator has many simplifications enabling us to develop it in a timely fashion and keep the user interface easy to use.

The strength of SimPandemic is to add to the quality of policy conversations by enabling an informed comparison of different ideas. SimPandemic simulations enable you to make quick and easy "what if" simulations of different scenarios, and it should be very good at helping to understand if an idea is plausible or ridiculous, whether it will help us or hurt us. We didn't design SimPandemic to make front-line operational decisions, like how many hospital beds do I need.

An old computer science adage says, garbage in, garbage out! That definitely applies to the use of SimPandemic. No model can give accurate results if you feed it bad data. At the present time, the data about COVID-19 is spotty and even conflicting. For the default Disease Characteristics parameters, we've taken our best shot at sorting through the emerging research. We will update them as better numbers become available, and you can also modify the parameters if you want. Remember, with the ability to change almost every parameter comes power—use it wisely!

How do I use SimPandemic?

The Notebook

What is a Notebook?

Notebooks guide your exploration of the pandemic with examples and descriptions. And, every Notebook includes one or more Sandboxes that let you run your own simulations. Ask,"What If?"

What are the different lines in the graphs (Infected, Symptoms, Immune, Deaths, and Econ)?

Econ is an abbreviation for Economic Output measured from 0 to 100% and read out on the y-axis to the right. Infected is the number of infected simulants each day. Symptoms are the number of simulants showing symptoms each day. Immune is the cumulative total of simulants who have acquired immunity. Deaths are the number of simulants who have died each day.

What are Total Infections in the table?

Total Infections are the total number of infected simulants for the entire duration of the simulation.

What is Hospital Capacity?

Hospital capacity is a rough approximation of the percentage of hospital capacity that will be required at the highest peak of infections during the simulation. It is based on the Total number of hospital beds per capita entered in Population Statistics and the Rate of hospitalization (percentage) entered in Disease Characteristics. In the United States there is approximately one hospital bed for every 357 people (or 0.28% per capita), more in some communities, less in others.

What is Total Deaths Caused by the Pandemic?

Total Deaths Caused by the Pandemic represents the total number of simulants who die from the pandemic disease during the entire duration of the simulation.

What is Total Deaths Expected from All Other Causes?

Total Deaths Expected from All Other Causes is calculated from the Baseline death rate entered in Population Statistics. This number stays the same in every simulation; it is included to provide perspective for the number of deaths caused by the pandemic. Other causes of death include heart disease, cancer, auto accidents, household & industrial accidents, old age, and every other cause except the pandemic itself. A pandemic might reduce some unrelated causes of death (maybe fewer auto accidents) and increase others (people might not seek medical care when they should because of fears about catching the disease). SimPandemic does not try to adjust for such impacts to the baseline death rate.

What is Economic Output (% full)?

Economic Output (% full) shows the percentage of economic output for the entire duration of the simulation. See also, How does SimPandemic calculate Economic Output?

What is Customize Settings?

Customize Settings is where you go to ask, "What if?" Change any simulation parameter and see the results seconds later in the Sandbox.

What is Re-run?

SimPandemic uses probabilities in many calculations, for example, how many simulants might be infected by another simulant. When you press Re-run SimPandemic "rolls the dice" again and the results might be slightly different, depending on the situation. Some conditions will be more susceptible to variation than others, and Re-run is your way to examine that. See also, Why does the output change when I do multiple runs with the same scenario or parameters?

How does Save/Share work?

Save/Share gives you a unique URL that you can copy and paste to save for future use or share with friends, colleagues, or students. The special code in the URL enables SimPandemic to remember all of your changes in the Sandboxes for the current Notebook. Just paste the special URL into your browser, press enter, and you will be right back where you left off. If you are working in the same browser on the same computer, the URL will not change.

Your edits are saved when you press Run Simulation. If you leave Custom Settings without pressing Run Simulation, your most recent edits will be lost.

IMPORTANT NOTE: If you use a different browser on the same machine or a different computer, you will need to do a new Save/Share to remember your most recent changes.

If you share your URL with someone else, any edits they make will not change what you saved. However, they will need to do a new Save/Share to remember any of their edits.

Customize Settings—Population Statistics

What is the Baseline death rate (annual)?

The Baseline death rate is the average, annual number of people who die divided by the total population. For the United States, the current number is approximately 0.864%.

What is the Total number of hospital beds per capita?

In the United States there is approximately one hospital bed for every 357 people (or 0.28% per capita), more in some communities, less in others.

What is the Percentage of total population?

Many viruses have differing impacts on different age groups. Percentage of total population establishes the size of the different groups and should total 100%. SimPandemic defaults to approximate percentages for the United States:

You can adjust these percentages for any other territory.

What is the Number of already infected (Individuals on day 1)?

To have a pandemic you must start with at least one infected individual. For most of SimPandemic's simulations we start with a Number of already infected from 1 to 25, depending on the scenario we discuss.

An alternative way to begin a pandemic is by using Travel In with infected individuals (this is in Customize Settings-Travel Restrictions).

What is the Percentage of immune individuals?

COVID-19 is a novel virus, so Percentage of immune individuals is 0% at the beginning of the pandemic. To the best of our knowledge, no one has immunity. For other viruses there may be some immune individuals at the start.

Customize Settings—Disease Characteristics

What is the Basic reproduction number, R₀?

The Basic reproduction number, R₀, is a measure of how many others an infectious individual will infect. It depends on the properties of the virus and environmental conditions such as population density and social customs. Because of this the R₀ for one community will be different from other communities.

What is the Latent period?

The Latent period is the number of days from the initial infection date until an individual is first able to infect others. The latent period varies from person to person, so we input a minimum and a maximum.

What is the Duration of infectiousness?

The Duration of infectiousness is the number of days that an individual with the disease is able to infect others. The duration of infectiousness varies from person to person, so we input a minimum and a maximum. We consider the end of the duration of infectiousness to be the "end of the disease." The simulant goes out of quarantine (if it was in quarantine) and becomes immune or dies, which is a simplification of real life.

What is the Chance of being asymptomatic (percentage)?

Not everyone infected with a disease will show symptoms. Chance of being asymptomatic (percentage) specifies the probability of an individual not showing symptoms.

What is the Rate of hospitalization (percentage of symptomatic cases)?

The Rate of hospitalization (percentage) specifies the percentage of symptomatic cases that will end up being hospitalized.

What is the Chance of death (percentage of symptomatic cases)?

Chance of death (percentage of symptomatic cases) specifies the percentage of symptomatic cases that will end up dying. NOTE: This is different than the number of infected cases that will end up dying.

Customize Settings—Interventions to Fight Pandemic

What is Social Distancing and Social-distancing Effectiveness?

According to the US Centers for Disease Control (CDC): "Social distancing, also called 'physical distancing,' means keeping space between yourself and other people outside of your home. To practice social or physical distancing:

Social distancing is one of the ways to mitigate a pandemic. Social-distancing Effectiveness specifies the efficacy of the mitigation. If adherence to the social-distancing guidelines is high and the guidelines are themselves effective, you would specify a high Social-distancing Effectiveness. If adherence is low or the guidelines themselves are poor, you would specify a low Effectiveness.

SimPandemic enables two separate rounds of social distancing that have different settings.

What are Business closures (percentage of total businesses) and Employees able to work at home (percentage)?

In SimPandemic, business closures reduce Economic Output; they have no impact on disease spread. As a simplification, "business" is broadly defined to include any place of employment in the private or public sector. If 25% of businesses are closed, Economic Output will drop by 25%, adjusted for the percentage of people who can work at home. If business closures are set to 25% and 50% can work from home, then 12.5% of simulants will no longer contribute to Economic Output for the duration of the closure.

SimPandemic enables two separate rounds of business closures that can have different settings.

What is Personal Protective Equipment (PPE) for the healthy and Effectiveness at keeping the wearer safe?

If a healthy person wants to protect against infection, they might wear a mask or disposable gloves when out in public. (Healthcare professionals might wear much more sophisticated PPE, but SimPandemic is simulating the general population.) Effectiveness at keeping the wearer safe specifies the efficacy of the PPE at preventing infection. Currently, data on the efficacy of masks is all over the place, and it varies by type of mask and whether proper procedures are followed for using the mask. A realistic value is probably well under 50%.

What is Personal Protective Equipment (PPE) for the infected and Effectiveness at preventing the wearer from spreading infection?

If an infected person wants to protect against infecting others, they might wear a mask when out in public. Effectiveness at preventing the wearer from spreading infection specifies the efficacy of the PPE at preventing the spread of infection. Currently, data on the efficacy of masks is all over the place, and it varies by type of mask and whether proper procedures are followed for using the mask. Currently, most experts feel that masks are probably more effective at preventing an infected wearer from spreading infection than they are at keeping an uninfected wearer safe. The data is still very uncertain.

What is a Quarantine? What is Quarantine Effectiveness?

In SimPandemic, a Quarantine starts when an infected simulant's latent period ends and infectiousness begins. A quarantine prevents an infectious simulant from infecting other simulants. Quarantine Effectiveness specifies the percentage of infectious simulants placed in quarantine and/or observing their quarantine orders. A quarantine can be whatever you want it to be, confining someone to their home or sending them to special facility; just specify the effectiveness correspondingly. A 100% effective quarantine would be extraordinarily difficult to implement in practice. On the one hand some percentage of people will try to cheat, and on the other hand some people would try to cover their symptoms to avoid quarantine.

What is Improved testing, Phase-in time and Effectiveness (percentage)?

Testing moves asymptomatic simulants into quarantine. It has no effect if a quarantine is not in place. The Phase-in time specifies a period of time for ramping up the testing. There is a direct relationship between the testing Effectiveness (percentage) and the percentage of simulants moved into quarantine. Keep in mind what amount of testing is practical (itself a moving target) when you specify testing effectiveness.

What are Improved therapy and Reduction in deaths (percentage)?

SimPandemic assumes that an Improved therapy is for treatment after contracting the disease (not a prophylactic). Reduction in deaths (percentage) specifies the proportion of deaths prevented by the therapy.

What is the field How long does it take to vaccinate everyone?

Vaccination of a large population cannot happen overnight. It takes time to produce the vaccines and then it takes time to administer them. How long does it take to vaccinate everyone? enables you to specify how long this process will take.

What is the field How long does it take for the vaccine to become effective in the body?

Vaccines stimulate the body to produce antibodies which protect against infection. This immune response can take 7 to 21 days or more depending on the vaccine. There is no protection until the immune response occurs.

What is Vaccine effectiveness?

Vaccines are not 100% effective. For example, flu vaccines are generally 40 to 60% effective according to the CDC. SimPandemic will protect the percentage of simulants that you specify here.

What are Travel restrictions, Percentage of the total population (per day)?

Travel restrictions, Percentage of the total population (per day) specifies the number of people travelling per day, as a percentage of the total population. Generally, you would specify a fairly small number. For example, all international travel into the United States in 2018 was equal to approximately 0.099% of our total population per day.

Where can I find statistics on international air travel to and from the U.S.?

A good source is: Bureau of Transportation Statistics Air Passenger Travel Arrivals in the United States from Selected Foreign Countries

Other Usage Questions

How do I model a school closure?

In SimPandemic you can think of school closures as a form of social distancing for kids. Use Custom Settings - Social Distancing to specify school closures for Pre-school (<5 years old) and/or School (5-17 years old) age groups.

How do I model for comorbidities?

SimPandemic doesn't have a specific population group for comorbidities. However, you can be creative and reassign the population groups throughout all the parameters in Custom Settings. For example, setup Population Statistics and Disease Characteristics such that one age group (perhaps Elderly (>= 65 years old)) is actually your comorbidity group. Adjust the other age groups appropriately.

How can I clear or reset SimPandemic to its original parameters?

Just go to https://www.sciencebuddies.org/simpandemic making sure that there are no characters after simpandemic.

Why doesn't SimPandemic show error bands on its output?

That would be a great feature! There are trade-offs that involve how long it would take for each run (quite a while) and the cost of our server bill if we enabled such a feature. (Did we remember to tell you that SimPandemic is free?) In the meantime, you can manually do multiple runs to see if your scenario has significant variation between different runs.

See also, Why does the output change when I do multiple runs with the same scenario or parameters?

Why do you start most scenarios with a fixed number of infections, instead of having simulants Travel In to begin the pandemic?

We think that it is better to start with a small, fixed number of infections (we currently use 10) if the purpose is to compare different scenarios (except when you are studying Travel In itself). When you start the infection using Travel In there will be large fluctuations in outcome between different runs because of chance. See, Why does the output change when I do multiple runs with the same scenario or parameters? and Do I need to be careful with small numbers?

Why does the output change when I do multiple runs with the same scenario or parameters?

Many events in the real world and in a simulation of the real world are based on chance. When you become infected in the real world, you often don't know when or where the infection occurred. Perhaps someone sneezed when you were randomly walking by them. A simulation cannot predict that you were going to get infected (except in special cases), but it can predict fairly well that someone would get infected. In SimPandemic there are many events where the simulation literally rolls virtual dice to determine when an infected simulant will transmit the disease to another (all within the bounds specified by the input parameters).

Most of the time, all of this rolling of the dice gets averaged out, but not always. You are most likely to see differing results when you are looking at small numbers, like when a pandemic is just starting or ending or when you are trying to model a very subtle effect. The dice roll differently on each run and you see the difference in the output.

If the behavior of a subtle effect is important to you, then you should do multiple runs and aggregate the results in an appropriate fashion.

Do I need to be careful with small numbers?

Yes! Even with a population of 100,000 simulants, some effects can be too small to reliably simulate. For example, Travel In and Out needs to be greater than 1/100,000 per day to have any effect on the simulation.

See also, Why does the output change when I do multiple runs with the same scenario or parameters?

Are you going to continue to develop SimPandemic?

Yes! Please give us your suggestions about how we can improve SimPandemic.

What are the sources for default values in Population Statistics and Disease Characteristics?

Baseline Death Rate

Centers for Disease Control and Prevention (2020). Deaths and Mortality. https://www.cdc.gov/nchs/fastats/deaths.htm Accessed 29 April 2020.

Percentage of Total Population (age groups)

United States Census Bureau. (2010). Age and Sex Composition 2010. https://www.census.gov/prod/cen2010/briefs/c2010br-03.pdf. Accessed 29 April 2020.

Total Number of Hospital Beds Per Capita

American Hospital Association. (2020). Fast Facts on American Hospitals. https://www.aha.org/statistics/fast-facts-us-hospitals. Accessed 29 April 2020.

Latent Period

World Health Organization. (April 2020). Coronavirus disease 2019 (COVID-19) Situation Report-73. https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200402-sitrep-73-covid-19.pdf?sfvrsn=5ae25bc7_2 Accessed 29 April 2020.

Lauer, S. Grantz, K. Qifant, B, Jones, K., Zheng, Q., Meredith, H., Azman, A., Reich, N., and Lessler, J. (2020). The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application. Annals of Internal Medicine. 12 March 2020, https://annals.org/aim/fullarticle/2762808/incubation-period-coronavirus-disease-2019-covid-19-from-publicly-reported .

Duration of Infectiousness

Liu, Y., Yan, L-M., Wan, L., Xiang, T-X., Le, A., Liu, J-M., et-al. (2020). Viral Dynamics in Mild and Severe Cases of COVID-19. The Lancet Infectious Diseases. 19 March 2020. https://www.thelancet.com/journals/laninf/article/PIIS1473-3099(20)30232-2/fulltext

Rate of Hospitalization

Centers for Disease Control and Prevention. (2020). Severe Outcomes Among Patients with Coronavirus Disease 2019 (COVID-19) — United States, February 12-March 16, 2020 https://www.cdc.gov/mmwr/volumes/69/wr/mm6912e2.htm#T1_down Accessed 29 April 2020.

Chance of Being Asymptomatic

Bendavid, E., et al. (2020). COVID-19 Antibody Seroprevalence in Santa Clara County, California.MedRxiv. https://www.medrxiv.org/content/10.1101/2020.04.14.20062463v1.full.pdf

Business Insider Italia. (2020). ESCLUSIVA I Nuovi Dati di Robbio, Unico Paese Italiano a Fare il Test Sull'immunità a Tutti i Cittadini. 70% di Asintomatici, https://it.businessinsider.com/esclusiva-cosa-rivelano-i-primi-test-di-robbio-primo-paese-italiano-a-fare-i-test-sullimmunita-a-tutti-i-cittadini/?refresh_ce Accessed 29 April 2020.

Sood, N., et al. (2020). Seroprevalence of SARS-CoV-2 Specific Antibodies Among Adults in Los Angeles County. https://cdn.arstechnica.net/wp-content/uploads/2020/04/Los-Angeles-County-Seroprevalence-Study.pdf

Chance of Death

Worldometer. (2020). Age, Sex, Existing Conditions of COVID-19 Cases and Deaths . https://www.worldometers.info/coronavirus/coronavirus-age-sex-demographics/. Accessed 29 April 2020.

2020/04/29

Free science fair projects.