Am I doing my microbiology meta-analysis correctly?

[Wisteria]

I'm working on an ISEF science fair project on novel antibiotic targets to combat antibiotic resistance, and because of this year's situation, I won't be able to test my idea in a laboratory. Instead, I decided to identify the most promising novel drug targets and design the molecules to target those areas on bacteria, while also conducting a meta-analysis of the use of gene silencing technology on bacteria. However, since most of the studies I collected reported in CFU or percentages, is it possible to report the results in percentages as a dichotomous variable? For example, could I report the percentage of bacteria expressing a certain gene after the gene silencing treatment as a dichotomous variable rather report in raw data? Thank you!

[MadelineB]

Hello Wisteria and welcome to Science Buddies!

Congratulations on being a finalist at ISEF!

Your projects sound very exciting and it looks like you have come up with a good approach to deal with the lack of access due to the pandemic. I removed your duplicate post since Science Buddies requests that you only post your question once. I've kept your post here in the Grades 9-12 Life, Earth and Social Sciences because I think these experts will be better able to help you.

I'm not an expert in the design of molecules or in gene-silencing technology so I hope experts in those areas will reply. My experience is as a statistician and data scientist in medical research. With regard to your question about how to report the data on the gene-silencing, as a statistician, I suggest that you report the data as close to how it is reported in the research that you are including in your meta-analysis. That is, use CFU where that was reported. The numbers of CFU will provide a much more sensitive measure of the effects of the gene-silencing technologies. I suggest that you definitely not dichotomize any of the results, since that would be the least sensitive measure.

I am curious how you are selecting the reports to include - are you considering more than gene-silencing technique? How many different bacteria are considered? Are you aiming to compare the effectiveness of 2 or more gene-silencing technologies and might the effectiveness vary depending on the bacteria studied? I'm asking lots of questions so we can help you make sure you are collecting data in sufficient detail so you can come to appropriate conclusions about gene-silencing technologies.

I hope this helps and please do not hesitate to ask more questions!

Best of luck,
Madeline

[MadelineB]

Hi Wisteria,

It would be helpful if you could include a link to one of the papers that reports the results in percentages and a link to a report based on numbers of CFU!

Thanks!
Madeline

[Wisteria]

Hello Madeline!
Thank you so much for your reply! Actually, I made a typo in my post, I'm actually submitting this to an ISEF-affiliated fair, not ISEF itself, but I'm hoping to be there soon! For my meta-analysis, I'm only considering one gene-silencing technology, which are PNAs, but the studies are targeting different genes since I couldn't find enough studies that target one specific gene. The heterogeneity between the studies is very significant, since they use different bacteria (although typically E. coli K12) and reporter/target genes, although they target the same general area on the genome. However, the problem is that I'm not sure if I should report my data as a continuous variable or dichotomous variable, because I want to find the number of bacteria still expressing the gene after treatment with PNAs at a certain concentration. I tried finding the mean, median, and standard deviation of the data so I could report it as a continuous variable, but I ended up with some very weird numbers and I'm really not sure how to do the meta-analysis with a continuous variable.

Anyway, here are two examples of the papers I'm using!
For this one, I need the information about the expression of gfp mRNA, which is reported in the text and reported as relative units on the graph: https://www.cell.com/molecular-therapy- ... all%3Dtrue
This one reports in turbidity and CFU: https://academic.oup.com/nar/article/34/20/5915/3100496

Most of my studies report in relative units, which seems to be similar to percentages. I just decided to convert all of the studies into percentages since I could find the percentages of all the studies, but not the raw data for all of the studies.

Thank you so much!

[MadelineB]

Hello Wisteria,

Thank you for the links to those papers. I have a couple more questions ... what are the questions you want to answer with your meta-analysis? What are the criteria for including a report in your analysis? For example, are you including just laboratory (in-vitro) studies or are you also including animal studies or clinical studies?

You may have read this, but if not, I suggest this is a good reference for planning your meta-analysis. It includes references to other useful guidelines for meta-analysis: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3049418/

In my understanding of the general approach to meta-analysis, I would expect that you would actually want your outcome measure to be continuous. The two papers you cite report results in numbers of CFU. That seems to be an appropriate measure for reporting in your study - depending on the overall aim of your meta-analysis.

You've chosen a very interesting and important area of medical research. I think a rigorous meta-analysis that includes careful design and statistical analyses will make a good science project. I hope that thinking about some of my questions will help you. Let me know if you have more questions!

Madeline

[Wisteria]

Hello Madeline,
Since this is a relatively new field, I don't think there are enough animal studies or clinical trials available. I have found some animal studies scattered in papers that mostly focus on laboratory studies, so most of my data will be from laboratory studies.

For the two studies I listed, the first one only has the results I'm looking for in relative units rather than CFU, the one listed in CFU is another variable that they tested, so I'm only looking for the results about the expression of GFP, which I've presented in relative units. Sorry for the confusion!

As for continuous data, how should I go about finding the mean and standard deviation of the data? I'm a bit confused on that part, so how should I find those given the types of data presented in the papers I have? Any help will be greatly appreciated!

Thank you so much for your help, all of this information has been super helpful to me!

[MadelineB]

Hello Wisteria,

I am still not clear about the design of your analysis and the scientific question you hope to address.

My understanding of meta-analysis is you would be looking at multiple reports where the researchers used the same technique and measured the same outcome. See for example: https://en.wikipedia.org/wiki/Meta-analysis

You ask how to find the mean and standard deviation of the data ... that will depend on the scientific question you are asking and what outcomes you are looking at for the reports in your study. So in order for me to help you, it would be good to clarify the scientific question, the specific technique(s) for gene-silencing of bacteria and the outcomes to measure the degree of silencing.

If you are planning to look at reports where the researchers used different techniques, then maybe you might want to refer to your project as a literature survey? The two papers you mention are from 2006 or so. I'm curious if the techniques for using gene-silencing on bacteria have changed in the 14-15 years since those reports? I think changes in techniques might affect the outcome measures. It would be important for your analyses to consider time trends - even if the name of the techniques haven't changed in since 2006.

I hope this helps ... remember that I am approaching your questions from the point of view of the design of your meta-analysis --- which will determine how best to do the statistical analyses! Experts in the field of gene-silencing could very well have different questions!

Madeline

[Wisteria]

Hello Madeline,
The goal of my meta-analysis is to identify whether or not peptide nucleic acids can be used to silence genes on plasmids, which seems to be a technique that is still used since I have a couple papers from 2018 and 2020 that use PNAs to silence genes, even though the technology has been around for a long time. For all the studies, they silence a specific gene on the plasmid using PNAs, although the genes that they silence are different. Since they have different targets, are the differences between the studies too much to be combined? They use the same technique but use that technique on different genes, since I couldn't find enough studies silencing one gene of interest... Since the differences between the studies may be too much to combine, I've been considering just doing them as a literature review. Do you think I could realistically conduct a meta-analysis that uses the same techniques but have different targets? The goal of all of the studies is just to silence a particular gene, and they use the same technique, but the gene they chose just happens to be different.

As for the continous data, for example, with the studies I sent, how would I input the data as a continuous variable if I'm looking for how much gene expression is shown after inhibition with PNAs?

Thank you so much for your help!

[MadelineB]

Hi Wisteria,

Thank you for the very clear explanation! But, I now have more questions (sigh!). Are there any other gene-silencing techniques where there might be more reports? Or, sticking with the use of PNAs, how is "silenced" reported - is this reported as "success" vs "failure" or as "percent disabled"? Are there more than one study for any of the genes? How many different genes have been studied? Are there differences in the amount of "silencing" between genes? Is it possible that gene-silencing might work for some genes and not others? I'm trying to see how you might present the results of your literature review.

And, maybe the hardest question, what is the rationale for silencing genes? Is the ultimate goal to develop therapies to cure or prevent some diseases? Would silencing different genes benefit different diseases? I ask these questions because science fair judges like to have you explain the ultimate benefit of a project.

If you haven't already written your background and rationale, maybe some of my questions will help. Here's a link to the Science Buddies project guide with some good suggestions that would apply regardless of the specific project:

https://www.sciencebuddies.org/science- ... ience-fair

I am not quite sure what you mean by "inputting data as a continuous variable"? The type of data will depend on how "silencing" is measured and reported in the studies you include in your review.

I hope this helps even though I've asked questions, not answered questions!

[Wisteria]

Hello Madeline,
Haha don't worry about the questions, I really appreciate you taking the time to ask and help me with my project! For other gene-silencing technologies, the meta-analysis is actually just half of my project, the other part was the actual design of the PNA, and I'm only doing the meta-analysis since I couldn't do any laboratory work this year because of well, the pandemic. I think PNAs would be the most reasonable given what I'm trying to achieve, since they have good thermal stability and resistance to degradation by proteases, which are important to my idea.

Considering each bacterium as a separate organism, it would be reported as "success vs failure", so one single bacteria could only either be inhibited or not inhibited, I don't think there could be any "this bacteria was somewhat inhibited." However, the data is often presented in percentages, since there are a large number bacteria, but when considering each bacterium as a single unit, it would be a success vs. failure type of reporting. This is why I wasn't sure if I should report in dichotomous or continuous data, since I could either find the number of bacteria that still expressed the gene after inhibition by PNAs or continuous as the data is reported on a continuous scale of PNA concentrations to percentage of bacteria still expressing the gene. There were around two or three studies that all targeted the same reporter gene (the gfp gene), but since the overall project isn't about the gfp gene, I thought it would just make more sense to include all studies. I'm just worried the heterogeneity is too much though. There is a huge variety in how much the gene expression was inhibited at set PNA concentration, for some studies, it's less than 1% of bacteria expressing the gene, while for others, it's around 77% of bacteria. I think a reason could be because the secondary structure prevented the PNA from binding to the mRNA, or because they didn't deliver the PNAs efficiently. However, I'm not very sure how to read the secondary structure of mRNA and evaluate whether a nucleic acid will be able to bind to the mRNA.

The reasoning behind my project is to combat antibiotic resistance through novel antibiotic targets, since another part of my project was finding an area on the bacteria that would cause cell death if the gene was silenced. However, since I wasn't able to get real laboratory data for my idea, I decided to try to conduct a meta-analysis to see if I could get some similar data to back up my idea.

Anyway, thank you so much for your help, and please let me know what you think!

[MadelineB]

Hello Wisteria,

Regarding your questions about what results to report, and continuous vs discrete ... let me ask another question or two!

It looks like some of the papers report "the number of bacteria that still expressed the gene after inhibition by PNAs" ...

Question: are the bacteria referred to all the same species (or subspecies or strain)?

If yes, then it seems to me to make sense to report either the number of bacteria that still expressed the gene, for example you want to report the number still expressing the gene and the total number of bacteria tested/treated with PNAs. And you would want to record those numbers separately for each report.

What would not make sense (at least not without a lot of science to back it up) would be to combine results for different species of bacteria.

When you talk about data is reported on a continuous scale of PNA concentrations to percentage of bacteria still expressing the gene, again, this would only make sense if that is being reported for the same species of bacteria. And, when you record that, I suggest recording (1) the total number of bacteria treated at each PNA concentration as well as (2) the PNA concentrations tested and (3) the number of bacteria still expressing the gene.

If you have multiple reports, all providing this type of information, you could make a spread sheet, with a row for each report X specific bacteria species X the PNA concentration, and columns for (1), (2), and (3).

Hopefully, there is more than report for a particular bacteria? When you say that there are reports 1% and another for 77%, are those reports testing the same bacteria? (for example, E. coli, or Staph aureus)?

If that is what you are finding, that would be of interest, although not good evidence for the promise of gene-silencing with PNA. On the other hand, if those are reports for different bacteria, that might suggest that gene-slicing with PNA is more promising for some bacteria than others.

Or maybe, I've left out an important part of the experiments: which gene is being targeted?! If so, you would need to add a column to record which particular gene for which bacteria at each PNA concentration ....

Does this "spreadsheet" approach make sense? Or am I missing what is important? Let me know if any of this answers any of your questions ....

[Wisteria]

Hello Madeline,
This is going to sound super confusing considering the previous messages, but the studies are focused on gene silencing in plasmids, so I'm not sure if the strain of the bacteria matters since the study is focused on whether or not they can silence the gene in the plasmid. Sorry I wasn't really clear about the plasmid thing! However, the target gene within the plasmid is different, and the plasmid itself is different. Would the type of plasmid or strain of bacteria matter in this type of study?

About "If yes, then it seems to me to make sense to report either the number of bacteria that still expressed the gene, for example you want to report the number still expressing the gene and the total number of bacteria tested/treated with PNAs. And you would want to record those numbers separately for each report," does that mean that it would be considered a dichotomous variable, since there's only two options? The spreadsheet approach sounds like a great idea! Currently, I've written down the number of bacteria still expressing the gene after PNAs and the total amount at a concentration of 2uM, but for dichotomous variables, would it be necessary for me to report all of the concentrations tested? Will it be possible to do a meta-analysis with the data gathered from the spreadsheet? Also, another issue - some of the studies report the actual number of bacteria, while the others just report them in percentages or relative units. Would it be possible for me to report them in different "units" (for lack of a better term), or should I convert them all into percentages so they're consistent?

Thank you so much for the spreadsheet suggestion, it was really helpful!

[MadelineB]

Hello Wisteria,

You ask whether the type of plasmid or strain of bacteria matters in this type of study ... I do not have the expertise in gene-silencing technologies to answer that. If you think that the value of PNA gene-silencing is an open question, it would seem worthwhile to consider the possibility that the value might well differ depending on the type of plasmid and/or the species/strain of bacteria.

Remember that the intent of a meta-analysis is to look at studies that have conducted experiments under the same conditions and look at the results. Look at the 2nd paragraph in the abstract of this paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3049418/ If you include studies that conducted experiments under different conditions and the results vary, you will not be able to sort out whether the results are different because of simple variation or due to some aspects of the differing conditions. Other researchers and perhaps the science fair judges might be concerned that combining results from experiments conducted under different conditions.

Also, I do not understand your question about what would be considered a dichotomous variable. Here's a link to a Science Buddies article that might help clarify the different types of variables.

https://www.sciencebuddies.org/science- ... e-projects

I'm glad you like the idea of using a spreadsheet for recording what you find. This would be useful for both a meta-analysis and/or a literature review. This would be the input into your analysis. I suggest that you expand your spreadsheet columns so you can record the results as reported in each study - that is, record the actual data reported. You can convert to percentages etc later. Better to record the actual values!

You ask about the need to record the PNA concentrations ... again, better to record that data so you can then evaluate whether/how the PNA concentration affects the results.

I know this looks like hard work ... progress in science depends on careful detailed records which then permit thoughtful analyses!

Let me know if this helps ...
Madeline

[MadelineB]

Hi Wisteria,

A couple more suggestions ...

Think of the spreadsheet as the raw data, which you can then use for analyses, either for a literature review or a meta-analysis.

You might want to check that you are recording enough of the useful information after you record the data for 3 or so reports. I'll be glad to review the design of your spreadsheet. You could print it to a pdf and upload it here. This would give us both the opportunity to discuss possible statistical analyses (and graphs) for your spreadsheet data.

I think you might want to record the year of each report. That would let you check for a time trend in some of your analyses. This might lead you to some very interesting findings, given that it looks as though the PNA gene-silencing has been in use since 2006 or maybe earlier.

Speaking of time trends, I'm wondering if the science fair judges might ask if there are other, perhaps newer, techniques for gene-silencing.

My suggestions are based on my experience in statistical design and analyses of medical research data but not in the field of gene-silencing. It would be good to talk with experts in gene-silencing to see their suggestions for the data to record from each study.

Hope some of this helps! Let me know if you have more questions.

Madeline

[Wisteria]

Hello Madeline!
Whew, sorry for the late reply! It sounds like I won't be able to combine my data as a meta-analysis, which makes sense since the heterogenity in study design is a lot, and it wouldn't really make sense to combine the data. For my project, since I wasn't able to test the idea in the laboratory, I just need to prove the fact that it is possible to use PNAs to silence genes on plasmids, which nearly all the studies suggest that it is possible.

However, since I contacted my local science fair and they said that a literature review wasn't really enough to be considered a project, can I use the spreadsheet idea, input the data into a meta-analysis software so it generates a forest plot, and use the forest plot as a visualization of the data to prove that all studies show that it is possible to use PNAs rather than as a true meta-analysis?

I'll start working on compiling the data for the spreadsheet idea, thank you for the suggestion! As for the time trends, I'll look into new gene-silencing techniques. I haven't found any studies or reviews that were against the use of PNAs or found any drawbacks significant to my project, but I did consider CRISPR or RNAi before I realized that CRISPR is difficult to use on bacteria itself and RNAi is a gene knockdown technique rather than knockout, which may be a benefit for some projects, but since I've already put a lot of work into designing the PNA, I just went with it. I'll definitely look into other methods so I can be prepared for questions though! Thank you so much!