Find out what each dot stands for
The graph looks reassuring. The dots cluster tightly, the curve is smooth and the error bars barely reach past the symbols. Then you read the legend and discover that every dot came from the same preparation. The plot may tell you plenty about measurement repeatability. Whether the result repeats in fresh preparations is a separate question.
The Assay Guidance Manual distinguishes technical measurements, distinct preparations and repeated runs. That vocabulary lets you describe what the study actually repeated. Start in the legend, then follow the description back to the methods. If the paper says n = 3, look for the noun that belongs after the number. Three wells, three lots and three runs make three very different captions.
Follow the sample back to its source
Imagine a scientist reading the same sample twice with two instruments. Both readings are valuable, especially when the question is whether the instruments agree. For a question about variation across samples, however, they are still readings of the same sample. What they share matters to the analysis.
Lazic uses the term pseudoreplication for a mistake in which dependent observations are treated as independent. This can happen when readings share a source or when several measurements are made on the same experimental unit. When assessing a peptide study, draw a line from each observation back to the preparation or unit that produced it. Observations that meet at the same starting point can share sources of variation. Check whether the analysis accounts for that structure.
A worked example with nine wells
Picture a fictional experiment that runs on three days. The team starts a fresh run each day and reads three wells from the same preparation within that run. The nine readings give the team a view of two kinds of variation. The wells within a run show how readings agree under shared conditions. The three runs show what happened when the work was repeated.
Suppose the wells agree closely every day, but the average shifts substantially on day three. Combining all nine readings in one unlabelled scatterplot would hide part of the story. Display the three days as separate groups and the shift becomes visible. The question then becomes whether the difference reflects the run, a shared input or something else that changed. Vaux and colleagues discuss the distinction between replication and repetition. The fictional example shows why preserving the hierarchy in the figure is useful.
Ask which question each control answers
A blank can show the background signal. A reference condition can show how much response the assay is capable of producing under a familiar comparison. If the reference response wanders between runs, a small shift in the test group takes on a different meaning than it would in a stable system.
The original Z-factor paper combines information about the separation and variability of control signals to characterize screening assay performance. You can apply the same reading habit without calculating that statistic. Look for the size of the gap and the amount of variation around it. Then ask whether the controls address the particular interpretation you are considering. A clean background, for example, tells you about baseline signal. The question of whether a sample changes the detector chemistry calls for a different comparison.
Read the error bars as carefully as the points
Find the definition in the legend before deciding what the error bars show. Standard deviation describes how spread out the observations are. Standard error addresses how precisely a mean has been estimated under the analysis assumptions. A confidence interval provides another way to describe uncertainty around an estimate. The number and kind of observations matter to all of these.
Also check what the plot leaves visible. If it shows individual values, can you tell which measurements share a run or sample? If it shows a representative experiment, can you find how many repeats were performed and whether their results are summarized elsewhere? A neat figure is easier to assess when the reader can trace how the numbers were assembled.
What to jot down beside a figure
Try a four-line note: the experimental unit, what was repeated, what each control establishes and how uncertainty is displayed. If a line is blank, put a question mark there. Compare two papers using those notes and you may find that apparently conflicting results came from quite different designs.
For instance, a short caption that reads three independent runs with triplicate wells and error bars showing variation across runs tells you far more than n = 9. It names both levels of measurement and makes clear which one was used to summarize uncertainty. That is a detail worth looking for even in a paper you otherwise trust.
Match your confidence to the design
There is room in a strong paper for more than one kind of replication. Repeated wells can answer questions about the instrument and handling. Independent preparations can show whether the result holds across samples. Inter-run measurements can show how it behaves across separate runs. Each contributes a different piece of the evidence, and the figure should let the reader see which piece is on display.
Frequently asked questions
Do three wells always mean three replicates?
They may be three technical measurements if they share a preparation. Look for what was prepared, assigned or run independently before interpreting n.
Sources: Dahlin and colleagues, Assay Guidance Manual, NCBI Bookshelf (2019): Basic Guidelines for Reporting Non-Clinical Data · Stanley E. Lazic, BMC Neuroscience (2010): The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis?
Can independent preparations be tested on the same day?
Yes. Whether samples are independent depends on how they were prepared and how the experiment was structured. A different day adds information about run-to-run variation.
Do overlapping error bars rule out a difference?
Start by identifying what the bars represent and which observations the analysis treats as independent. A visual comparison of bars alone cannot answer the same question as the study’s statistical model.
Sources: Dahlin and colleagues, Assay Guidance Manual, NCBI Bookshelf (2019): Basic Guidelines for Reporting Non-Clinical Data · Stanley E. Lazic, BMC Neuroscience (2010): The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis?
Is a good assay control enough to establish the study conclusion?
It supports the comparison it was designed to check. Read it alongside the other controls, sample structure and uncertainty before evaluating the broader result.
Sources
- Basic Guidelines for Reporting Non-Clinical DataDahlin and colleagues, Assay Guidance Manual, NCBI Bookshelf · 2019
- The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis?Stanley E. Lazic, BMC Neuroscience · 2010 · DOI: 10.1186/1471-2202-11-5
- Replicates and repeats—what is the difference and is it significant?Vaux, Fidler and Cumming, EMBO Reports · 2012 · DOI: 10.1038/embor.2012.36
- A simple statistical parameter for use in evaluation and validation of high throughput screening assaysZhang, Chung and Oldenburg, Journal of Biomolecular Screening · 1999 · DOI: 10.1177/108705719900400206
Published by EliteGen Research for research education. Studies describe their own materials and experimental conditions; they do not establish the identity, quality, or suitability of an EliteGen product. This article does not provide guidance for personal, human, or veterinary use.



