Skip to content
No Vape

What did an attention check remove from a vaping study?

Follow instructed-response and source-recognition checks from the questionnaire into the study’s analysis count.

No Vape · Updated

Stack of books on a table
Illustrative image · Laura Ohlman / Unsplash

Read the check before treating it as a quality label

Svenson and colleagues’ experiment used two different checks in a study about e-cigarette information. One asked participants to identify the source of the material they had seen, with the expected answer depending on the assigned condition. Another instructed the participant to select a specified agreement option. The paper describes these items separately from the main attitude questions.

A source-identification item and an instructed response offer different opportunities for an error. One depends on identifying something about the material. The other depends on following the instruction on that screen. The report’s terms let you describe the recorded failure without knowing why an individual gave that answer.

Nicotine & Tobacco Research: Tackling Smoker Misperceptions About E-cigarettes Using Expert Videos

Follow the route from completion to analysis

The Results report exclusions for eligibility and for failing an attention check. Write those reasons separately even if both shrink the final count. Ask whether the authors report how many people completed the study and how many they kept in each assigned condition. That information tells you who the reported comparison represents.

In an invented example, 60 people finish a web experiment and the report excludes four under its check rule. The analysis described uses 56 if no other exclusions apply. If three removed records came from one condition and one from another, keep that difference visible. The example provides no actual study figures or assumption about how the estimates changed.

Nicotine & Tobacco Research: Tackling Smoker Misperceptions About E-cigarettes Using Expert Videos

Look for what was done with flagged answers

Check whether authors removed whole records, set aside one answer or kept the records and compared results under another rule. In a randomized experiment, ask whether the exclusions came after assignment. If readers only have the overall number excluded, they cannot infer how those records were distributed across study groups.

A flagged answer cannot tell you why a respondent missed the check. The task could involve reading, remembering, interpreting a choice or operating a screen. Describe the rule used and the reported comparison without turning a questionnaire event into a judgment about a person.

Keep the wording about the final sample specific

An evidence note can say which check was used, at what stage it was evaluated and what happened to failed records. If the analysis with all records is unavailable, say that the effect of changing the rule is unknown. An attention check can help define the analyzed dataset, but the main conclusion still depends on the outcome and study design.

The cited experiment is a methods example. Its check results offer no way to evaluate your own attention during a craving or decide which cessation support fits.

Questions about this step

Can I count a failed attention item as a missed survey?

Check the participant-flow description. A person can submit a complete questionnaire and later meet an analysis exclusion rule, which leaves a record of completion and a separate record of exclusion.

Sources and scope

Sources inform the guidance. The worksheets and examples are No Vape editorial suggestions, not tested treatments.

  1. Nicotine & Tobacco Research — Tackling Smoker Misperceptions About E-cigarettes Using Expert Videos

Written with AI assistance and checked against the cited sources. No clinical review is claimed. This guide covers practical planning, not individual medical treatment. How we research and write