Scientific Critical Thinking

Problem

Research papers, experimental protocols, and scientific claims often fail not because they lack results, but because the results do not support the conclusions: the design may not match the question, the statistical inference may be fragile, bias may be uncontrolled, and the evidence may be weaker than the claim implies. This skill turns those review checks into a concrete workflow, using GRADE and Cochrane ROB to evaluate methods, design, statistics, bias, and evidence strength.

How It Works

• Methodological critique: reviews study design, internal/external/construct/statistical validity, randomization, blinding, and measurement quality.
• Bias identification: covers confirmation bias, publication bias, selection bias, measurement bias, analysis bias, and confounding.
• Statistical review: checks sample size, power, multiple comparisons, p-value interpretation, effect sizes, confidence intervals, missing data, and model assumptions.
• Evidence grading: combines design hierarchy, convergence, and GRADE up/downgrading factors to judge how strongly a conclusion can be supported.
• Claim evaluation: traces the move from correlation to causation and from data to conclusion, flagging logical fallacies and overgeneralization.

The skill first classifies the request, then loads the relevant references checklist. If key details are missing, it asks targeted questions about design, sample size, and outcomes before producing structured feedback ordered by severity: serious issues, important concerns, and minor limitations.

Boundaries

It is useful for paper review, protocol critique, systematic reviews, meta-analyses, and claims assessment. It does not replace clinical judgment, statistical computation, or domain expertise. When evidence conflicts or information is missing, it should list open questions and uncertainty instead of forcing a single conclusion.