Full Stack Data Analysis Project Workflow

Problem

Data analysis projects often stall on fragmented workflows: goals are not broken into executable questions, dependencies and directories are unconstrained, chapter outputs scatter across the repo, script paths and report formats are inconsistent, and delivery is hard to reproduce.

How It Works

The skill advances through 7 stages: parameter initialization, goal definition, project convention, research design, task dispatch, code scaffolding, and quality checking.

• It starts with project_params.json to fix the project name, raw data file, entity config, chapter list, chapter dependencies, Python version, and environment name.
• It then uses docs/analysis_goals.md to frame descriptive, diagnostic, predictive, or prescriptive goals and constrain scope with SMART criteria.
• It generates requirements.txt, docs/flow_design.md, and docs/execution_prompts.md, breaking each chapter prompt into steps, inputs, outputs, checks, and exception handling.
• During execution, src/utils/ modules cover data loading, visualization, metrics, output archiving, task dependency graphs, and global config, reducing hard-coded paths.
• Batches are validated after completion, and outputs/quality_report.md checks directories, dependencies, scripts, reports, archiving, and conclusion consistency.

Boundaries

It fits multi-chapter, dependency-heavy analysis projects that need reproducible delivery; it is heavier than needed for one-off questions, unclear data sources, or single-chart tasks.