Dynamic Functional Connectivity
The runnable commands on this page use the packaged demo commands, which demonstrates the pipeline described below. The full preprocessing pipeline as shown is currently for illustration only, not part of 0.0.1a1. Future release will include a real-data bootstrap. See Scope and Limitations.
This page uses a compact dynamic functional connectivity example to show how
NIPACT keeps cleaned ROI timeseries, sliding-window features, state models,
subject summaries, covariate inputs, group summaries, trace results, and GUI
review connected. The runnable commands use the packaged dfc demo and small
generated timeseries, so the focus is workflow inspection rather than DFC
inference.
The QC and model-result walkthrough later in the page illustrates a fuller DFC workflow with KMeans state centers, HMM state models, subject-level summaries, and phenotype-group comparisons.
Tutorial Overview​
The first section walks through the mechanics with the packaged dfc demo. You
initialize and validate a demo project, inspect source mappings and workflow
shape, run one selected cohort summary step, and trace the selected cohort
summary back to the timeseries used to produce it.
The DFC analysis walkthrough then uses documentation examples to show the review targets a larger DFC workflow would expose.
Initialize The Demo​
nipact init \
--demo dfc \
--project-dir demos/dfc/project \
--runtime-dir demos/dfc/runtime
The command writes a project root, a runtime root, and a workspace-local
nipact.contexts.yaml entry for the dfc context. The project root contains
nipact.yaml, sources.yaml, manifests/init.yaml, step YAML, and workflow
YAML. The runtime root contains runtime/database/registry.db and tiny
generated .npy timeseries files under data/dfc/.
The selected project root and runtime root must be empty and must not contain each other.
Validate​
nipact validate --context dfc
Validation checks the prepared project structure, manifests, step and workflow
YAML, sources.yaml shape and path safety, and registry preflight when
runtime/database/registry.db exists.
Source Index​
The init manifest defines the example subject IDs. Source file bindings live
in sources.yaml, not in the manifest body.
entities:
sub_001:
timeseries: data/dfc/sub_001_timeseries.npy
sub_002:
timeseries: data/dfc/sub_002_timeseries.npy
The paths above are recorded as data/... paths. They resolve under
runtime_root/data/... when NIPACT builds a run plan and when Snakemake executes
the selected jobs.
Workflow Shape​
The packaged workflow is small:
init manifest
-> dfc_source
-> dfc_clean_timeseries
-> dfc_group_summary
dfc_source brings each subject/session timeseries from sources.yaml into the
workflow as a raw_timeseries output.
dfc_clean_timeseries is a subject/session-level transform. It consumes
raw_timeseries and produces one clean_timeseries output per subject/session.
dfc_group_summary is a cohort analysis step. It consumes all
clean_timeseries outputs selected by its bound manifest and writes one
connectivity_summary output for the cohort.
Manifests remain membership lists. Covariates, group labels, design matrices, or statistical formulas should be explicit source files, workflow inputs, or parameters; they are not hidden inside manifest metadata.
Inspect And Run​
nipact workflow list --context dfc
nipact workflow steps \
--context dfc \
--workflow base
nipact workflow plan \
--context dfc \
--workflow base \
--step dfc_group_summary
nipact workflow graph \
--context dfc \
--workflow base \
--step dfc_group_summary
Run a dry-run first if you want to inspect Snakemake's planned jobs:
nipact workflow run \
--context dfc \
--workflow base \
--step dfc_group_summary \
--cores 1 \
--dry-run
Then run the selected workflow step:
nipact workflow run \
--context dfc \
--workflow base \
--step dfc_group_summary \
--cores 1
The non-dry-run records the source files used for the run, records produced
workflow outputs, publishes the selected connectivity_summary output, and
records lineage between inputs and outputs. The cohort input records include the
manifest digest and the number of selected subject/session outputs.
Trace And GUI​
Trace the selected cohort summary by workflow coordinate:
nipact trace \
--context dfc \
--workflow base \
--step dfc_group_summary \
--output connectivity_summary \
--address init
The trace should report provenance_status=complete and include the timeseries
source files and clean_timeseries workflow outputs used for the cohort.
The read-only GUI can inspect the same registry state:
nipact gui --context dfc
The GUI shows the project summary, workflow graph, manifests, registered source files, registered workflow outputs, selected output detail, and lineage.
DFC Analysis Walkthrough​
The walkthrough below illustrates a larger DFC analysis workflow using cleaned
timeseries with 480 timepoints by 400 ROIs. The initial init manifest has 12
subject/session records. The expanded init_expanded manifest has 20 records
and uses explicit _expanded steps so the two analyses remain separately
traceable.
Read this section as a provenance-aware QC tour: first compare the cleaned timeseries and correlation structure, then inspect how modeling choices define state centers and occupancy summaries, and finally compare derivative analyses while remembering that state labels are local to each fitted model.
The selected KMeans branch uses the expanded 20-record manifest, 30-timepoint windows, a 5-timepoint step, full ROI upper-triangle edge features, subject demeaning, and five KMeans states. This branch uses these suffixed workflow steps:
init_expanded manifest
-> dfc_source_subject_demeaned_k5.clean_timeseries
-> dfc_window_features_subject_demeaned_k5.window_features
-> dfc_kmeans_fit_subject_demeaned_k5.state_centers
-> dfc_state_assignments_subject_demeaned_k5.state_assignments
-> dfc_analysis_sources_subject_demeaned_k5.{covariates, stat_summary_config}
-> dfc_subject_summary_subject_demeaned_k5.state_summary
-> dfc_statistical_summary_subject_demeaned_k5.statistics
Loading graph...
The tutorial figures focus on a k=5 subject-demeaned KMeans reference branch and a same-shape derivative that changes only the sliding-window length.
| Workflow | Purpose |
|---|---|
base | Runs the initial 12-record cohort through the full DFC analysis path. |
brain_states_w30 | Uses a 30-timepoint sliding window. |
brain_states_w45 | Uses a 45-timepoint sliding window. |
brain_states_w30_expanded | Runs the expanded 20-record branch with explicit _expanded steps. |
brain_states_w30_subject_demeaned_k5 | Selected KMeans reference branch for the tutorial figures. |
brain_states_w45_subject_demeaned_k5 | Same-shape derivative of the selected k=5 branch with window_length: 45. |
The key workflow outputs are:
| Output | Scope | Meaning |
|---|---|---|
clean_timeseries.npy | subject/session | Source-imported cleaned ROI timeseries for one subject/session. |
window_features.npy | subject/session | Subject/session sliding-window upper-triangle correlation vectors. |
state_centers.npy | cohort | KMeans state centers fitted from subject/session window features. |
state_assignments.npy | subject/session | Window-level nearest-state labels for one subject/session. |
state_summary.tsv | subject/session | State fractions, dominant state, and transition count. |
covariates.tsv | cohort | Explicit imported analysis table with group labels and covariates. |
stat_summary_config.json | cohort | Explicit imported configuration for the statistical summary. |
statistics.tsv | cohort | Small OLS-style group summary over state-fraction outcomes. |
metadata.json | step-specific | Parameters, array shapes, counts, model settings, or label notes recorded by the step that produced it. |
The HMM branch adds a parallel model-family lane with a smaller feature space:
| HMM Output | Scope | Meaning |
|---|---|---|
network_mapping.csv | cohort | Imported CLMS Schaefer-400 ROI to 34 left/right network mapping. |
network_timeseries.npy | subject/session | Subject-standardized 480 timepoint by 34 network-timeseries array. |
hmm_model.pkl | cohort | Fitted five-state diagonal Gaussian HMM. |
state_means.npy | cohort | HMM emission means, one vector per latent state. |
state_variances.npy | cohort | Diagonal HMM emission variances, one vector per latent state. |
transition_matrix.npy | cohort | Cohort HMM state-transition probability matrix. |
posterior_probabilities.npy | subject/session | Timepoint by state posterior probabilities for one subject/session. |
NIPACT records whether a file enters as a source input, contributes to a cohort model fit, carries a fitted model downstream, or serves as an analysis input such as covariates or model-summary configuration:
| Role | Where it appears |
|---|---|
source_inputs | dfc_source imports subject/session timeseries from sources.yaml; dfc_analysis_sources imports covariates and analysis config. |
fit_input | dfc_kmeans_fit collects subject/session window_features outputs into one cohort fit. |
collective_fit | dfc_state_assignments and dfc_subject_summary consume the cohort state_centers output; the HMM network step consumes the cohort network mapping, and HMM assignment consumes the fitted cohort model. |
analysis_input | dfc_statistical_summary collects subject/session state_summary rows selected by its manifest binding. |
source_input | The statistical step consumes the imported covariates and stat_summary_config workflow outputs. |
These roles are recorded in the run plan, dependency records, and trace output. They are why the walkthrough keeps covariates and analysis configuration as explicit imported files instead of hiding labels or model settings inside manifest metadata.
DFC QC And Model Results​
The PNG and JSON files in this section are documentation examples for visual
explanation. They are separate from local runtime outputs and are not included in
nipact trace results.
The first figure set uses the expanded 20-record DFC branch. It shows two views
of the same cleaned-timeseries outputs from brain_states_w30_expanded /
dfc_source_expanded.clean_timeseries.
Loading figures...
The selected KMeans figure set uses full sliding-window ROI-edge vectors after subtracting each subject/session's own mean window-feature vector. The fitted centers are therefore subject-demeaned dynamic-deviation patterns, not absolute functional connectivity states. State labels are arbitrary within the fit and should not be compared by number to labels from other models or reruns.
Loading figures...
The KMeans group-result views below switch from model-QC plots to subject-level
analysis summaries. The reference panel uses
brain_states_w30_subject_demeaned_k5: 30-timepoint windows, subject-demeaned
window features, k=5 KMeans, and the expanded 20-record cohort. Each panel shows
the distribution of one state-occupancy metric across phenotype labels. This is
a raw-metric boxplot view: it keeps HC, RRMS, and SPMS on the same
scale rather than mixing a reference intercept with group-difference
coefficients. Fraction axes are auto-scaled within their valid bounds so the
subject-level spread remains visible.
Loading figures...
The derivative result panel below keeps the same step sequence and changes only
the sliding-window length to 45 timepoints. Compatible source timeseries,
covariates, and analysis-configuration inputs are reused from
brain_states_w30_subject_demeaned_k5; the changed window features and
downstream k=5 state summaries are recomputed for the derivative workflow.
State numbers are local to each KMeans fit. The state_0_fraction panel in the
30-window reference should not be interpreted as the same state as
state_0_fraction in the 45-window derivative.
Loading figures...
The HMM branch is a second model family, not another KMeans tuning branch. It starts from the same cleaned 400-ROI timeseries outputs, averages ROIs into 34 CLMS left/right network channels, standardizes each subject's network channels over time, and fits a five-state diagonal Gaussian HMM over ordered timepoint sequences. This keeps the first HMM feature space tractable for the 20-record demo cohort.
The HMM states are latent sequence states. Their labels are arbitrary and are not matched to the KMeans state numbers above. The shared choice of five states keeps the summary shape comparable; it is not a formal model-selection claim.
| Branch | Feature Unit | Feature Dimension | Model | Assignment Unit |
|---|---|---|---|---|
| KMeans | sliding-window full ROI-edge FC | 79,800 | subject-demeaned KMeans, k=5 | window |
| HMM | CLMS left/right network activity | 34 | diagonal Gaussian HMM, five states | timepoint |
| HMM derivative | CLMS left/right network activity without per-subject channel standardization | 34 | diagonal Gaussian HMM, five states | timepoint |
Loading graph...
Loading figures...
The HMM derivative keeps the same 34-network representation and five-state HMM, but changes one preprocessing parameter: the network channels are no longer standardized within each subject before fitting. This is not raw scanner data; ROIs are still averaged into CLMS left/right network channels. The derivative is useful because network amplitude differences can now influence the fitted HMM emission profiles, transition structure, and subject-level dynamics.
The derivative graph has the same visible step topology as the standardized HMM branch; the workflow name records the parameter change.
Loading graph...
Loading figures...
The final HMM dynamics view uses the same distribution-first presentation as the
KMeans analysis view. It reads the HMM state_summary outputs,
subject transition matrices, and covariates to draw raw subject-level metrics by
phenotype. The uninformative "states observed" metric is omitted here because
all expanded-cohort subjects visit all five fitted HMM states. The cohort-level
analysis_results table is still exported as JSON context, but it is not the
primary figure because the OLS intercept is a reference-group mean while the
phenotype terms are deviations from that mean. Those quantities should not be
read as comparable bars. Fraction and probability axes are auto-scaled within
their valid bounds so the subject-level spread remains visible.
Loading figures...
The derivative group view uses the same summaries and figure layout, but changes
one recorded preprocessing parameter
(dfc_hmm_network_timeseries_34net_k5.params.standardize) from true to
false. Upstream source timeseries, covariates, and analysis-configuration
inputs can be reused, while the network-timeseries outputs, HMM fit,
assignments, subject summaries, and final dynamics analysis are recomputed for
the derivative workflow.
Loading figures...
The TSV model summaries and subject-summary TSVs are NIPACT workflow outputs with recorded lineage. The PNG and JSON files shown above are static documentation files.
Scope​
This page covers NIPACT workflow inspection, DFC QC interpretation, model-result comparison, and lineage for the synthetic demo. See Scope and Limitations for what is out of scope in this release.