Projects and runs#
A simulation that finishes in a minute can live in memory. One that takes an afternoon should not: the process may be a notebook kernel that gets restarted, the results are wanted in another session or on another machine, and someone will want to look at the run while it is still marching. Magnelio’s answer is the project: a directory the analysis writes into as it goes, readable by any other process at any time, and resumable from its last checkpoint.
proj = mio.AnalysisScatteringTD(mesh=mesh, project="magic_tee").run(excited=["port3", "port4"])
Given a project= directory, run() streams everything into it and
returns the reader of that directory — the same object
open_project() returns, so a post-processing script
and the return value of run() are one and the same thing:
proj = mio.open_project("magic_tee")
This chapter is the vocabulary of that directory: what a project and a run are, the states they pass through, what the clock on them means, and how to read one that is still being written.
Vocabulary#
Project. A directory holding project.json (the index: setup,
status, the run table), the mesh, the geometry when one was given, and
one sub-directory per run. The handle is a
Project. It is also the result it holds — a
scattering project answers to S, db, plot_s and the Touchstone
export, an eigenmode project (AnalysisEigenmode(..., project=…)) to
frequencies, n_modes, field, show and plot — so a script
written for the in-RAM result runs unchanged against it.
Run. One march of the time-domain solver. A scattering analysis
makes one run per excited channel, named after it (port3_mode0); the
general time-domain analysis names its runs run_1, run_2, … or
whatever name= it was given. The handle is a
Run, handed out by proj.runs[name]; it carries
the run’s state, its step count, the stop criteria and the reason the
marching ended, its clock, its energy trace, and gives access to the
run’s result and monitors.
Run states. A run is pending from the moment the analysis plans
it (no directory on disk yet), running while the solver marches, and
ends done or aborted. A fifth reading, stale, is not stored: it
is running on disk with nobody writing — the solver process the run
names no longer exists on this host, because the kernel died or the
machine was rebooted. Resume the run, or run the analysis again.
Project status. created before the first run, running while
any run is planned or marching, then done when every planned run is
done, aborted when one ended on a graceful stop or an error (the
analysis call goes with it, so its planned siblings never start), and
stale under the same rule as for a run.
Reading a project#
Typing the project’s name at a prompt, or leaving it as the last expression of a notebook cell, prints its state and a table of its runs:
Project wr90_demo
analysis AnalysisScatteringTD
status done
last call finished 2026-09-05 08:55:15 in 0.5 s
created 2026-09-05 08:55:15
runs 2
run excited state steps energy elapsed stop reason
──────── ─────── ───── ───── ──────── ─────── ───────────
p1_mode0 p1:0 done 701 -47.0 dB 0.2 s energy
p2_mode0 p2:0 done 701 -47.0 dB 0.2 s energy
(A WR-90 section with two ports, excited from each side in turn and
stopped 40 dB below the energy peak — a small run, so the clocks are
short.) proj.runs is a read-only mapping of run names to run
objects; each run prints as a summary of its own, and its attributes
are the columns of the table and more:
run = proj.runs["p1_mode0"]
run.state, run.n_steps, run.stop_reason # 'done', 701, 'energy'
run.energy_db # the last energy sample, dB below the peak
run.energy_trace # the whole trace: step, time, energy
run.result() # the run as a TDResult
run.monitors # the run's monitors by name
Run 'p1_mode0'
state done
excited p1:0
steps 701
stops at energy -40 dB or port signal -60 dB
energy -47.0 dB below peak
stop reason energy
started 2026-09-05 08:55:15
finished 2026-09-05 08:55:15
elapsed 0.2 s
dt 2.154e-12
checkpoint yes
The S-parameters of a scattering project come off the project itself,
derived from the stored port signals: proj.S("port1", "port3"),
proj.plot_s(...). The general time-domain analysis has no S-matrix;
its runs are read through run.result().
A project that is still being written#
Every energy sample the solver takes goes to disk at once, and the run
index is replaced atomically whenever a run starts or ends, so a
second process may open the project while the solver marches and see
its current state. Until the project is finished, the reader re-reads
the index whenever the file changed — typing proj again shows the
current step count, energy and elapsed time without any call on your
side. Once the project is done or aborted the parsed index is
kept; refresh() re-reads it, for a run that
was resumed elsewhere after you opened the project.
A running run’s elapsed counts the time since its current march
started on top of what earlier marches booked, so the clock in the
table keeps moving while the solver does, and its n_steps is the
step of the latest energy sample on disk until the march books its
final count.
Watching from another process#
Three tools turn “look again” into something a script or a notebook does on its own; the how-to Watching a simulation that is still running shows all three on one run.
watch() polls the store every interval seconds and reports every
change — a run starting or ending, a new energy sample, a status
change — until the project is finished (done, aborted or
stale), or until timeout seconds have passed. As a generator it
yields the project itself at every change; the first report comes at
once and the final state is always delivered:
for proj in mio.open_project("magic_tee").watch(interval=5):
print(proj) # the run table, as it moves
With on_change= the loop runs inside watch, which calls the
callable with the project at every change and returns the project
when the run is finished — the place for “redraw the figure”, “append
to a log”, “send a message”. Whatever the loop body does, it has to
print or plot what it wants seen: a bare expression inside a loop
displays nothing, in a notebook as anywhere else.
follow() is the zero-code form of the same loop: it shows the
project’s summary and run table at every change and replaces the
previous one instead of scrolling below it — in a notebook the cell
output is cleared and redrawn, on a terminal the table is redrawn
over its own lines. It blocks until the project is finished and
returns it:
proj = mio.open_project("magic_tee").follow(interval=5)
follow(plot=True) adds the energy plot below the table, redrawn
with it. A figure drawn inside a loop of your own would not show
until the cell ends — the notebook’s inline backend flushes a cell’s
figures when the cell is over — so follow renders the picture at
every change and replaces it with the table. A callable
plot(project, ax) draws a picture of your own into the fresh axes
it is given, the place for your own limits or an extra curve:
def draw(proj, ax):
proj.plot_energy(ax=ax)
ax.set_ylim(-80, 0)
proj = mio.open_project("magic_tee").follow(interval=5, plot=draw)
plot_energy() draws every run’s stored energy in dB below its peak,
one curve per run, with the energy criterion as a dashed line when the
runs share one. It is the same figure the progress line reports and
the table lists, and the same method exists on a single run and on
the in-RAM results. The axis runs from ten dB below the criterion
to +5 dB — lower only where a run ended deeper, −100 dB when there is
no criterion — because the first samples of a run are the empty grid,
which in dB is a plunge to −3000 that would squash the decay into a
line along the top; floor_db= pins the lower end by hand.
monitor() returns a notebook widget — the run table above the
energy plot — that a background thread refreshes every few seconds
until the project is finished. Left as the last expression of a
cell, the cell returns at once and the panel keeps moving while other
cells run; panel.stop() ends it early. It needs the jupyter
extra.
Polling is the deliberate choice. The store is written by another process, often on another file system; a subscription to file-system events would need a dependency, would miss events on network mounts, and could not tell a half-flushed HDF5 write from a whole one. A poll every few seconds sees everything, because every energy sample is flushed as it is taken and the index is replaced atomically.
Time#
Every run carries its wall clock: started and finished (UTC), and
elapsed, the wall time of the marching — summed over the marches
of a resumed run. The same three numbers are on every result object
and on the project, where they span its runs. The analysis call that
produced the runs, setup included, is a separate figure: the
finished in line the call prints, and the last call entry at the
top of the project’s summary. The two differ by the setup — the
stability time step and the port mode solves — which the progress
output accounts for phase by phase.
Checkpoints and resume#
A streamed run writes a resume checkpoint about eight times over its
expected length, once more when it finishes, and once on a graceful
stop (Ctrl-C). resume() continues a run from its last
checkpoint — to finish an aborted one, or to march a finished one
further under a deeper stop criterion — and appends to the same
streams, bit-exact with an uninterrupted run of the same length.
proj.checkpoint_state(name) (or run.checkpoint_state()) reads the
checkpoint back as a CheckpointState: a
read-only mapping with the solver’s own state layout — the completed
step, the peak energy and port signal, the field vectors, and a group
per boundary, port and monitor. Printing it shows the step and the
sizes, not the field vectors.
What prints#
A result, a run, a project or a checkpoint answers a bare name at the
prompt with what it is, how large it is and what state it is in — never
with its arrays. In a notebook the same summaries render as tables.
The arrays are one attribute away (result.energy_trace,
s_params.matrix); the summary is the part meant to be read.