Threading

Changed in version 1.5: The threading model changed substantially. In 1.4 the processing thread was spawned per operation (com.thread). On master there is one persistent worker thread for the whole application, fed by a FIFO job queue, with an explicit cancellation signal. This chapter documents that model. The authoritative source is the header comment in src/core/processing_thread.h — read it alongside this page.

This chapter covers who runs on which thread and how jobs are controlled. For the synchronisation primitives (which lock guards what, and the deadlock rules) see Locking.

Thread topology

  • GTK main thread — runs GApplication, widget callbacks, idle functions and the renderer. The only thread allowed to touch GTK.

  • The persistent processing worker thread (worker_thread_main in processing_thread.c) — one thread for the application lifetime. Jobs are serialised through a FIFO queue; never more than one job runs at a time. Every heavy operation runs here.

  • OpenMP teams — spun up inside a job for data-parallel loops (splitting an image across cores, or processing several sequence frames at once).

  • The script/CLI thread (com.script_thread) — reads a script and issues its commands, which in turn use the worker thread.

  • Per-Python-script worker threads — one C thread per running Python script, servicing that script's IPC connection (see Python integration).

Submitting a job

The entry point is start_in_new_thread (and its aliases start_in_reserved_thread and start_and_wait_from_main_thread). Submission is context-aware, decided by context_requires_wait() which is true when com.script or com.python_command is set:

  • From a script or Python-command thread — the call blocks until the job completes. This is what makes "one command finishes before the next starts" automatic in scripts, with no explicit join.

  • From any other thread (GUI main thread included) — the job is queued fire-and-forget; the caller returns immediately.

  • Re-entrant submission — if called from the worker thread itself (the already_in_a_thread pattern), the function is invoked directly and synchronously rather than re-queued. processing_in_worker_thread() reports whether you are on the worker thread.

waiting_for_thread retrieves a job's return value (blocking for a background caller, an instant cached fetch in the script/Python context, and an immediate 0 on the GTK main thread to avoid deadlocking the idle dispatcher).

Tip

For a GUI-thread caller that must wait for its result, use start_and_wait_from_main_thread, which pumps the main loop until the job finishes. The job it submits must not call execute_idle_and_wait_for_it internally, or the two mutually wait.

Cancellation

Cancellation is a separate signal from "is a job active?".

  • processing_request_cancel sets a sticky cancel_flag.

  • Worker hooks and long loops must poll processing_should_continue (declared in processing.h, worker-facing only) and bail out when it returns FALSE. The generic sequence worker already polls it each frame.

  • cancel_flag is cleared automatically when the next queued job starts, and also by reserve_thread.

Warning

Stale cancel flag on synchronous embedding. A caller that runs generic_sequence_worker directly with already_in_a_thread = TRUE (e.g. compositing alignment) polls processing_should_continue() but does not go through the queue that clears the flag. If a previous stop_processing_thread left cancel_flag set, the embedded worker aborts on frame 0. The fix — and the contract — is to call reserve_thread first: it atomically claims the slot and clears cancel_flag. This exact bug has shipped. See the comment on reserve_thread in processing_thread.c.

Reserving the thread

reserve_thread / unreserve_thread claim and release the "active slot" for short synchronous operations that must block concurrent submissions without actually running on the worker thread. reserve_thread is an atomic test-and-set returning FALSE if the slot is already busy. processing_is_job_active reports TRUE while a job runs or the slot is reserved — use it (not processing_should_continue) for GUI button-sensitivity checks and pre-command guards.

The Python thread claim

A running Python script that needs exclusive access claims the worker slot:

  • claim_thread_for_python drains the queue, waits for the active job to finish, then gates all further submissions. It returns 0 on success, 1 if busy/already reserved, and 2 if an image-processing dialog is open.

  • python_releases_thread releases the claim.

  • While claimed, GUI-initiated submissions are rejected.

Warning

claim_thread_for_python() waits on the queue condition while a job is active. Never call it from code that the active job can itself be waiting on — that is a circular wait.

Preview cancellation

cancel_and_wait_for_preview sets the cancel flag and then pumps the GTK main loop until the active job finishes. It is safe only for preview jobs that clean up with an asynchronous idle (siril_add_idle) and never call execute_idle_and_wait_for_it. A preview job that uses a synchronous idle would deadlock against the pumping loop.

Important

Preview-style jobs must use async idles only. This rule is stated in processing_thread.h and echoed in the deadlock catalogue in Locking.

The thread budget

The number of threads a job may use is a budget, not a free-for-all:

  • com.max_thread is the total budget; com.max_images / com.fftw_max_thread are related caps.

  • threading_type values (MULTI_THREADED / SINGLE_THREADED / an explicit N) express intent. check_threading resolves MULTI_THREADED to com.max_thread; limit_threading additionally caps the count to the number of work chunks (total_iterations / min_iterations_per_thread).

  • For sequences, compute_thread_distribution splits com.max_thread between the images processed in parallel (max_parallel_images) and the OpenMP subthreads each image's image_hook receives (nb_subthreads).

Warning

An image_hook that is handed nb_subthreads (sequence) or max_threads (single image) must not spawn more than that. A whole class of oversubscription bugs came from legacy single-image operations reading com.max_thread directly instead of obeying the worker's generic_img_args->max_threads. When you convert an operation to the generic worker, make it honour the passed-in count.

Per-format reentrancy decides whether OpenMP parallelism is enabled at all for a sequence: SER and fitseq are reentrant (parallel), film formats via FFMS2 are not (serial), and single-FITS depends on fits_is_reentrant(). See generic_sequence_worker.

The GTK rule

Only the main thread may touch GTK. GTK is not thread-safe, and Siril runs headless where GTK is absent entirely. Therefore:

  • Gather all GUI-derived parameters before submitting the job.

  • During processing, use only the thread-safe bridges: logging (siril_log_message), progress (via gui_iface.set_progress), and other gui_iface slots — never GTK directly.

  • After processing, GUI updates run on the main thread via idle functions registered with siril_add_idle (which no-ops headless). The generic workers handle this for you through args->idle_function.

Warning

Never call execute_idle_sync (or execute_idle_and_wait_for_it) from the GTK main thread — it waits on the main thread to run the idle, which is itself, so it deadlocks. The gui_iface implementations guard against this; see Locking.

Sequence parallelism philosophy

Siril's preferred mode of sequence parallelism is to work on several images at once. When there are not enough images to keep every thread busy (live stacking is the motivating case), spare threads are handed to each image's processing instead, as nb_subthreads passed to the image_hook. Prefer frame-parallelism; reach for per-frame parallelism only when frames are too big or too few to fill the thread pool one-per-thread.