Generic workers --------------- Siril has three generic workers that handle the boilerplate of running an operation — memory checks, threading, locking, progress, undo, idle dispatch — so your code only provides the algorithm (a *hook*) and a parameter struct. This chapter is the complete reference; :ref:`writingfeature:writing a feature` is the tutorial that walks you through using them. The three workers live in :file:`src/core/processing.c`, with their argument structs defined (and richly commented) in :file:`src/core/processing.h`: - ``generic_image_worker`` — one operation on one image. - ``generic_sequence_worker`` — one operation across a sequence. - ``generic_mask_worker`` — a mask-producing/mask-editing operation. .. important:: Your hook runs with the relevant fits **already locked on your behalf**. Do not take the fits ``rwlock`` on the fits you were handed — the lock is non-recursive and you would self-deadlock. See :ref:`locking:locking`. generic_image_worker ==================== Runs ``args->image_hook(args, fit, max_threads)`` on a single image and installs the result. Driven by ``struct generic_img_args``. Fields ****** - ``fit`` — the input image. When it is ``gfit``, the fast *swap path* is used (below); otherwise the *in-place path*. - ``image_hook`` — your algorithm: ``int (*)(struct generic_img_args *, fits *, int nb_threads)``, returning 0 on success. It must operate **only** on the ``fit`` it is handed. - ``mem_ratio`` — peak memory requirement as a multiple of the image size. When > 0 the worker runs a pre-flight check (``default_img_mem_hook``): ``mem_ratio × rx × ry × channels × bytes-per-sample`` against ``get_available_memory``, and aborts if it does not fit. See :ref:`memorymanagement:memory management`. - ``max_threads`` — thread budget for the hook; defaults to ``com.max_thread`` when left < 1. **The hook must not exceed it** (see :ref:`threading:threading`). - ``log_hook`` — ``gchar *(*)(gpointer user, log_hook_detail)`` producing a ``SUMMARY`` string (the undo label and, in condensed form, feeds it) and a ``DETAILED`` string (logged, and written as a FITS HISTORY card). If NULL, ``description`` is used for both. - ``description`` — terse progress text. - ``verbose`` — extra logging (forced on unless ``for_preview``). - ``command`` / ``command_updates_gfit`` — set **only** from :file:`command.c`. ``command`` marks a command-invoked op; ``command_updates_gfit`` says the command needs ``gfit`` refreshed. GUI operations leave these FALSE and update through an idle instead. - ``user`` — operation-specific data. **By convention its first member must be a destructor**, called by ``free_generic_img_args`` (see below). - ``for_preview`` — a preview op: no undo state is saved. - ``for_roi`` — the op is applied to an ROI only. See :ref:`roi_processing:region of interest processing`. - ``skip_generic_undo`` — the worker creates no undo state; the caller owns undo. Used by operations that manage their own undo (e.g. stretches) and by measurement-only commands (``stat``, ``bg``, ``bgnoise``, …) for which undo would be meaningless. - ``mask_aware`` — the op respects an active mask (the worker blends the result through the mask). - ``has_mask`` — internal; captured from ``fit->mask`` before the writer unlock. - ``idle_function`` — a custom completion idle (GUI path). Should be NULL when called from :file:`command.c`. - ``populate_roi_on_complete`` — **deprecated and ignored**; the worker now repopulates the ROI itself. Retained only so old call sites compile. The two execution paths *********************** The worker chooses its strategy from whether ``args->fit == gfit``: **Swap path** (``fit == gfit``). The worker takes a brief reader lock on ``gfit`` and deep-copies it into a private ``orig`` (``CP_DEEPCOPY | CP_ALLOC``, so WCS-dependent hooks see the astrometry), then **drops the lock**. Your hook runs on ``orig`` with **no gfit lock held**, so the GUI keeps reading ``gfit`` freely (motion handlers, ROI fills). On success the worker takes gfit's writer lock for the microseconds it takes to ``fits_swap_all_except_rwlock(gfit, orig)``, installing the result; ``orig`` then holds the pre-operation image — exactly what ``undo_save_state`` wants. An aborted op skips the swap, leaving ``gfit`` pristine. This is why the GUI stays responsive during long single-image operations. **In-place path** (``fit`` is ``gui.roi.fit`` or any other fits). The worker holds ``args->fit``'s **writer lock for the whole hook** (the legacy pattern). Because the lock is not on ``gfit``, it does not block GTK main-thread readers. Undo, mask, HISTORY ******************* - **Undo** is created only on the swap path, and only when not previewing, not scripted, and ``skip_generic_undo`` is unset. When no undo state is created but the image changed (``command_updates_gfit``), a fallback FITS HISTORY card is appended so provenance is not lost. - **Mask blending**: when ``mask_aware`` and the fit has an active mask, the worker blends the hook's output against the pre-op pixels through the mask (``blend_fits_with_mask``, supporting 8/16/32-bit masks). See :ref:`masks:masks`. - After installing the result the worker calls ``notify_gfit_data_modified`` — **outside** the writer-lock window (see the deadlock note in :ref:`locking:locking`) — which invalidates stats, recomputes the histogram and remaps the viewports. Idle dispatch and ownership *************************** Who frees ``args`` and how completion is dispatched depends on the context: - **command + headless** — the worker calls ``stop_processing_thread`` and ``free_generic_img_args`` inline. - **command + GUI** — dispatched **synchronously** via ``gui_iface.execute_idle_sync`` (``end_generic_image_update_gfit`` when the command updates gfit, else ``end_generic_image``). - **GUI op** — dispatched asynchronously via ``siril_add_idle``, using ``args->idle_function`` if provided, else ``end_generic_image_update_gfit``. In every case the idle (or the inline headless path) frees ``args``. **Nothing after the idle is registered may touch** ``args`` — the worker copies the fields it still needs into locals first. generic_sequence_worker ======================= Runs an operation over every (filtered) frame of a sequence, driven by ``struct generic_seq_args``. Call order per run: 1. ``compute_mem_limits_hook`` (default ``seq_compute_mem_limits``) → ``max_parallel_images``; then ``compute_thread_distribution`` assigns ``nb_subthreads`` per parallel image. This step only runs in builds with OpenMP (it sits inside ``#ifdef _OPENMP``), and only when the caller has not already set ``max_parallel_images``. 2. ``prepare_hook`` (default ``seq_prepare_hook`` when there is output, which also opens the writer via ``seq_prepare_writer``). 3. ``compute_size_hook`` (if ``has_output``) → disk-space check via ``test_available_space``. 4. Per frame, in parallel: optional ``image_read_hook`` (skip loading pixels when only metadata is needed) → read (full or partial) → ``image_hook(args, out_frame, in_index, fit, area, nb_subthreads)`` → ``save_hook`` (default ``generic_save``) when ``has_output``. 5. ``finalize_hook`` (default ``seq_finalize_hook``) — closes the writer; called even on the error path. 6. Idle: ``idle_function`` or ``end_generic_sequence``. Key fields ********** - ``seq`` — the sequence. - ``filtering_criterion`` / ``filtering_parameter`` — which frames to include (helpers in :file:`sequence_filtering.h`, e.g. ``seq_filter_included``); ``nb_filtered_images`` caches the count for smoother progress. - ``partial_image`` / ``area`` / ``layer_for_partial`` / ``regdata_for_partial`` — read only a region; when ``regdata_for_partial``, the area is moved by the registration transform per frame. - ``has_output`` — the op writes a new sequence. Then ``new_seq_prefix``, ``load_new_sequence``, ``output_type``, ``upscale_ratio``, and the SER/fitseq forcing flags (``force_ser_output`` / ``force_fitseq_output``) apply. Output to a SER or fitseq container engages the **seqwriter**. - ``stop_on_error`` — TRUE aborts the whole run on a frame failure; FALSE unselects the failing frame and continues. - ``user`` — operation data, by convention freed in ``finalize_hook``. - ``already_in_a_thread`` — the run is embedded in an existing thread; the idle function is suppressed and cleanup runs inline. See the reserve-thread contract in :ref:`threading:threading`. - ``parallel`` / ``max_parallel_images`` — enable and cap frame-parallelism. - ``lock`` — an ``omp_lock_t`` the worker initialises for hooks that need cross-thread accumulation (use it in ``finalize``-bound shared state). Parallelism and the seqwriter ***************************** Frame-parallelism is enabled only when ``parallel`` is set, more than one image fits in memory, and the format is reentrant: SER and fitseq yes, FFMS2 film formats no, single-FITS per ``fits_is_reentrant``. When writing a SER or fitseq output, the seqwriter provides backpressure: ``seqwriter_wait_for_memory`` blocks a producer until a write block frees up, bounded by the limit ``seq_prepare_writer`` sets. Memory sizing splits the budget between the compute side and the writer side (``for_writer`` argument to ``compute_mem_limits_hook``). See :ref:`memorymanagement:memory management`. generic_mask_worker =================== Runs ``args->mask_hook(args)`` to create or edit a fits's mask, driven by ``struct generic_mask_args`` (``fit``, ``mask_hook``, ``mem_ratio``, ``log_hook``, ``description``, ``verbose``, ``command``, ``user`` with a leading destructor, ``mask_creation``, ``max_threads``). .. versionchanged:: 1.5 Unlike ``generic_image_worker``, this worker currently holds ``args->fit->rwlock`` as a **writer for the entire duration** of the ``mask_hook`` — there is no swap path yet (a code TODO in :file:`processing.c`). It is fine while mask ops stay short; a long-running mask hook would block GUI readers. When ``args->fit == gfit`` and this is not a command, the worker saves an undo state before running the hook. ``mask_creation`` TRUE means the mask is set active on completion (``set_mask_active``). Completion dispatches ``end_generic_mask`` (which calls ``gui_iface.on_mask_state_changed``), inline in the command+headless case. Gotchas ======= Each of these has caused, or is designed to prevent, a real bug. - **Never touch** ``args`` **after submission.** The idle (or headless path) frees it. Inside the worker, copy any field you need past the idle into a local first — the workers do exactly this. - **The destructor-first convention.** ``free_generic_img_args`` (and the mask equivalent) call ``destroy_any_args(args->user)``, which invokes the destructor stored as the **first member** of your ``user`` struct. Allocate ``user`` with ``calloc`` and set that destructor. Do **not** also free ``user`` yourself after a failed ``start_in_new_thread`` — that double frees. - ``mask_aware = FALSE`` **with an active mask** installs the hook's output **without** blending. If the op should respect masks and you forget the flag, the result is silently wrong. - ``already_in_a_thread = TRUE`` **suppresses the idle entirely** — the caller is responsible for cleanup, and (for sequence embedding) must ``reserve_thread`` first to clear a stale cancel flag. - **A failed** ``image_hook`` **with an active seqwriter** does not currently notify the writer of the failed frame (a known TODO in the worker). Choose ``stop_on_error`` deliberately for output sequences. - **Sequence hooks must be re-entrant** — they run concurrently across frames. Use ``args->lock`` for any shared accumulation destined for ``finalize``. - ``populate_roi_on_complete`` **is deprecated** — the worker repopulates the ROI automatically on the swap path; leave the field alone.