Fix background assembly waiting after task result expiration

[nonprofittechy]

Fixes #1069.

Returning to a completed interview after Celery expires its result currently sends users back to the document-generation waiting screen indefinitely. This change adds saved completion gates that recognize the files persisted by the background response callback, including when the user leaves before generation finishes.

The standard task-start blocks clear the saved files and completion gates when explicitly reconsidered. The runnable example uses the new gates and includes a regenerate action. Existing interviews must replace their direct .ready() checks; upgrading AssemblyLine alone cannot change those interview conditions.

Validation: 51 automated tests passed, plus localhost expiration, reopen, leave-during-generation, and regeneration checks for PDF, DOCX, and preview. The full example interview also passed. Details and concrete migration examples follow; no documentation file is included in this PR.

---

Background assembly that survives task-result expiration

Use a saved completion variable in your interview order, instead of calling a
Celery task's .ready() method. Celery's result is temporary; the files saved to
the interview by AssemblyLine's completion callback are durable.

Standard workflow

After collecting all answers needed by your attachments, use one of these
variables:

Output	Interview-order variable	Task to reconsider when regenerating
Preview PDF	al_user_bundle.preview_ready	al_user_bundle.generate_preview_task
Final PDFs	al_user_bundle.downloads_ready	al_user_bundle.generate_downloads_task
Final PDFs and editable DOCX files	al_user_bundle.downloads_with_docx_ready	al_user_bundle.generate_downloads_with_docx_task

For example, an interview with preview and editable downloads can end this way:

mandatory: True
code: |
  # Your interview's existing question/logic blocks go before these lines.
  al_user_bundle.preview_ready
  preview_screen
  al_user_bundle.downloads_with_docx_ready
  download_screen
---
continue button field: preview_screen
question: |
  Preview your documents
subquestion: |
  ${ al_user_bundle._preview_file }
---
event: download_screen
question: |
  Download your documents
subquestion: |
  ${ al_user_bundle.download_list_html(use_previously_cached_files=True, include_full_pdf=True) }

The standard gates start the task once and show the standard waiting screen until
its callback saves the files. They then save True to the completion variable.
They do not ask Celery whether an old task is still ready. If the user closes the
browser while generation is running, the callback still saves the files. On
return, the gate recognizes those files even if Celery has already expired its
result. An empty bundle is also a completed result; the check uses defined(),
not the truthiness of the saved files.

For a runnable example with attachments, see
test_aldocument_background_assembly.yml.

Regenerate after editing answers

Once the user has finished editing and all attachment inputs are defined again,
explicitly reconsider the appropriate task. For example, an action linked from a
completed download screen can use:

event: regenerate_downloads
code: |
  reconsider("al_user_bundle.generate_downloads_with_docx_task")

Link to that action with:

${ action_button_html(url_action('regenerate_downloads'), label='Regenerate documents') }

The task-start block clears the saved download files and both download completion
variables before queuing a new task. The interview order then reaches
al_user_bundle.downloads_with_docx_ready again and waits for the replacement
files. Reconsidering al_user_bundle.generate_preview_task similarly clears the
preview file and preview_ready. If your preview screen has a continue-button
variable such as preview_screen, undefine that variable too when you want the
user to see the preview screen again.

Reconsidering only the completion variable rechecks the current saved result;
it does not regenerate documents. Do not place the task's reconsider() in a
mandatory block, because that would restart assembly on every reload. Use one
final-download mode per bundle at a time, and finish an existing task before
starting another for that same bundle. The PDF and DOCX modes share a saved file
cache; starting either mode invalidates the other mode's task handle and gate.

If your interview separately assembled attachments in the foreground before an
edit, invalidate those attachment variables and any document/bundle caches as
part of your existing edit workflow. Restarting the background task clears the
background result, not every possible foreground attachment cache.

Updating an existing interview

Replace this older pattern:

code: |
  if not al_user_bundle.generate_downloads_with_docx_task.ready():
    al_download_waiting_screen

with a reference in your interview order:

code: |
  al_user_bundle.downloads_with_docx_ready

Use the other gates in the table for PDF-only downloads or previews. Keep
use_previously_cached_files=True on the download screen. Existing saved sessions
with the standard callback's cached files can use the new gates without running
their expired tasks again. Updating AssemblyLine alone cannot rewrite an
interview's existing .ready() condition.

Custom background actions

A custom action should save its result with background_response_action() and
use an intermediate completion block that checks for the saved result. Do not
save a one-time False result while waiting. For example:

mandatory: True
code: |
  report_complete
  show_report
---
code: |
  undefine("report_result", "report_complete")
  report_task = background_action("build_report")
---
code: |
  report_task
  if not defined("report_result"):
    report_waiting
  report_complete = True
---
event: build_report
code: |
  # Replace this expression with your computation using already-defined inputs.
  result = {"message": "Report finished"}
  background_response_action("save_report", result=result)
---
event: save_report
code: |
  report_result = action_argument("result")
  background_response()
---
event: report_waiting
question: |
  Preparing your report
reload: True
---
event: show_report
question: |
  Your report
subquestion: |
  ${ report_result["message"] }

To regenerate, call reconsider("report_task") from an explicit action after any
required edits. This invalidates both the old result and the intermediate gate.
See docassemble's background action documentation
for how the callback saves changes to the interview.

Expiration and verification

Celery defaults to retaining results for one day. The localhost docassemble
1.10.10 installation exposes celery result retention days; its
config_worker.py converts that value to seconds for Celery's result_expires.
Older versions may not expose this setting. Increasing retention merely delays
the failure of an interview that repeatedly checks .ready().

The issue #1069 regression was tested on localhost using uniquely named interview
fixtures and fresh sessions. Only the fixtures' exact celery-task-meta-<UUID>
Redis keys had their TTLs shortened to two seconds. No global configuration,
worker restart, queue purge, or changes to another test's result keys were needed.

For each of PDF downloads, DOCX downloads, and previews, verify:

1. Finish using the old .ready() condition; expire its result; reopen the
   session. It returns to the waiting screen despite having saved files.
2. Finish using the new gate; expire its result; reopen. Saved files remain
   downloadable while .ready() is False, without a replacement task.
3. Leave at the waiting screen; let the callback finish; expire its result;
   reopen. The gate recognizes the saved files on the first return.
4. Explicitly reconsider the task after changing an input. The waiting screen
   returns, a new task runs, and the downloaded PDF contains the updated input.

The focused automated tests in test_background_assembly.py cover expired-task
independence, waiting until the callback saves a result, empty results, and
invalidation when regeneration starts.

Observed on localhost on September 26, 2026 (docassemble 1.10.10):

Check	PDF downloads	DOCX downloads	Preview
Old condition returns to waiting after expiration	Reproduced	Reproduced	Reproduced
New gate reopens completed session after expiration	Passed	Passed	Passed
New gate reopens after leaving during assembly and expiring result	Passed	Passed	Passed
Regeneration waits and produces updated PDF content	Passed	Passed	Passed

The DOCX downloads were also fetched after expiration and their document text
checked, including the changed input after regeneration. The original TTLs of
the test results were 86,392–86,400 seconds before being shortened to two seconds.
The focused tests plus the existing test_al_document.py suite passed (51 tests).
An isolated copy of the runnable example above also passed its full preview,
download, and regenerate flow on localhost.

[Copilot]

Copilot review overview

🟢 Approval recommended

No unresolved blocking issues were identified.

Review effort: Lite
Findings: None

What changed in this PR

Fixes background document assembly getting stuck after Celery task-result expiration by using durable completion gates.

Changes:

• Adds PDF, DOCX, and preview completion gates.
• Invalidates cached results during regeneration.
• Updates the example interview and adds focused tests.

File	Description
docassemble/​AssemblyLine/​test_background_assembly.py	Tests expiration-safe completion and regeneration.
docassemble/​AssemblyLine/​data/​questions/​test_aldocument_background_assembly.yml	Demonstrates durable gates and regeneration.
docassemble/​AssemblyLine/​data/​questions/​al_document.yml	Implements completion gates and cache invalidation.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[rajeswaripedaballi]

Looks good to me, but one thing I noticed while testing is I made the job crash on purpose, and the page stayed on the waiting screen and kept refreshing with the error showing in the worker log. I think with the old check a crashed job would still let the person reach the download screen, so I wanted to know if this is intended?

[rajeswaripedaballi]

I tested the crash case again, the page now shows an error with Try again and it makes the documents. Looks good to me.