Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 11 additions & 12 deletions wiki/cascade-cms-wiki/docs/advanced/cpu-intensive.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@ By default `.then()` callbacks run on the async event loop, which is fine for I/

## When to Use `ProcessPoolExecutor`

- Image resizing, compression, or optimization
- Parsing large XML/JSON structures or heavy data transformation
- Running intensive regular expressions across large HTML bodies
- Light callbacks like simple string replacements or metadata dictionary updates do not need it and perform better on the default thread pool or event loop.
- Image resizing and optimization
- Large-file parsing and processing
- Heavy data transformations
- Bulk regular expression operations on large HTML bodies
- Light callbacks (like title string replacement or a simple metadata dictionary update) do not need it and perform better with the default `ThreadPoolExecutor`.

---

Expand All @@ -18,15 +19,13 @@ By default `.then()` callbacks run on the async event loop, which is fine for I/
```python
from concurrent.futures import ProcessPoolExecutor
from os import cpu_count
from cascade_cms import Cascade

env = {"SERVER": "prod", "API_KEY": "secret", "CASCADE_URL": "https://cascade.example.com"}

# Define a top-level function so it can be pickled
def optimize_image(asset):
# CPU-bound image processing work here
# Perform heavy CPU-bound processing
return asset

with Cascade(env, {}) as cascade:
with Cascade(env_vars) as cascade:
with ProcessPoolExecutor(max_workers=cpu_count()) as executor:
cascade.operations.read(id).then(optimize_image)
results = cascade.submit_requests(executor=executor)
Expand Down Expand Up @@ -58,12 +57,12 @@ cascade.operations.read(id).then(transform)

## `ProcessPoolExecutor` vs `ThreadPoolExecutor`

`ProcessPoolExecutor` provides true parallelism across multiple CPU cores in separate memory spaces, but requires all callback functions and data to be picklable and incurs inter-process communication overhead. `ThreadPoolExecutor` uses shared memory with minimal overhead and no pickling restrictions, but remains limited by Python's Global Interpreter Lock (GIL) for CPU-bound tasks. Use `ProcessPoolExecutor` for heavy CPU computation, and stick with `ThreadPoolExecutor` (the default) for I/O-bound work.
`ProcessPoolExecutor` provides true parallelism by running tasks in separate processes with separate memory spaces, but requires all callbacks and data to be picklable and incurs higher startup overhead. `ThreadPoolExecutor` (the default) shares memory and has lower overhead, making it ideal for I/O-bound work, but remains constrained by the Global Interpreter Lock (GIL) for CPU-bound tasks. Choose `ProcessPoolExecutor` for heavy CPU-bound tasks and `ThreadPoolExecutor` for general I/O-bound operations.

---

## Performance Considerations

Spawning and managing worker processes carries an initialization overhead, meaning extremely fast callbacks may run slower in a process pool than synchronously due to serialization costs. Additionally, offloading CPU work via the executor does not change the network concurrency rules; the driver's underlying `MAX_REQUESTS` semaphore still governs HTTP request limits.
Spawning worker processes has an upfront startup cost, so for very fast callbacks the IPC and pickling overhead may outweigh the concurrency benefit. Additionally, the driver's underlying request semaphore (`MAX_REQUESTS`) still governs HTTP concurrency independently of whichever executor you pass for your callbacks.

<!-- synthesized-for: 3.1.1 -->
<!-- synthesized-for: 3.9.2 -->
4 changes: 2 additions & 2 deletions wiki/cascade-cms-wiki/docs/advanced/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Advanced

Welcome to the advanced power user guide for tuning and scaling your application. This section covers configuration and performance topics that go beyond the defaults, including caching behavior, debug logging, and CPU-intensive callback patterns.
Welcome to the advanced section, designed for power users who want to push the library beyond its default settings. Here, you will find in-depth guides on configuring debug logging and handling CPU-intensive callback patterns efficiently.

---

Expand All @@ -11,4 +11,4 @@ Welcome to the advanced power user guide for tuning and scaling your application
| [Logging & Debugging](logging.md) | Enabling debug mode, configuring the debug dict, reading log output, and walking through a sample logfile |
| [CPU-Intensive Tasks](cpu-intensive.md) | When to use `ProcessPoolExecutor`, pickling constraints, and performance trade-offs vs `ThreadPoolExecutor` |

<!-- synthesized-for: 3.1.1 -->
<!-- synthesized-for: 3.9.2 -->
36 changes: 17 additions & 19 deletions wiki/cascade-cms-wiki/docs/advanced/logging.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,40 @@
# Logging & Debugging
# Logging & Database

Operation logger owns all console, logfile, and (verbose mode) request/response file output for the cascade_cms library. Two output modes exist: normal mode for everyday use and debug mode for diagnosing failures, controlled solely by whether `debug_config` is `None`. Both produce logfiles; debug mode adds a verbose nested call-chain log alongside a quieter console output.
The `cascade_cms` library provides two distinct output modes: a normal mode for everyday use and a debug mode for diagnosing failures. Both modes produce logfiles, but debug mode adds a verbose nested call-chain log alongside a quieter console output.

---

## Normal Mode Output

Normal mode produces minimal console output and a simple logfile. It logs lifecycle markers such as `[INIT]`, `[RUNNING]`, `[DONE]`, and `[EXIT]`, along with batch tallies and operation progress lines. The normal logfile is named `{SERVER}_{timestamp}.log` and records one line per chain.
Normal mode uses a minimal console and simple logfile to record session lifecycle markers (`[INIT]`, `[RUNNING]`, `[DONE]`, `[EXIT]`), per-operation progress lines, and a logfile named `{SERVER}_{timestamp}.log` with one line per operation.

### Normal Logfile Format

```text
[INIT]: Connecting to myserver.com
[RUNNING]: myscript.py
[INIT]: Connecting to my-server
[RUNning]: script.py
(uuid_or_path, asset_type) OP1 -> fn_name: Type -> ...
1/1 succeeded
[DONE]: 1 assets processed in 0.5s
[EXIT]: Disconnecting from myserver.com
[DONE]: 1 assets processed in 1.0s
0 failed, 1 succeeded
[EXIT]: Disconnecting from my-server
```

---

## Enabling Debug Mode

```python
wrapper = CascadeWrapperBase("myserver.com", debug_config={
"log_dir": "./logs",
"show_network_headers": True
})
wrapper = CascadeWrapperBase(
server="my-server",
auth=("user", "pass"),
debug_config={"log_dir": "./logs"}
)
```

---

## Debug Configuration Options

Recognized `debug_config` keys control the log directory and network header visibility.

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| log_dir | str | "./logs" | Directory for the logfile and (verbose mode) request/response JSON files. |
Expand All @@ -45,14 +44,13 @@ Recognized `debug_config` keys control the log directory and network header visi

## Debug Logfile Format

The debug logfile uses the naming convention `{SERVER}_debug_{timestamp}.log`. It features a quiet console, a verbose logfile, and per-request JSON files. There is no longer a separate "log operations vs. callbacks vs. responses" toggle — `_is_debug` is the single on/off switch for verbose behavior.
Debug mode uses a quieter console and a verbose logfile named `{SERVER}_debug_{timestamp}.log` alongside per-request JSON files. The two modes are controlled solely by whether `debug_config` is `None`; `_is_debug` serves as the single on/off switch for verbose behavior.

### Sample Debug Log

```text
>>>> START REQUEST <<+
>>>> START REQUEST <<<<
(uuid_or_path, asset_type) OP1 -> fn_name: Type -> ...
[GET] https://myserver.com/api/v1/read/site/path
1/1 succeeded
>>>> END REQUEST <<<<
```
Expand All @@ -61,6 +59,6 @@ The debug logfile uses the naming convention `{SERVER}_debug_{timestamp}.log`. I

## Interpreting Errors in Debug Mode

Error lines in debug mode utilize a `v` marker followed by an `!ERROR:` block indicating the failure at the specified step index. Chain-level failures go through `flush_chain_error` which has step-index context. `log_cascade_error` provides a thin wrapper to log a `CascadeError` (API-level failure) outside of chain context, while `log_python_error` logs an unhandled Python exception with traceback, file name, and line number.
Error output utilizes a `v` marker and `!ERROR:` block positioned beneath the failing step. For NETWORK/LIBRARY failures, `[NETWORK]` or `[CASCADE-REST-CMS]` is prepended to the `!ERROR:` message, while CASCADE and CALLBACK failures get no prefix. The alignment column is computed by reconstructing the line's literal text up to (not including) the failing label and measuring its length, where each ` -> ` separator is counted as 4 characters.

<!-- synthesized-for: 3.1.1 -->
<!-- synthesized-for: 3.9.2 -->
213 changes: 54 additions & 159 deletions wiki/cascade-cms-wiki/docs/core-concepts/index.md
Original file line number Diff line number Diff line change
@@ -1,197 +1,92 @@
# Core Concepts

## The `Cascade` class
This page unpacks the mental model behind Cascade CMS Python: a context manager wrapper (`Cascade`) coordinates a fluent builder (`Operations`) that constructs singly-linked operation chains (`OperationChain`), executed concurrently via `submit_requests()`.

`Cascade` is the single entry point for all library interactions. Open a
session with it as a [context manager](https://docs.python.org/3/reference/datamodel.html#context-managers):

```python
from cascade_cms.wrapper import Cascade, EnvironmentVars

env: EnvironmentVars = {
"SERVER": "my-site",
"API_KEY": "your-api-key",
"CASCADE_URL": "https://your-cascade-instance.com",
}
---

with Cascade(env) as cascade:
# queue operations here
...
```
## Basic Operation Calls

Previously known as `CascadeWrapperBase`; the alias still works but `Cascade`
is the current name.
Every script follows the same skeleton: open the wrapper as a context manager, queue operations on `cascade.operations`, chain callbacks with `.then()`, then call `submit_requests()` once to execute all chains concurrently. Individual chains run their steps strictly in order, stopping at the first failure without affecting others.

### Constructor
### Example

```python
Cascade(
environmentVariables: EnvironmentVars,
debug: dict[str, Any] | None = None,
*,
exit_on_failure: bool = True,
log_dir: str | os.PathLike | None = None,
)
# Open the Cascade context manager with your environment variables
with Cascade(env_vars) as cascade:
# Queue a read operation on an identifier
cascade.operations.read(identifier)

# Submit all queued operations concurrently
results = cascade.submit_requests()

# Check if the result is a CascadeError
for result in results:
if isinstance(result, CascadeError):
print(f"Error: {result.message}")
```

`EnvironmentVars` is a `TypedDict` with three required keys:

| Key | Description |
|---|---|
| `SERVER` | Label used in log output (not a network address) |
| `API_KEY` | Cascade REST API bearer token |
| `CASCADE_URL` | Base URL of your Cascade CMS instance |

`exit_on_failure=True` (default) ends the script non-zero on any failure.
Pass `False` for long-lived processes like the MCP server.

---

## Operations

`cascade.operations` is the entry point for all CMS operations. It returns
an `Operations` builder that accepts the first call in a chain:
### Expected Output

```python
with Cascade(env) as cascade:
chain = cascade.operations.read("abc123")
Asset(
_asset_type='page',
_data={
'id': '1234567890abcdef1234567890abcdef',
'name': 'index',
'path': '/index',
'displayName': 'Home Page',
...
}
)
```

Available operations mirror the Cascade REST API:

| Operation | Description |
|---|---|
| `.read(id)` | Read one asset |
| `.edit(payload)` | Edit an asset via callback or direct payload |
| `.publish(id)` | Publish an asset |
| `.delete(id)` | Delete an asset |
| `.copy(id, dest)` | Copy an asset |
| `.move(id, dest)` | Move an asset |
| `.checkIn(id)` | Check in an asset |
| `.checkOut(id)` | Check out an asset |
| `.search(query)` | Search assets |
| `.create(asset)` | Create a new asset |

See the [API Reference](../operations/operations.md) for full signatures.

---

## Operation chains

Each operation call returns an `OperationChain`, not a result. Nothing
executes until you call `.submit_requests()`. This design lets you compose
multi-step workflows before any network requests are made:

```python
from cascade_cms.cmstypes import Asset

def set_searchable(asset: Asset) -> Asset:
asset.searchable = True
return asset

with Cascade(env) as cascade:
results = (
cascade.operations
.read("abc123")
.edit(set_searchable)
.publish("abc123")
.submit_requests(Asset)
)
```

The type argument to `.submit_requests()` is for the type checker. `Asset`
covers most cases; use a more specific type when the operation returns one.

---
## Payload Models

## `edit()` and callbacks
Payload models are strongly typed Pydantic objects that pair with specific operations to ensure the API endpoint receives the expected fields. They provide compile-time safety and self-documenting structure compared to unstructured dictionaries.

`edit()` accepts a callable, a single asset, or a list of assets. The
callable form is the most common in practice:
### Example: `SearchInformation` paired with `search`

```python
def update_keywords(asset: Asset) -> Asset:
asset.keywords = "updated"
return asset

cascade.operations.edit(update_keywords)
```

The callback receives the current asset and must return the modified asset.
The identifier is derived from the asset's own `id`/`path`/`siteName` fields
— there is no separate identifier argument on `edit()` in 3.9.1.

---

## Batch execution and `ChainGroup`

Passing a list to operations that accept identifiers — `read`, `delete`,
`publish`, etc. — creates one independent `OperationChain` per identifier,
returned as a `ChainGroup`:
# Construct a SearchInformation payload specifying site name, terms, fields, and types
payload = SearchInformation(
siteName="Default",
searchTerms="blog",
searchFields=["name"],
searchTypes=["page"]
)

```python
with Cascade(env) as cascade:
results = (
cascade.operations
.read(["id1", "id2", "id3"])
.submit_requests(Asset)
)
# Pass the payload to the search operation
cascade.operations.search(payload)
```

Failures in one chain do not cancel others. The returned `ChainResults`
covers all chains in the group.
The `SearchInformation` model enforces valid parameters before any HTTP request leaves the library, guaranteeing that required fields are present and properly aliased (`siteName`, `searchTerms`, `searchFields`, `searchTypes`). Other operations follow this exact pattern using corresponding payload models like `deleteParameters`, `auditParameters`, and `copyParameters`.

---

## Result handling
## CPU-Intensive Operations

`.submit_requests()` returns `ChainResults[T]` — a list subclass with
`.success` and `.failed` properties. See [Result Handling](../getting-started/result-handling.md)
for full details.
For operations involving heavy computation in `.then()` callbacks — image processing, data transformation, bulk string manipulation — offload work to a `ProcessPoolExecutor` rather than running it on the async event loop.

```python
results = cascade.operations.read("abc123").submit_requests(Asset)
from concurrent.futures import ProcessPoolExecutor
from os import cpu_count

for asset in results.success:
print(asset.path)

for failure in results.failed:
print(f"[{failure.category}] {failure.message}")
with ProcessPoolExecutor(max_workers=cpu_count()) as executor:
cascade.operations.read(id).then(optimize_image)
results = cascade.submit_requests(executor=executor)
```

---

## Script logging

`script_log.note()` writes annotated log lines from within a script:

```python
from cascade_cms.utils import script_log

with Cascade(env) as cascade:
script_log.note("Starting run")
results = cascade.operations.read("abc123").submit_requests(Asset)
script_log.note(f"Done — {len(results.success)} succeeded")
```
!!! note "Module-level functions only"
Callbacks passed to `ProcessPoolExecutor` must be defined at module level — lambdas and nested functions are not picklable and will raise at runtime.

Calls outside the `with` block drop silently with a `RuntimeWarning`.
API keys are automatically redacted from log output.
See [Advanced: CPU-Intensive Tasks](../advanced/cpu-intensive.md) for full configuration details, performance trade-offs, and `ThreadPoolExecutor` comparison.

---

## Identifiers

The `utils.identifiers` module (added in 3.8.3) coerces raw Cascade
identifier dicts to `IdentifierType`:

```python
from cascade_cms.utils.identifiers import to_identifier, to_identifiers

identifier = to_identifier({"id": "abc123", "type": "page", "path": None})
identifiers = to_identifiers([{"id": "a"}, {"id": "b"}])
```
## Next Steps

Use these when processing raw API responses before passing identifiers to
operations.
Ready to go deeper? The [Advanced](../advanced/index.md) section covers configuration topics for power users: debug logging and CPU-intensive workload patterns.

<!-- synthesized-for: 3.9.1 -->
<!-- synthesized-for: 3.9.2 -->
Loading