Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 13 additions & 15 deletions wiki/cascade-cms-wiki/docs/advanced/cpu-intensive.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@ By default `.then()` callbacks run on the async event loop, which is fine for I/

## When to Use `ProcessPoolExecutor`

- Image resizing, compression, or optimization
- Parsing large XML/JSON structures or heavy data transformation
- Running intensive regular expressions across large HTML bodies
- Light callbacks like simple string replacements or metadata dictionary updates do not need it and perform better on the default thread pool or event loop.
- Image resizing, conversion, or heavy graphical optimization.
- Parsing and manipulating very large XML/HTML payloads or files.
- Executing complex data transformations or intensive calculations on asset metadata.
- Running bulk regular expressions across large text bodies.
- Light callbacks (such as a simple title string replacement or metadata dictionary update) do not need `ProcessPoolExecutor` and run efficiently on the default executor or event loop.

---

Expand All @@ -18,18 +19,15 @@ By default `.then()` callbacks run on the async event loop, which is fine for I/
```python
from concurrent.futures import ProcessPoolExecutor
from os import cpu_count
from cascade_cms import Cascade

env = {"SERVER": "prod", "API_KEY": "secret", "CASCADE_URL": "https://cascade.example.com"}

# Define a module-level helper function for the callback
def optimize_image(asset):
# CPU-bound image processing work here
# Perform CPU-intensive processing here
return asset

with Cascade(env, {}) as cascade:
with ProcessPoolExecutor(max_workers=cpu_count()) as executor:
cascade.operations.read(id).then(optimize_image)
results = cascade.submit_requests(executor=executor)
with ProcessPoolExecutor(max_workers=cpu_count()) as executor:
cascade.operations.read(id).then(optimize_image)
results = cascade.submit_requests(executor=executor)
```

---
Expand Down Expand Up @@ -58,12 +56,12 @@ cascade.operations.read(id).then(transform)

## `ProcessPoolExecutor` vs `ThreadPoolExecutor`

`ProcessPoolExecutor` provides true parallelism across multiple CPU cores in separate memory spaces, but requires all callback functions and data to be picklable and incurs inter-process communication overhead. `ThreadPoolExecutor` uses shared memory with minimal overhead and no pickling restrictions, but remains limited by Python's Global Interpreter Lock (GIL) for CPU-bound tasks. Use `ProcessPoolExecutor` for heavy CPU computation, and stick with `ThreadPoolExecutor` (the default) for I/O-bound work.
`ProcessPoolExecutor` provides true parallelism across multiple CPU cores by utilizing separate memory spaces, making it ideal for CPU-bound tasks, though it requires all callbacks and arguments to be picklable and incurs higher task overhead. `ThreadPoolExecutor` uses shared memory with lower overhead and no pickling restrictions, making it great for I/O-bound work, but it remains subject to the Python GIL and is not truly parallel for CPU-heavy processing. Use `ProcessPoolExecutor` for heavy compute operations and `ThreadPoolExecutor` (or the default) for network-bound or mixed I/O and light processing workflows.

---

## Performance Considerations

Spawning and managing worker processes carries an initialization overhead, meaning extremely fast callbacks may run slower in a process pool than synchronously due to serialization costs. Additionally, offloading CPU work via the executor does not change the network concurrency rules; the driver's underlying `MAX_REQUESTS` semaphore still governs HTTP request limits.
Spawning and communicating with worker processes incurs a startup and serialization cost; for very fast callbacks, this overhead can outweigh the parallel execution benefits. Additionally, the driver's underlying `MAX_REQUESTS` semaphore continues to govern HTTP concurrency independently of whether you use a thread or process pool for your callback functions.

<!-- synthesized-for: 3.1.1 -->
<!-- synthesized-for: 3.9.2 -->
4 changes: 2 additions & 2 deletions wiki/cascade-cms-wiki/docs/advanced/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Advanced

Welcome to the advanced power user guide for tuning and scaling your application. This section covers configuration and performance topics that go beyond the defaults, including caching behavior, debug logging, and CPU-intensive callback patterns.
Welcome to the advanced section for power users who want to push the library beyond its default capabilities. This guide covers complex configuration settings, performance tuning, and strategies for handling CPU-intensive workloads.

---

Expand All @@ -11,4 +11,4 @@ Welcome to the advanced power user guide for tuning and scaling your application
| [Logging & Debugging](logging.md) | Enabling debug mode, configuring the debug dict, reading log output, and walking through a sample logfile |
| [CPU-Intensive Tasks](cpu-intensive.md) | When to use `ProcessPoolExecutor`, pickling constraints, and performance trade-offs vs `ThreadPoolExecutor` |

<!-- synthesized-for: 3.1.1 -->
<!-- synthesized-for: 3.9.2 -->
35 changes: 15 additions & 20 deletions wiki/cascade-cms-wiki/docs/advanced/logging.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,36 @@
# Logging & Debugging
# Logging & Definitions

Operation logger owns all console, logfile, and (verbose mode) request/response file output for the cascade_cms library. Two output modes exist: normal mode for everyday use and debug mode for diagnosing failures, controlled solely by whether `debug_config` is `None`. Both produce logfiles; debug mode adds a verbose nested call-chain log alongside a quieter console output.
The `cascade_cms` library provides two distinct output modes: a normal mode for everyday use and a debug mode for diagnosing failures. Both modes produce logfiles, but debug mode adds a verbose nested call-chain log alongside a quieter console output.

---

## Normal Mode Output

Normal mode produces minimal console output and a simple logfile. It logs lifecycle markers such as `[INIT]`, `[RUNNING]`, `[DONE]`, and `[EXIT]`, along with batch tallies and operation progress lines. The normal logfile is named `{SERVER}_{timestamp}.log` and records one line per chain.
Normal mode uses a minimal console and a simple logfile to track execution, writing lifecycle markers (`[INIT]`, `[RUNNING]`, `[DONE]`, `[EXIT]`), per-operation progress lines, and a normal logfile named `{SERVER}_{timestamp}.log` with one line per operation.

### Normal Logfile Format

```text
[INIT]: Connecting to myserver.com
[RUNNING]: myscript.py
(uuid_or_path, asset_type) OP1 -> fn_name: Type -> ...
1/1 succeeded
[INIT]: Connecting to example.com
[LOG]: ./logs/example.com_2023-10-27T12-30-00-000000_1.log
(mySite/blog/post-1, page) READ -> edit -> publish
[DONE]: 1 assets processed in 0.5s
[EXIT]: Disconnecting from myserver.com
0 failed, 1 succeeded
[EXIT]: Disconnecting from example.com
```

---

## Enabling Debug Mode

```python
wrapper = CascadeWrapperBase("myserver.com", debug_config={
"log_dir": "./logs",
"show_network_headers": True
})
wrapper = CascadeClient(server="example.com", debug_config={})
```

---

## Debug Configuration Options

Recognized `debug_config` keys control the log directory and network header visibility.

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| log_dir | str | "./logs" | Directory for the logfile and (verbose mode) request/response JSON files. |
Expand All @@ -45,14 +40,14 @@ Recognized `debug_config` keys control the log directory and network header visi

## Debug Logfile Format

The debug logfile uses the naming convention `{SERVER}_debug_{timestamp}.log`. It features a quiet console, a verbose logfile, and per-request JSON files. There is no longer a separate "log operations vs. callbacks vs. responses" toggle — `_is_debug` is the single on/off switch for verbose behavior.
The debug logfile uses the naming convention `{SERVER}_debug_{timestamp}.log` and captures quiet console, verbose logfile output plus per-request JSON files.

### Sample Debug Log

```text
>>>> START REQUEST <<+
(uuid_or_path, asset_type) OP1 -> fn_name: Type -> ...
[GET] https://myserver.com/api/v1/read/site/path
>>>> START REQUEST <<<<
[GET] https://example.com/api/v1/read/page/mySite/blog/post-1
(mySite/blog/post-1, page) READ -> edit -> publish
1/1 succeeded
>>>> END REQUEST <<<<
```
Expand All @@ -61,6 +56,6 @@ The debug logfile uses the naming convention `{SERVER}_debug_{timestamp}.log`. I

## Interpreting Errors in Debug Mode

Error lines in debug mode utilize a `v` marker followed by an `!ERROR:` block indicating the failure at the specified step index. Chain-level failures go through `flush_chain_error` which has step-index context. `log_cascade_error` provides a thin wrapper to log a `CascadeError` (API-level failure) outside of chain context, while `log_python_error` logs an unhandled Python exception with traceback, file name, and line number.
Error blocks in debug mode render a `v` marker followed by a `!ERROR:` block aligned under the failing step's index. Network and library failures prepend `[NETWORK]` or `[CASCADE-REST-CMS]` to the error message, while cascade and callback errors receive no prefix, appending `@{file}:{line}` to the last line of the message.

<!-- synthesized-for: 3.1.1 -->
<!-- synthesized-for: 3.9.2 -->
213 changes: 53 additions & 160 deletions wiki/cascade-cms-wiki/docs/core-concepts/index.md
Original file line number Diff line number Diff line change
@@ -1,197 +1,90 @@
# Core Concepts

## The `Cascade` class

`Cascade` is the single entry point for all library interactions. Open a
session with it as a [context manager](https://docs.python.org/3/reference/datamodel.html#context-managers):

```python
from cascade_cms.wrapper import Cascade, EnvironmentVars

env: EnvironmentVars = {
"SERVER": "my-site",
"API_KEY": "your-api-key",
"CASCADE_URL": "https://your-cascade-instance.com",
}

with Cascade(env) as cascade:
# queue operations here
...
```

Previously known as `CascadeWrapperBase`; the alias still works but `Cascade`
is the current name.

### Constructor

```python
Cascade(
environmentVariables: EnvironmentVars,
debug: dict[str, Any] | None = None,
*,
exit_on_failure: bool = True,
log_dir: str | os.PathLike | None = None,
)
```

`EnvironmentVars` is a `TypedDict` with three required keys:

| Key | Description |
|---|---|
| `SERVER` | Label used in log output (not a network address) |
| `API_KEY` | Cascade REST API bearer token |
| `CASCADE_URL` | Base URL of your Cascade CMS instance |

`exit_on_failure=True` (default) ends the script non-zero on any failure.
Pass `False` for long-lived processes like the MCP server.
This page breaks down the library's core mental model: wrapping your session with `Cascade`, queueing operations through `Operations`, building sequential workflows via `OperationChain`, and executing them concurrently with `submit_requests()`.

---

## Operations

`cascade.operations` is the entry point for all CMS operations. It returns
an `Operations` builder that accepts the first call in a chain:

```python
with Cascade(env) as cascade:
chain = cascade.operations.read("abc123")
```

Available operations mirror the Cascade REST API:

| Operation | Description |
|---|---|
| `.read(id)` | Read one asset |
| `.edit(payload)` | Edit an asset via callback or direct payload |
| `.publish(id)` | Publish an asset |
| `.delete(id)` | Delete an asset |
| `.copy(id, dest)` | Copy an asset |
| `.move(id, dest)` | Move an asset |
| `.checkIn(id)` | Check in an asset |
| `.checkOut(id)` | Check out an asset |
| `.search(query)` | Search assets |
| `.create(asset)` | Create a new asset |

See the [API Reference](../operations/operations.md) for full signatures.

---
## Basic Operation Calls

## Operation chains
Every script follows the same skeleton: open the wrapper as a context manager, queue operations on `cascade.operations`, chain callbacks with `.then()`, then call `submit_requests()` once to execute all chains concurrently.

Each operation call returns an `OperationChain`, not a result. Nothing
executes until you call `.submit_requests()`. This design lets you compose
multi-step workflows before any network requests are made:
### Example

```python
from cascade_cms.cmstypes import Asset

def set_searchable(asset: Asset) -> Asset:
asset.searchable = True
return asset

with Cascade(env) as cascade:
results = (
cascade.operations
.read("abc123")
.edit(set_searchable)
.publish("abc123")
.submit_requests(Asset)
)
with Cascade(env_vars) as cascade:
# Queue a read operation for an asset
chain = cascade.operations.read(identifier)

# Submit all queued operations in a batch
results = cascade.submit_requests()

# Check the result
result = results[0]
if isinstance(result, CascadeError):
print(f"Operation failed: {result.message}")
```

The type argument to `.submit_requests()` is for the type checker. `Asset`
covers most cases; use a more specific type when the operation returns one.

---

## `edit()` and callbacks

`edit()` accepts a callable, a single asset, or a list of assets. The
callable form is the most common in practice:
### Expected Output

```python
def update_keywords(asset: Asset) -> Asset:
asset.keywords = "updated"
return asset

cascade.operations.edit(update_keywords)
Asset(
_asset_type='page',
_data={
'id': '1234567890abcdef1234567890abcdef',
'name': 'index',
'path': '/index',
'siteName': 'default',
'displayName': 'Home Page',
...
}
)
```

The callback receives the current asset and must return the modified asset.
The identifier is derived from the asset's own `id`/`path`/`siteName` fields
— there is no separate identifier argument on `edit()` in 3.9.1.

---

## Batch execution and `ChainGroup`
## Payload Models

Payload models are typed objects (such as `SearchInformation`) that pair with specific operations to ensure the API endpoint receives the expected fields. They provide type-safety, validation, and clarity by structuring inputs correctly before they reach the REST driver.

Passing a list to operations that accept identifiers — `read`, `delete`,
`publish`, etc. — creates one independent `OperationChain` per identifier,
returned as a `ChainGroup`:
### Example: `SearchInformation` paired with `search`

```python
with Cascade(env) as cascade:
results = (
cascade.operations
.read(["id1", "id2", "id3"])
.submit_requests(Asset)
)
payload = SearchInformation(
siteName="default",
searchTerms="blog",
searchFields=["name", "summary"],
searchTypes=["page", "file"]
)
cascade.operations.search(payload)
```

Failures in one chain do not cancel others. The returned `ChainResults`
covers all chains in the group.
These models enforce required attributes and validation rules via Pydantic before requests are dispatched. Passing raw dictionaries is generally not supported for complex operations, and this pattern is shared across all parameter models like `deleteParameters`, `auditParameters`, and `copyParameters`.

---

## Result handling
## CPU-Intensive Operations

`.submit_requests()` returns `ChainResults[T]` — a list subclass with
`.success` and `.failed` properties. See [Result Handling](../getting-started/result-handling.md)
for full details.
For operations involving heavy computation in `.then()` callbacks — image processing, data transformation, bulk string manipulation — offload work to a `ProcessPoolExecutor` rather than running it on the async event loop.

```python
results = cascade.operations.read("abc123").submit_requests(Asset)

for asset in results.success:
print(asset.path)
from concurrent.futures import ProcessPoolExecutor
from os import cpu_count

for failure in results.failed:
print(f"[{failure.category}] {failure.message}")
with ProcessPoolExecutor(max_workers=cpu_count()) as executor:
with Cascade(env_vars) as cascade:
cascade.operations.read(id).then(optimize_image)
results = cascade.submit_requests(executor=executor)
```

---
!!! note "Module-level functions only"
Callbacks passed to `ProcessPoolExecutor` must be defined at module level — lambdas and nested functions are not picklable and will raise at runtime.

## Script logging

`script_log.note()` writes annotated log lines from within a script:

```python
from cascade_cms.utils import script_log

with Cascade(env) as cascade:
script_log.note("Starting run")
results = cascade.operations.read("abc123").submit_requests(Asset)
script_log.note(f"Done — {len(results.success)} succeeded")
```

Calls outside the `with` block drop silently with a `RuntimeWarning`.
API keys are automatically redacted from log output.
See [Advanced: CPU-Intensive Tasks](../advanced/cpu-intensive.md) for full configuration details, performance trade-offs, and `ThreadPoolExecutor` comparison.

---

## Identifiers

The `utils.identifiers` module (added in 3.8.3) coerces raw Cascade
identifier dicts to `IdentifierType`:

```python
from cascade_cms.utils.identifiers import to_identifier, to_identifiers

identifier = to_identifier({"id": "abc123", "type": "page", "path": None})
identifiers = to_identifiers([{"id": "a"}, {"id": "b"}])
```
## Next Steps

Use these when processing raw API responses before passing identifiers to
operations.
Ready to go deeper? The [Advanced](../advanced/index.md) section covers configuration topics for power users: debug logging and CPU-intensive workload patterns.

<!-- synthesized-for: 3.9.1 -->
<!-- synthesized-for: 3.9.2 -->
Loading