Support JCB-generated component API contracts while preserving Joomla MCP behavior

.github/workflows/ci.yml

@@ -50,7 +50,7 @@ jobs:
         run: |
           set -euo pipefail
           mkdir -p build/evidence
-          for suite in run catalogue schema-shapes schema-discovery session-storage wire-protocol stdio-endurance execution actions write-verification message-snapshot article-deletion custom-fields field-defaults template-styles menu-components planned jobs process job-process jcb jcb-plan-preview protocol http ownership native/run release; do
+          for suite in run catalogue catalogue-refresh schema-shapes schema-discovery session-storage wire-protocol stdio-endurance execution actions write-verification message-snapshot article-deletion custom-fields field-defaults template-styles menu-components planned jobs process job-process inventory-transport jcb jcb-plan-preview generated-api-catalogue generated-api-transport generated-api-inventory jcb-form-contracts jcb-api-verification protocol http ownership native/run release; do
             php "tests/$suite.php" | tee "build/evidence/${suite//\//-}.log"
           done
       - name: Preserve the tested PHP runtime for reproducible investigation

CHANGELOG.md

@@ -2,6 +2,19 @@
 
 ## [[[NEXT_VERSION]]]
 
+### Fix
+
+- Synchronize full-size installed generated API catalogues with bounded shared form-contract transport and incremental inventory fingerprints. Release the previous catalogue snapshot before refreshing, avoid an unused catalogue preload during synchronization, and reject invalid inventories or failed refreshes before reusing definitions. Preserve every native route and validation policy, and exercise the complete supplied API-enabled JCB package through repeated installed synchronization and HTTP discovery.
+
+- Preserve effective administrator input restrictions during catalogue upgrades when only the referenced schema was customized. Keep its tool, action, prompt or binding attached to that policy, including hash-detected edits and disabled schemas; verify allowed and denied requests before and after upgrade.
+
+- Identify Joomla core components through the native core-extension catalogue during API synchronization. Preserve their existing MCP bindings while supporting enabled third-party components independently of uninstall and disable protection flags; exercise registered-route synchronization and nested-filter reads in the installed Joomla and JCB checks.
+
+- Describe generated API inputs from their installed native forms, including nested subforms, GUID relationships and validation metadata. Generate a required record GUID only when the native contract calls for one, freeze it in the approved plan, and verify writes through an independently bound item read. Preserve omitted PATCH fields and report unverifiable native responses truthfully.
+
+- Support observed GUID and alternate unique-key routes across installed generated component APIs, with scoped provider permissions, bounded multiselect filters and exact independent item-read bindings. Preserve existing Joomla route encoders and unselected provider definitions during synchronization.
+
+- Accept bounded nested JSON inputs in the generic API and companion read tools while retaining selected-action validation, existing Joomla MCP behavior and administrator-owned schema policies.
 ### Addition
 
 - Add package-first Joomla setup, administrator-area, token/ACL, tool, confirmed-write, JCB and recovery guidance, with linked AI and direct-client connection instructions.
@@ -132,4 +145,3 @@
 - External client and remote stdio ownership separated into mcp_client.
 
 Development baseline entries describe existing source, not previously published releases. Published immutable tags establish release availability.
-

README.md

@@ -38,13 +38,15 @@ Published database records define providers, schemas, actions, bindings, tools,
 
 HTTP uses Joomla's authenticated API user and intersects component permissions, viewing-access levels, row assets and target-resource ACL. Remote stdio through the external client retains that identity. Direct local Joomla console execution is a separate trusted server track; remote requests cannot select it. Confirmed remote writes require explicit grants, reviewed plans, confirmation, idempotency and read-back. See [security](SECURITY.md).
 
-Joomla core remains usable without JCB. For JCB capabilities, install and enable JCB and its native registration plugins, then refresh definitions using **Operations → Refresh JCB definitions** or:
+Joomla core remains usable without JCB. JCB is a required project integration alongside Joomla core. For JCB capabilities, install and enable JCB and its native registration plugins. Synchronize installed component APIs and JCB commands using **Operations → Refresh JCB definitions** or:
 
 ```bash
 php cli/joomla.php joomla:mcp:jcb-sync
 ```
 
-Synchronization inspects actual installed JCB API routes and registered command definitions, persists schemas/actions/bindings/targets, preserves administrator customization and refuses unsupported contracts. It creates no invented endpoints when an API distribution is absent. Ordinary discovery reads the stored catalogue; repeat refresh after changing JCB or its registration plugins. Disabling or uninstalling a required registration plugin hides the affected operations from discovery and execution.
+Synchronization inspects registered routes for enabled installed third-party components under native administration permission, plus JCB command input definitions when JCB and its command plugin are available. Joomla's native core-extension inventory identifies core components, which retain their existing catalogue; the MCP component is also excluded. Extension protection/locking flags do not determine API permissions. Synchronization persists component-owned schemas, actions, bindings and targets, preserves administrator customization and reports unsupported contracts. An absent API distribution produces no invented endpoints. Ordinary MCP discovery reads the stored catalogue and does not modify it. Repeat synchronization after changing a component or its route/command plugins. Disabling or uninstalling a required registration plugin hides its affected operations from discovery and execution.
+
+Generated API bindings describe literal native forms, including defaults, conditional rules, relationships and subforms; they support reviewed numeric, GUID and unique-key item routes. Defaults are descriptive and omitted PATCH fields stay omitted. Plans disclose and freeze an automatically generated primary GUID only when the installed create form explicitly requires it without a native default or detected server generation. Independent API read-back verifies resource identity and declared observable values. Native validation, ACL and errors remain authoritative.
 
 JCB package `get`, `init`, `pull`, `push`, `reset` and compilation are effectful operations. Plans freeze inputs, options and relevant definition/configuration fingerprints. Native background execution uses isolated PHP workers and durable principal-owned jobs; compiler archives become bounded, hash-verified artifact references. Cancellation and uncertain outcomes retain recovery evidence; cancellation does not roll back side effects. [JCB setup](docs/GETTING-STARTED.md#enable-jcb-operations-and-background-jobs) covers worker prerequisites; the [JCB contract and acceptance matrix](docs/integrations/JCB.md) records tested scenarios and native limitations.
 
@@ -56,11 +58,10 @@ For development or independent component maintenance, a GitHub source ZIP of a r
 
 [OctoJPack](https://github.com/octoleo/octojpack) builds the combined package using the fixed `.octojpack` configuration and each extension's latest tag, then publishes it to `mcp_package`. The component version determines the package version. The manual **Release** workflow freezes component changelogs, creates its immutable tag, updates and hashes its own feed with OctoShoom, then publishes the package. The package tag starts its separate feed/hash workflow. [Release instructions](docs/RELEASE.md) cover workflow order and secrets.
 
-Native administration, installed core tests, JCB synchronization and job/artifact runtime are implemented. [Implementation status](docs/IMPLEMENTATION.md) separates historical installed Joomla/JCB evidence from verification of current source and releases. Package availability does not enlarge those runtime-specific verification boundaries.
+Native administration, installed core tests, JCB synchronization and job/artifact runtime are implemented. [Implementation status](docs/IMPLEMENTATION.md) separates historical installed Joomla/JCB evidence from verification of current source and releases. Package availability does not enlarge those runtime-specific verification boundaries. The generic adapter supports installed component APIs, native form contracts and GUID/unique-key routes. The schema-only upgrade regression has 130 behavioral checks, invoked by the existing catalogue CI suite; readiness requires the current PR head to pass all checks. Fresh live generated JCB GUID CRUD acceptance and four historical native POST failures remain explicit evidence boundaries.
 
 The original migration snapshot is pinned at `joomengine/joomla-mcp@2cff50f4f6b440da3c684f9995a77efad32e1a36`. Imported native handlers retain behavioral contracts and source attribution. Joomla 6 native contracts are authoritative; repository/MVC/XML placement follows JCB's extension-root layout. This is hand-authored JCB-aligned source, not a claim of an imported JCB blueprint. Explicitly unavailable inherited operations are recorded in [migration provenance](docs/migration/README.md).
 
 Before changing runtime code, read [architecture](docs/ARCHITECTURE.md), [database design](docs/DATABASE.md), [migration plan](docs/MIGRATION.md), [security](SECURITY.md) and [agent instructions](AGENTS.md). User-facing resource details include [custom fields](docs/CUSTOM-FIELDS.md) and [native Joomla API limitations](docs/testing/native-api-limitations.md).
 
 Changes are recorded in [CHANGELOG.md](CHANGELOG.md) and [changelog.xml](changelog.xml). Pending entries use `[[[NEXT_VERSION]]]`; the release workflow assigns their version.
-

admin/cli/jcb.php

@@ -11,6 +11,7 @@
 use Joomla\Application\Event\ApplicationEvent;
 use Joomla\CMS\Application\ApiApplication;
 use Joomla\CMS\Event\Application\BeforeApiRouteEvent;
+use Joomla\CMS\Extension\ExtensionHelper;
 use Joomla\CMS\Factory;
 use Joomla\CMS\Language\LanguageFactoryInterface;
 use Joomla\CMS\Plugin\PluginHelper;
@@ -27,12 +28,16 @@
 use VDM\Component\JoomEngineMcp\Administrator\Console\WorkerApplication;
 use VDM\Component\JoomEngineMcp\Administrator\Domain\OperationException;
 use VDM\Component\JoomEngineMcp\Administrator\Jcb\ApiRegistry;
+use VDM\Component\JoomEngineMcp\Administrator\Jcb\CatalogueBuilder;
 use VDM\Component\JoomEngineMcp\Administrator\Jcb\CommandOutput;
+use VDM\Component\JoomEngineMcp\Administrator\Jcb\GeneratedApiInventory;
+use VDM\Component\JoomEngineMcp\Administrator\Jcb\InventoryTransport;
 use VDM\Component\JoomEngineMcp\Administrator\Jcb\RegistrationObserver;
 use VDM\Component\JoomEngineMcp\Administrator\Jcb\Worker;
 use VDM\Component\JoomEngineMcp\Administrator\Security\ConsoleIdentity;
 use VDM\Component\JoomEngineMcp\Administrator\Security\JoomlaPrincipal;
 use VDM\Component\JoomEngineMcp\Administrator\Security\LocalPrincipal;
+use VDM\Component\JoomEngineMcp\Administrator\Service\Json;
 
 if (PHP_SAPI !== 'cli')
 {
@@ -45,6 +50,7 @@
 // MCP artifact storage applies its own explicit private directory/file modes.
 $level = ob_get_level();
 ob_start(static fn (string $output): string => '', 4096);
+$json = null;
 
 try
 {
@@ -99,7 +105,8 @@
 		$principal = new JoomlaPrincipal($user);
 		$app->loadIdentity($user);
 
-		if (!$principal->authorise('mcp.access', 'com_joomengine_mcp') || !$principal->authorise('core.admin', 'com_componentbuilder'))
+		$asset = $request['operation'] === 'jcb.inventory' ? 'com_joomengine_mcp' : 'com_componentbuilder';
+		if (!$principal->authorise('mcp.access', 'com_joomengine_mcp') || !$principal->authorise('core.admin', $asset))
 		{
 			throw new OperationException('JCB_ACCESS_DENIED', 'The original Joomla user no longer authorizes JCB execution.');
 		}
@@ -142,17 +149,37 @@
 	if ($request['operation'] === 'jcb.inventory')
 	{
 		$commands = $worker->inventory();
+		if (!$principal->isLocal() && CatalogueBuilder::hasCommandScope($commands)
+			&& !$principal->authorise('core.admin', 'com_componentbuilder'))
+		{
+			throw new OperationException('JCB_CATALOGUE_DENIED', 'Native JCB administration permission is required to synchronize registered JCB commands.');
+		}
 		$api = $container->get(ApiApplication::class);
 		$api->loadIdentity($app->getIdentity());
 		Factory::$application = $api;
 
 		try
 		{
+			$database = $container->get(DatabaseInterface::class);
+			$query = $database->createQuery()->select($database->quoteName(['type', 'element', 'enabled']))
+				->from($database->quoteName('#__extensions'))
+				->where($database->quoteName('type') . ' = ' . $database->quote('component'));
+			$coreComponents = array_map(static fn (array $extension): string => $extension[1], array_filter(
+				ExtensionHelper::getCoreExtensions(), static fn (array $extension): bool => $extension[0] === 'component'));
+			$components = GeneratedApiInventory::components($database->setQuery($query)->loadAssocList(), $coreComponents);
+			$components = array_values(array_filter($components, static fn (string $component): bool =>
+				$principal->isLocal() || $principal->authorise('core.admin', $component)));
+			if (in_array('com_componentbuilder', $components, true))
+			{
+				$commands['component'] = 'com_componentbuilder';
+			}
 			$router = new ApiRouter($api);
 			PluginHelper::importPlugin('webservices', null, true, $dispatcher);
 			$owners = $observer->dispatch($dispatcher, new BeforeApiRouteEvent('onBeforeApiRoute', ['router' => $router, 'subject' => $api]),
 				static fn (): array => $router->getRoutes());
-			$result = ['commands' => $commands, 'api' => (new ApiRegistry($router, $owners))->inventory()];
+			$inventory = (new ApiRegistry($router, $owners, $components))->inventory();
+			$result = ['commands' => $commands, 'api' => GeneratedApiInventory::enrich($inventory,
+				JPATH_ADMINISTRATOR . '/components', JPATH_ROOT . '/api/components')];
 		}
 		finally
 		{
@@ -169,9 +196,19 @@
 	}
 
 	$result = ['protocol' => 'joomengine-worker/1'] + $result;
+	if ($request['operation'] === 'jcb.inventory' && isset($request['inventory_format']))
+	{
+		if ($request['inventory_format'] !== InventoryTransport::FORMAT)
+		{
+			throw new OperationException('JCB_INVENTORY_INVALID', 'The requested native inventory encoding is unsupported.');
+		}
+		$result = InventoryTransport::pack($result);
+		$json = Json::encode($result, InventoryTransport::MAX_WIRE_BYTES);
+	}
 }
 catch (Throwable $error)
 {
+	$json = null;
 	$result = ['protocol' => 'joomengine-worker/1', 'error' => $error instanceof OperationException
 		? $error->toArray() : ['code' => 'JCB_WORKER_FAILED', 'message' => 'The native JCB worker could not complete this request.']];
 }
@@ -181,7 +218,7 @@
 	ob_end_clean();
 }
 
-$json = json_encode($result, JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR);
+$json ??= json_encode($result, JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR);
 for ($offset = 0, $length = strlen($json); $offset < $length; $offset += $written)
 {
 	$written = fwrite(STDOUT, substr($json, $offset));