diff --git a/.agents/skills/AGENTS.md b/.agents/skills/AGENTS.md index 6a5b9c612..22e468de6 100644 --- a/.agents/skills/AGENTS.md +++ b/.agents/skills/AGENTS.md @@ -4,3 +4,4 @@ - State each skill's platform scope explicitly; keep Android and iOS procedures separate. - Describe reusable workflows in terms of the target extension. Named extensions may be optional examples, never mandatory dependencies for unrelated ports. - Obtain build commands, versions, artifact names, and paths from the target extension's current scripts and configuration. Do not treat one extension's settings as repository-wide defaults. +- Every extension/platform skill must preserve the standing repository-layout rule in fusion-extension-porting/SKILL.md: adapt existing libraries in place with narrow macros or in the corresponding platform component directory. Apply this to future ports automatically; do not invent mobile subtrees in Windows plugin/library projects. diff --git a/.agents/skills/fusion-android-porting/SKILL.md b/.agents/skills/fusion-android-porting/SKILL.md index d084913fe..fdf5a741e 100644 --- a/.agents/skills/fusion-android-porting/SKILL.md +++ b/.agents/skills/fusion-android-porting/SKILL.md @@ -8,7 +8,7 @@ description: Port Clickteam Fusion C++ extensions to Android, or diagnose Androi ## Scope Apply the [common porting rules](../fusion-extension-porting/SKILL.md), including -Git checkout normalization and artifact hash verification, before this workflow. +Git checkout normalization and artifact verification, before this workflow. Use this skill only for Fusion Android/JNI implementation, builds, publishing, or troubleshooting. For a multi-platform task, apply it to the Android portion; still assess shared C++ changes for their @@ -19,8 +19,23 @@ Keep algorithms in shared C++; use Java for Fusion Runtime integration and JNI m Compile native libraries during extension development/publishing, then let Fusion use its own default build templates when exporting an APK. +If encryption or encrypted assets are involved, also read and follow +[`fusion-shared-encryption`](../fusion-shared-encryption/SKILL.md). Reusing its source +is not sufficient: mobile plugins must consume one shared encryption runtime. + ## Establish the target +Apply the common skill's standing repository-layout requirement to every plugin +and dependency adaptation without asking the author to restate it. + +Keep Android implementation and development assets under +`Extensions/Android_Gradle`, following its existing Java/JNI, `cmake/` +and `tests/` layout. `Extensions/` is the Windows VS project; +do not add an Android or Mobile subtree there or in Windows library directories. +Adapt an existing library in its original files with narrow macros, or put its +Android-specific code under the corresponding app/src/main component directory. +Preserve existing `ToInstall` distribution paths. + 1. Identify the extension, its Windows ACE definitions, existing mobile code, and the installed Fusion Runtime and development SDK versions. 2. Inspect repository instructions, branch state, and local changes. Preserve unrelated work. diff --git a/.agents/skills/fusion-extension-porting/SKILL.md b/.agents/skills/fusion-extension-porting/SKILL.md index 6bc417d8f..6b689446b 100644 --- a/.agents/skills/fusion-extension-porting/SKILL.md +++ b/.agents/skills/fusion-extension-porting/SKILL.md @@ -13,6 +13,40 @@ or [iOS porting](../fusion-ios-porting/SKILL.md) for the relevant target. Derive paths and commands from the target project; do not require another plugin. Write skills and repository documentation in English. +## Repository layout + +This is a standing author requirement for every plugin and library port, not a +Fini-specific convention. Apply it automatically at the start of future work; +do not wait for the author to repeat it or ask them to choose the layout again. + +The original `Extensions` tree contains Windows Visual Studio projects and their +libraries. Android_Gradle and iOS are independent platform project roots, even +though their current repository paths are nested below Extensions. Do not treat +that historical nesting as a reason to put mobile adaptations into Windows +library directories. + +`Extensions/` is the Windows Visual Studio project directory. Keep iOS +projects, adapters, build scripts, tests and native handoff inputs under +`Extensions/iOS/`. Keep Android adapters, JNI sources, CMake/toolchain +configuration, tests and documentation under `Extensions/Android_Gradle`, using +its existing source/cmake/test directories. Do not nest `iOS`, `Android` or +`Mobile` development trees inside a Windows plugin or library directory. For +library adaptations, either extend the original source with narrow platform or +feature macros, or put platform-specific code under the corresponding platform +project's component/source directory (Android uses app/src/main). Reuse the +original implementation; do not create a second cipher or algorithm copy. +Existing `ToInstall` release layouts and VS publishing hooks remain +the distribution entry points, not platform development roots. + +Before creating or moving files, determine whether each file extends an existing +shared implementation or belongs to a platform project. Use the original file +with a narrow macro for the former, and the corresponding platform component +directory for the latter. A shared runtime does not justify putting Android/iOS +build, packaging or handoff trees in a Windows library directory. Follow existing +platform paths; their historical nesting does not authorize relocating the whole +platform project. After a move, update includes, build/project references, +publishers, manifests and documentation, and verify the resulting source inventory. + Use the current Windows C++ implementation as the behavioral reference for portable features. Reuse shared algorithms; adapt platform-specific object, resource and Runtime access through the actual target SDK. Keep shared C++ diff --git a/.agents/skills/fusion-ios-porting/SKILL.md b/.agents/skills/fusion-ios-porting/SKILL.md index 111d97e1c..e6566de39 100644 --- a/.agents/skills/fusion-ios-porting/SKILL.md +++ b/.agents/skills/fusion-ios-porting/SKILL.md @@ -17,15 +17,25 @@ for other plugins. Write this skill, references and operational docs in English. Preserve shared C++ interfaces; submit shared behavioral changes for author review. Keep private SDK/exporter material outside the public repository. +Apply the common skill's standing repository-layout requirement to every plugin +and dependency adaptation without asking the author to restate it. Adapt existing +libraries in their original files with narrow macros, or put iOS-specific +integration under the iOS project's corresponding component directory. Keep +Apple build, packaging and handoff inputs there, outside Windows library trees. + +If encryption is involved, also apply +[shared encryption](../fusion-shared-encryption/SKILL.md): that runtime is an +author-approved shared service, not a rule to dynamically split every helper. + ## Workflow 1. Inspect current Windows ACE definitions, mobile adapters, the installed exporter and a real exported project. Inventory IDs, property layout, parameter decoding, events, selection, creation and unsupported operations before implementation. 2. Treat the Windows C++ implementation as the behavioral oracle for portable - features. Add direct comparisons against its actual classes for paths, flags, - units/ZOC, area layers, line ranges, AOE and caching; do not use Android output - as the specification. Adapt only platform-specific I/O/object mechanics using + features. Add direct comparisons against its actual classes for the target + plugin's data, flags, numeric/string semantics and caching; do not use Android + output as the specification. Adapt only platform-specific I/O/object mechanics using the actual iOS SDK. Reuse computational algorithms in shared C++. Keep Runtime objects, selection, creation and event dispatch in an owned adapter. Compare helpers with current Windows and Android versions; preserve each Runtime's actual representations. @@ -39,12 +49,16 @@ Keep private SDK/exporter material outside the public repository. fixed-width values, explicit buffer/string lengths and exception translation. Avoid STL and C++ layout exposure. Retain owners across callback reentrancy; snapshot iteration results and define destruction/replacement behavior. -4. Keep each plugin's build project under Extensions/iOS/. Use CMake and +4. Keep the plugin's iOS project, Runtime adapter, build scripts, tests and native + handoff inputs under Extensions/iOS/. Extensions/ is its VS + project; do not put iOS or Mobile development trees there. Use CMake and Ninja/Make for scripted compilation; use Apple CLI tools for SDK selection and XCFramework creation. Full Xcode supplies iOS SDKs; standalone Command Line Tools are insufficient. Do not require opening Xcode GUI to build the core. -5. Build separate iOS device and simulator platform slices. The current task - supports arm64 only; do not add Intel/Rosetta/32-bit targets. Use a static +5. Build separate iOS device and simulator platform slices. Derive architectures + from the target project and author requirements. The FTW reference uses arm64 + only; do not make that a universal rule or add Intel/Rosetta/32-bit slices + without a target requirement. Use a static XCFramework when it matches the exported project's linking model. Prebuilding isolates language settings, not the deployment availability of libc++ APIs. 6. Stage binaries with source/configuration and per-file hashes, normalize Git @@ -56,7 +70,9 @@ Keep private SDK/exporter material outside the public repository. 7. Publish the extension as .ext, including required native and adapter material. Native-only manifests are valid inputs to a Windows packager: verify compiled dependencies, then package the current Runtime adapter there. Do not force a - Mac rebuild merely because the thin source adapter changed. + Mac rebuild for adapter sources shipped separately from the compiled core. + If an adapter is compiled into the transferred library, its changes do require + a rebuild; record that boundary instead of applying a blanket exemption. Validate its actual encoding and export round trip; do not rename an arbitrary ZIP to .ext. Fusion may export files without adding their Xcode references. Verify library/source references, include paths, system dependencies and final @@ -95,8 +111,10 @@ Keep private SDK/exporter material outside the public repository. ## Reusable Runtime helpers and exported-project patches -Use `Helpers/ObjectBase.h`, `ObjectSelection.h` and `ObjectCreation.h` as separate -generic classes, with common OI access in ObjectBase. Compare the current Windows +When lookup/selection/creation infrastructure is needed, use the maintained +`Helpers/ObjectBase.h`, `ObjectSelection.h` and `ObjectCreation.h` as separate +generic classes, with common OI access in ObjectBase. Do not add a plugin core +dependency merely to obtain these helpers or import unused plugin semantics. Compare the current Windows `Extensions/_3rdLib/Object*.h` and Android `_3rdLib` counterparts, including APIs not yet used by the consuming extension. Do not rename them for that extension. diff --git a/.agents/skills/fusion-shared-encryption/SKILL.md b/.agents/skills/fusion-shared-encryption/SKILL.md new file mode 100644 index 000000000..609f17185 --- /dev/null +++ b/.agents/skills/fusion-shared-encryption/SKILL.md @@ -0,0 +1,71 @@ +--- +name: fusion-shared-encryption +description: Reuse and publish the common encryption runtime when porting or updating Fusion plugins. Prevent private per-plugin crypto implementations and preserve the Mac build/push to Windows pull/VS packaging workflow. Covers shared contracts; platform procedures are separate references. +--- + +# Shared encryption for Fusion plugins + +## Scope + +Use when a plugin consumes encryption, key derivation or encrypted assets, or when +changing the shared mobile encryption library/build/distribution. This is a shared +runtime contract, not permission to alter unrelated plugins or merge a PR. +Apply [common porting rules](../fusion-extension-porting/SKILL.md), including +compiled-component assessment and text/binary digest policy. Read the relevant +[Android](../fusion-android-porting/SKILL.md) or +[iOS](../fusion-ios-porting/SKILL.md) workflow before platform changes. + +The shared encryption runtime is an explicit author requirement for this project. +It does not imply that every utility header, template or plugin algorithm should +be moved into a DLL. Keep small/stateless helpers modular; assess actual emitted +code, state ownership and exporter deployment before introducing another boundary. + +## Required architecture + +- Reuse `Extensions/_DeLib/Encryption`; preserve existing cipher, key derivation, + Unicode and asset formats. Do not write a Java, Objective-C or plugin-local cipher. +- **Shared means shared by plugins at runtime**, not merely common source copied + or statically linked into every plugin. Clients include `Encryption/CAPI.h` + and call the stable `Encryption_*` C ABI. Keep algorithms, KeyGen and OpenSSL + inside the single common runtime. Its implementation stays in the original + `Encryption` sources; Android build integration lives under + `Android_Gradle/app/src/main/cpp/Encryption` and iOS build integration under + `iOS/Encryption`. Do not add a Mobile subtree to the Windows library. +- Do not put libcrypto, a private encryption implementation, or a renamed copy of + the common library in each plugin package. Interface headers are permitted. + Publish/build/cache the common runtime independently and declare client dependencies. +- Keep OpenSSL/C++ implementation symbols private; export only the supported C ABI. + Validate ABI/version, target architectures, library identity and compatible STL. + Never silently overwrite a conflicting common runtime already installed by another plugin. +- Inspect current shared modules and the target plugin's real publisher/VS hooks + before adding another tool path. Reference existing publishing scripts (such as + FTW) for discovery, incremental builds, errors, copy/install and archive behavior; + do not invent a second manual delivery path or treat FTW-specific constants as universal. + +## Cross-machine handoff + +When Apple compilation belongs to the author, generate the real Xcode project, +shared schemes and commands even if the agent has no Xcode. The Mac build stages +owned plugin and common-library artifacts separately with SHA256/source manifests. +The author compiles and pushes those files; Windows git pull receives them, and +VS RunOnly invokes the publisher to validate, generate and update distribution +packages. Windows must not require Xcode or rebuild Apple code. Do not push +unbuilt placeholders or claim Apple/device validation from project-generation tests. + +Preserve signing, runtime loader and exporter constraints. A generated ZIP is not proof of exporter integration. For the supported legacy +format, reuse the independent `.ext` codec and validate the decoded payload and +six project-insertion blocks. Distinguish a verified package from a successful +export, final link or device test. +Do not redistribute licensed exporter headers, Runtime sources or test projects. + +## Validation and review + +Check the client binary's actual dynamic references and absence of defined crypto +symbols, shared runtime exports, all supported slices, separate package ownership, +hashes/stale-source rejection and one-time dependency installation. Include failure +checks for missing, corrupt or conflicting artifacts. Run the actual publishing +entry point. Distinguish host fixtures, target builds, APK/exporter builds and devices. +Keep shared source changes reviewable and open/update the requested PR; do not merge. + +- Android procedure: [references/android.md](references/android.md) +- iOS/Mac-to-Windows procedure: [references/ios.md](references/ios.md) diff --git a/.agents/skills/fusion-shared-encryption/references/android.md b/.agents/skills/fusion-shared-encryption/references/android.md new file mode 100644 index 000000000..90b9dbfee --- /dev/null +++ b/.agents/skills/fusion-shared-encryption/references/android.md @@ -0,0 +1,26 @@ +# Android shared encryption + +Use alongside fusion-android-porting and android-runtime-parity. Keep Fusion object +access/events in Java and algorithms in C++; use JNI to call the plugin native core. +The plugin library imports libofe_encryption.so through the C ABI. Only that common +library links libcrypto. Reuse +`Android_Gradle/app/src/main/cpp/Encryption/CMakeLists.txt` and +`Android_Gradle/cmake/Encryption/android_runtime.py`; both consume the original +Encryption sources. Keep Android adaptations in its platform project. + +Use one common build/cache/output across plugins. Emit a separate OFEEncryption.zip +with matching libc++_shared.so, licenses, ABI/version and hashes. Client ZIPs own only +their JNI library/Java/resources and an explicit dependency; no common implementation +or duplicate crypto binaries. Do not replace application Gradle/ProGuard/Runtime files. +Stage the common dependency once in the final exported app, with conflict/hash checks. +Do not assume Fusion includes an arbitrary common ZIP just because it is installed. + +Follow the target publisher's SDK/NDK/CMake/Ninja discovery and sdkmanager preparation +flow; search installed SDK/VS/PATH tools first. Keep work/cache/logs outside commits. +Check actual readiness after installation. Preserve proxy and certificate settings. + +Inspect DT_NEEDED/SONAME and dynamic symbols: the client imports Encryption_* from +the common library and defines no encryption/EVP implementation. The shared library +exports its C ABI and hides EVP symbols. Inspect each ABI's final package and APK, +including one compatible STL. Apply 16 KiB checks to supported 64-bit Android targets; +do not misreport the NDK's 32-bit 4 KiB STL as a broken 64-bit build. diff --git a/.agents/skills/fusion-shared-encryption/references/ios.md b/.agents/skills/fusion-shared-encryption/references/ios.md new file mode 100644 index 000000000..d2b753b8f --- /dev/null +++ b/.agents/skills/fusion-shared-encryption/references/ios.md @@ -0,0 +1,34 @@ +# iOS shared encryption and Mac-to-Windows delivery + +Use Objective-C++ to call the existing C++ plugin core and Encryption C ABI; JNI is +Android-only. Fini's standalone project is an example, not a dependency for other ports. +Build a client archive/XCFramework without libcrypto. Build one dynamic +OFEEncryption.framework/XCFramework through `iOS/Encryption/build.py` for all plugins, with a +stable install name and C export list. Embed/sign the common framework once in the app. +Keep its native handoff inputs in `iOS/Encryption/Prebuilt` and build tooling in +the iOS platform project; compile the original Encryption sources directly. + +Generate an actual .xcodeproj and shared schemes on the agent host. Configure local +licensed Runtime header paths through an ignored local xcconfig. Keep public SDK +locations portable and supply device/simulator build targets. A missing Xcode prevents +Apple compilation, not generation of the requested project and delivery scripts. + +Honor this repository's author handoff: Mac builds -> separately stages owned client +and shared artifacts with manifests -> author commits/pushes -> Windows git pull -> +VS RunOnly's existing publisher validates/stages those artifacts and creates/updates +release packages. Preserve the single publishing entry point and propagate errors. +Do not ask the author to manually reconstruct packages after every Mac build. + +Do not run git push from the Mac build script, commit machine-local configs or include +licensed SDK source. The author controls publishing compiled Apple files. Windows can +skip Apple packaging while no Mac build is available, but must reject incomplete, +corrupt or source-stale committed artifacts without replacing a known-good package. + +Preserve the verified PR #100 .ext flow: Windows consumes native artifacts and +packages the current loader/adapter using the independent shared format codec. +Verify decoded payload/CRC and PBX insertion blocks. A native XCFramework ZIP +alone is not an installable .ext, and a valid .ext is not proof of final linkage. +Use an idempotent target-aware configurator for missing exported project references; +preserve device signing, unrelated extensions/flags and a higher deployment floor. +Validate generated project syntax/references and the Mac/Windows manifest handoff +without calling those checks an Apple build. The author supplies Apple/device results. diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 000000000..8521a00aa --- /dev/null +++ b/.gitattributes @@ -0,0 +1,4 @@ +# Native handoff archives stay byte-exact across Mac and Windows checkouts. +*.xcframework.zip binary +*.a binary +*.ext binary diff --git a/.gitignore b/.gitignore index 6ddb8ba9c..636a34158 100644 --- a/.gitignore +++ b/.gitignore @@ -25,3 +25,6 @@ __pycache__/ # Generated OpenSSL SDK, vcpkg checkout and dependency caches /Extensions/_3rdLib/Encrypt-sdk/ /Extensions/_3rdLib/Encrypt-build/out/ + +# Local builds, toolchains, downloads and verification logs +/work/ diff --git a/Extensions/AGENTS.md b/Extensions/AGENTS.md index 0f474882f..ae57de1da 100644 --- a/Extensions/AGENTS.md +++ b/Extensions/AGENTS.md @@ -9,10 +9,30 @@ such update, synchronize portable behavior to Android, and preserve necessary platform-specific runtime implementations. The skill includes the comparison paths, reviewed revision, compatibility requirements and validation workflow. -# iOS Runtime object helpers +# Common encryption runtime -For iOS Runtime and object helper changes, follow +For plugin encryption, encrypted-asset integration, or shared encryption build/package +changes, read and follow +[`fusion-shared-encryption`](../.agents/skills/fusion-shared-encryption/SKILL.md). +The encryption runtime is shared by plugins, not privately embedded in each plugin. +For Apple artifacts preserve Mac build/push -> Windows pull -> VS RunOnly packaging. + +# Common and iOS porting rules + +The layout below is a standing author requirement for all future plugin and +library adaptations. Apply it automatically; do not ask the author to repeat it. + +Platform development belongs under `Extensions/iOS/` or the existing +`Extensions/Android_Gradle` hierarchy. `Extensions/` is a Windows VS +project, not a parent for platform development trees. These are independent +platform roots despite the historical nesting under Extensions. Adapt libraries +in their original files behind narrow macros, or in the corresponding platform +component directory (Android native sources use app/src/main/cpp). Do not add +Mobile build/package trees to Windows library directories. Preserve ToInstall. + +For any extension port, read +[`fusion-extension-porting`](../.agents/skills/fusion-extension-porting/SKILL.md). +For Apple Runtime, project or publishing changes, also follow [`fusion-ios-porting`](../.agents/skills/fusion-ios-porting/SKILL.md). -Keep generic ObjectBase, ObjectSelection and ObjectCreation classes separate -from plugin adapters. Compare their portable APIs with the current Windows and -Android helpers, and validate generic helpers without linking a plugin core. +Use independent generic object helpers where applicable, compare their portable +API with Windows/Android, and test them without linking a plugin core. diff --git a/Extensions/Android_Gradle/Fini-JNI.md b/Extensions/Android_Gradle/Fini-JNI.md new file mode 100644 index 000000000..40d666651 --- /dev/null +++ b/Extensions/Android_Gradle/Fini-JNI.md @@ -0,0 +1,217 @@ +# Fini Android JNI port + +## Architecture + +Android development files live in this Gradle tree: `app/src/main` for the +Java/JNI adapters, `cmake/Fini` for native/toolchain configuration and +`tests/fini` for validation and the ACE contract. The portable C++ Runtime is +enabled by `FINI_PORTABLE_RUNTIME` in the original `../Fini/IniCore.h`; +iOS development and Mac handoff inputs live in +`../iOS/Fini`. `../Fini` remains the Windows VS project and release staging area. + +Android uses `CRunFini.java` -> `FiniNative.java` -> JNI `libfini.so` -> the shared +C++ implementation. Java handles only Fusion parameters, runtime objects, events, +file/resource resolution and lifecycle. INI storage remains SimpleIni. Windows +and mobile share `Fini/IniCore.h`; native codecs reuse `_3rdLib/Base64.h`, +`Compress/zlibInterface.h`, `_DeLib/StrNum.h`, and the existing +`Encryption/Core.h` + `Encryption/KeyGen.h`. There is no Java encryption or INI engine. +JNI native objects have validated registry handles and explicit, idempotent cleanup. +Requests belong to Fusion's runtime thread. Java UTF-16 is converted explicitly, +including supplementary characters and embedded NUL in string codec operations. + +Encryption uses the original `_DeLib/Encryption` sources. Android integration +lives in `app/src/main/cpp/Encryption` and `cmake/Encryption`; iOS integration +lives in `../iOS/Encryption`. Fini includes only CAPI.h and calls the existing +Encryption_* C ABI. Android links libfini.so against libofe_encryption.so. iOS +links Fini's adapter archive against a single dynamic OFEEncryption.framework. +Only the common runtime links libcrypto. See the shared module's README for the +one-time exported-app installation and framework embed/sign steps. + +For iOS, CRunFini.mm uses Objective-C++ and the same C++ core; JNI is Android only. +The generated standalone `../iOS/Fini/Fini.xcodeproj` can be opened directly in Xcode. +Its FiniXCFramework scheme builds the adapter and prepares the separate shared +runtime. SDK sources are referenced locally and are never copied into plugin ZIPs. + +## Windows compatibility and reviewed shared changes + +The original condition IDs 0..3, action IDs 0..19 and expression IDs 0..10 are +preserved; `tests/fini/ACE.csv` records parameter order/types and generated events. Version 2/3 edit properties and keep-over-frame state are supported. +Object serialization uses Win32's 4-byte flags, 16-byte numeric CValue records, +little-endian position integers and NUL-terminated UTF-16LE strings, with the +target object's value/string counts. Invalid or mismatched object blobs are +rejected before changing the target. Runtime object fields themselves are never +reinterpreted as Windows structures. Numeric parsing uses the existing `_stof`. + +Encryption reuses legacy Win32 key derivation, AES-128-CBC and PKCS#7, with raw +ciphertext and no new header. Empty passwords mean plaintext. Win64-generated +short-password assets require explicitly different derivation and are not +silently auto-detected. INI and Base64 strings are UTF-8. SimpleIni emits native +line endings (LF on mobile, CRLF on Windows); both load on either platform. + +Shared changes in the PR for author review: + +- Extracted SimpleIni create/set/delete/copy/save operations to `IniCore.h`. + Windows create/set/delete/copy now call that code too. +- Narrowed `StrNum.h`'s include from the umbrella to its actual StringUtilities + dependency so native builds do not require iconv for numeric parsing. +- Narrowed zlibInterface's include to PathUtilities, guarded MSVC linker pragmas, + and added an optional output bound plus zero-size/growth checks. The existing + four-argument API remains valid; mobile sets a 64 MiB decompression limit. + +Intentional defensive differences: uninitialized access is safe; malformed +Base64/object data produces an empty/false result and Android `lastError` (iOS +logs the error); failed file loads retain existing INI state. Failed auto save +stops data replacement instead of discarding modified data. Release suppresses +auto save. A destroyed localization link falls back to the original text. +Android relative output names resolve under the app's files directory. Reads use +Fusion's `openHFile`, including embedded resources; writes require filesystem +paths, not content-provider URIs. iOS uses `getPathForWriting`/`loadResourceData`. + +## Android developer publishing + +Python 3.9+, JDK compatible with the installed exporter, CMake 3.22+, Ninja and +NDK 28+ are required. The existing Android development configuration selects +NDK 28.2.13676358. The standalone Fini publisher builds native API 26 and defaults +to armeabi-v7a, arm64-v8a, x86 and x86_64. Supply an OpenSSL static/PIC build for +each ABI at `//include` and `lib/libcrypto.a`, built for the +same native API/STL. Put OpenSSL's LICENSE.txt at that root, or pass --openssl-license. The publisher +includes it and the NDK NOTICE with the redistributed libraries. +No host Linux crypto library may be used in an Android build. + +```sh +python3 Extensions/_Script/Fini.py --ndk /path/to/ndk/28.2.13676358 \ + --openssl /path/to/android-openssl --cmake /path/to/cmake --ninja /path/to/ninja +``` + +`Fini.bat` is the Windows launcher for the same publisher. VS `Run_Only Unicode` +invokes it after Build, including Java-only changes; Release and Debug remain +Windows builds. Toolchain settings can also use ANDROID_SDK_ROOT/ANDROID_HOME, +ANDROID_NDK_HOME and FINI_OPENSSL_ROOT. Optional `--fusion ` +installs into that instance only after successful compilation/validation. +The publisher preserves `work/fini-android/publisher.log` and fails on missing +build tools or target OpenSSL; it does not substitute stale native libraries. + +Outputs: + +- `Extensions/Fini/ToInstall/Files/Data/Runtime/Android/Fini.zip` +- `Extensions/Fini_Release.zip` + +The runtime ZIP owns the two Java classes, each selected ABI's libfini.so, +additive JNI/R8 rules and the plugin's source/license material. It carries a +manifest dependency on OFEEncryption/1, not a private crypto or STL implementation. +The separate common package is +`Extensions/Android_Gradle/cmake/Encryption/ToInstall/OFEEncryption.zip`. +Stage it once in an exported Android project using the common module's installer +or the publisher's `--android-project` option. Automatic dependency selection by +Fusion itself has not been established; merely copying the common ZIP into +Data/Runtime/Android is insufficient evidence of integration. + +NDK discovery/preparation follows FTW's workflow: explicit paths, SDK environment +variables and local.properties, platform defaults, SDK/VS/PATH CMake/Ninja, then +sdkmanager for missing pinned components. If Command-line Tools are absent, the +publisher can fetch the pinned revision using Google metadata and its checksum. +Run `Fini.py --prepare-only` to verify tool readiness. Existing SDK licenses are +respected; first-time installations need the SDK's license setup. Build logs and +Android user caches stay under work/. On Unix, absent target OpenSSL prefixes are +prepared from pinned source by the common library's script. Native Windows builds +can supply ABI-specific OpenSSL with --openssl. + +64-bit native libraries are checked for 16 KiB load alignment; 32-bit NDK STL +libraries retain their shipped 4 KiB alignment. Source/SDK Runtime/Gradle template +replacements are excluded. Fusion's ordinary APK export does not compile this C++. + +## iOS developer build (separate workflow) + +The ready project and instructions are in `../iOS/Fini/README.md`. On your Mac: + +```sh +python3 Extensions/iOS/Fini/generate_project.py --runtime /path/to/exported-project +open Extensions/iOS/Fini/Fini.xcodeproj +``` + +Select FiniXCFramework and Product > Build. It uses Xcode 15+/iOS 17+, the licensed +Runtime headers, and the shared OpenSSL preparation script. An explicit target +OpenSSL prefix can be configured with --openssl; otherwise a common cache is used. +Outputs are `work/fini-ios/Fini.xcframework` (adapter, no crypto) and +`work/mobile-encryption/ios/OFEEncryption.xcframework` (shared dynamic runtime). +Embed/sign OFEEncryption once in the consuming app; add Fini's library and retain +its class with -ObjC. Link libc++, Foundation and libz. Other plugins use the same +framework. Do not merge libcrypto into individual plugin archives. + +Runtime sources live in iOS/Fini/Runtime; licensed SDK sources stay local. +Windows creates and round-trips Fini.ext using the shared codec +from PR #100. The exported configurator links native archives and embeds the +common framework once. Apple compilation, Fusion export/final-link and device +validation remain separate checks on the author's Mac. + +## Validation performed in this environment + +```sh +export JAVA_HOME=/workspace/java-toolchain/usr/lib/jvm/java-21-openjdk-amd64 +export PATH="$JAVA_HOME/bin:$PATH" +python3 Extensions/Android_Gradle/tests/fini/run.py \ + --cmake /workspace/onboarding-venv/bin/cmake --json work/fini-json +python3 Extensions/Android_Gradle/tests/fini/test_publisher.py +python3 Extensions/Android_Gradle/tests/object_helpers/run.py +git diff --check +``` + +The test-only JSON-java 20240303 sources were fetched from stleary/JSON-java into +work/fini-json; Android uses its built-in org.json. The host runner can instead +accept a local JSON-java JAR. Runtime test doubles exercise the actual production +Java adapter with the actual JNI dynamic library and SDK CValue source. + +Passed: shared-core INI/UTF-8/case lookup, numeric values, section copy/delete, +Base64/zlib, invalid and truncated inputs, Windows object/position formats, +auto save/release, localization, all 20 actions/11 expressions/4 conditions, +Unicode/NUL JNI strings, invalid/closed handles, cross-frame transfer, destruction +inside an iteration callback, exact decryption and ciphertext reproduction of +the existing NeoPic Windows sample. Two publisher tests cover SDK exclusion, +additive R8 rules, ABI/alignment rejection and output preservation on failure. +The unchanged object-helper suite passed 56 checks. + +Windows helpers ObjectBase/ObjectSelection/ObjectCreation were compared with +reviewed revision 234676b7ed306f19a35ab6d69519a0ba6aa52598: no changes, so no helper +synchronization or reviewed-revision update was needed. Android's existing runtime +representations and documented platform differences remain intact. + +The NDK and CMake were subsequently prepared through the new publisher flow. +Shared runtime and plugin target-ABI builds are validated separately from the host +tests. The final binary/package audit checks all four ABI outputs, DT_NEEDED, +shared API exports, absence of crypto definitions in libfini, package ownership, +and one-time common-runtime staging. Xcode project syntax was parsed with the +independent OpenStep parser 2.0.3. Generated IDs/references/schemes were also checked. +Not established here: Windows MSBuild, a full Fusion Android APK build, Apple +compilation/linking, real Fusion .ext export/final-link and device validation. + +A review source bundle (not an installable runtime) can be created with: + +```sh +python3 Extensions/_Script/Fini.py --source-bundle work/Fini-mobile-source.zip +``` + +## Mac build/push -> Windows pull/VS packaging + +After a successful FiniXCFramework build, the scripts stage these owned artifacts: + +- `Extensions/iOS/Fini/Prebuilt/Fini.xcframework.zip` and `manifest.json`. +- `Extensions/iOS/Encryption/Prebuilt/OFEEncryption.xcframework.zip` + and `manifest.json`, shared once by all dependent plugins. + +The author commits/pushes those four files after building on Mac. No script pushes +Git automatically. On Windows, git pull followed by VS Run_Only Unicode invokes +the existing Fini.bat/Fini.py entry point. It validates SHA256, required slices and +source freshness, then updates Fini-iOS.zip and the separate OFEEncryption-iOS.zip, +installs them when FusionRoot/the normal Fusion directory exists, and rebuilds +Fini_Release.zip. Windows does not need Xcode. To run just this handoff step: + +```sh +python Extensions/_Script/Fini.py --ios-only +``` + +No Mac binaries are included until the author builds them. Before that first +build the publisher reports the missing handoff and leaves Apple outputs alone; +an incomplete, corrupt or source-stale committed pair fails instead of replacing +a known-good package. The handoff creates native XCFramework distribution ZIPs; +Windows also generates a format-verified Fini.ext with the shared PR #100 codec. +Actual exporter round-trip, final-link and device checks remain separate. diff --git a/Extensions/Android_Gradle/app/.gitignore b/Extensions/Android_Gradle/app/.gitignore index 5cd56ed6c..a94c6dd4f 100644 --- a/Extensions/Android_Gradle/app/.gitignore +++ b/Extensions/Android_Gradle/app/.gitignore @@ -30,3 +30,5 @@ !/src/main/java/Extensions/CExtLoad.java !/src/main/java/Extensions/CRunTemplate.java + +!/src/main/java/Extensions/CRunFini.java diff --git a/Extensions/Android_Gradle/app/src/main/cpp/Encryption/CMakeLists.txt b/Extensions/Android_Gradle/app/src/main/cpp/Encryption/CMakeLists.txt new file mode 100644 index 000000000..1a56318b8 --- /dev/null +++ b/Extensions/Android_Gradle/app/src/main/cpp/Encryption/CMakeLists.txt @@ -0,0 +1,18 @@ +cmake_minimum_required(VERSION 3.22) +project(OFEMobileEncryption LANGUAGES CXX) +# One runtime dependency for every plugin. Consumers include only CAPI.h. +if(ANDROID AND OFE_OPENSSL_ROOT) + set(OPENSSL_INCLUDE_DIR "${OFE_OPENSSL_ROOT}/include" CACHE PATH "Target crypto headers" FORCE) + set(OPENSSL_CRYPTO_LIBRARY "${OFE_OPENSSL_ROOT}/lib/libcrypto.a" CACHE FILEPATH "Target static crypto inside common runtime" FORCE) +endif() +find_package(OpenSSL REQUIRED COMPONENTS Crypto) +add_library(ofe_encryption SHARED "${CMAKE_CURRENT_LIST_DIR}/../../../../../../_DeLib/Encryption/CAPI.cpp") +target_compile_features(ofe_encryption PRIVATE cxx_std_17) +target_include_directories(ofe_encryption PUBLIC "${CMAKE_CURRENT_LIST_DIR}/../../../../../../_DeLib/Encryption") +target_compile_definitions(ofe_encryption PRIVATE ENCRYPTION_BUILD ENCRYPTION_SHARED _FILE_OFFSET_BITS=64) +target_link_libraries(ofe_encryption PRIVATE OpenSSL::Crypto) +set_target_properties(ofe_encryption PROPERTIES CXX_VISIBILITY_PRESET hidden VISIBILITY_INLINES_HIDDEN ON) +if(ANDROID) + target_link_libraries(ofe_encryption PRIVATE dl) + target_link_options(ofe_encryption PRIVATE "-Wl,-z,max-page-size=16384" "-Wl,--exclude-libs,ALL") +endif() diff --git a/Extensions/Android_Gradle/app/src/main/cpp/Fini/CMakeLists.txt b/Extensions/Android_Gradle/app/src/main/cpp/Fini/CMakeLists.txt new file mode 100644 index 000000000..5f672f53d --- /dev/null +++ b/Extensions/Android_Gradle/app/src/main/cpp/Fini/CMakeLists.txt @@ -0,0 +1,26 @@ +cmake_minimum_required(VERSION 3.22) +project(FiniMobile LANGUAGES CXX) +if(OFE_ENCRYPTION_LIBRARY) + add_library(ofe_encryption SHARED IMPORTED) + set_target_properties(ofe_encryption PROPERTIES IMPORTED_LOCATION "${OFE_ENCRYPTION_LIBRARY}") +elseif(NOT TARGET ofe_encryption) + add_subdirectory("${CMAKE_CURRENT_LIST_DIR}/../Encryption" common-encryption) +endif() +find_package(ZLIB REQUIRED) +add_library(fini_core INTERFACE) +target_compile_features(fini_core INTERFACE cxx_std_20) +target_include_directories(fini_core INTERFACE "${CMAKE_CURRENT_LIST_DIR}/../../../../../../Fini" "${CMAKE_CURRENT_LIST_DIR}/../../../../../../_DeLib") +target_compile_definitions(fini_core INTERFACE _FILE_OFFSET_BITS=64 FINI_PORTABLE_RUNTIME) +target_link_libraries(fini_core INTERFACE ofe_encryption ZLIB::ZLIB) +option(FINI_BUILD_TESTS "Build host core tests" OFF) +if(FINI_BUILD_TESTS) + enable_testing() + add_executable(fini_tests ../../../../../tests/fini/core.cpp) + target_link_libraries(fini_tests PRIVATE fini_core) + add_test(NAME fini_core COMMAND fini_tests) + # Compile the original API without the mobile opt-in or its dependencies. + add_executable(fini_default_ini ../../../../../tests/fini/default_ini.cpp) + target_compile_features(fini_default_ini PRIVATE cxx_std_17) + target_include_directories(fini_default_ini PRIVATE "${CMAKE_CURRENT_LIST_DIR}/../../../../../../Fini") + add_test(NAME fini_default_ini COMMAND fini_default_ini) +endif() diff --git a/Extensions/Android_Gradle/app/src/main/cpp/Fini/FiniJNI.cpp b/Extensions/Android_Gradle/app/src/main/cpp/Fini/FiniJNI.cpp new file mode 100644 index 000000000..835c8137f --- /dev/null +++ b/Extensions/Android_Gradle/app/src/main/cpp/Fini/FiniJNI.cpp @@ -0,0 +1,46 @@ +#include "IniCore.h" +#include +#include +#include +#include + +namespace { +std::mutex handlesMutex; +std::unordered_map> handles; +std::atomic nextHandle{1}; +std::shared_ptr retain(jlong handle) { + std::lock_guard lock(handlesMutex); + auto it=handles.find(handle);fini::check(it!=handles.end(),"Closed or invalid Fini handle");return it->second; +} +std::string text(JNIEnv* env,jstring value) { + fini::check(value!=nullptr,"Null JNI string"); + const auto* units=env->GetStringChars(value,nullptr); + fini::check(units!=nullptr,"Read JNI string"); + std::u16string copy; + try {copy.assign(reinterpret_cast(units),env->GetStringLength(value));} + catch(...){env->ReleaseStringChars(value,units);throw;} + env->ReleaseStringChars(value,units); + return std::wstring_convert,char16_t>{}.to_bytes(copy); +} +void fail(JNIEnv* env,const std::exception& error) { + if(env->ExceptionCheck())return; + auto cls=env->FindClass("java/lang/IllegalArgumentException"); + if(cls){env->ThrowNew(cls,error.what());env->DeleteLocalRef(cls);} +} +} +extern "C" JNIEXPORT jlong JNICALL Java__1DeLib_FiniNative_create(JNIEnv* env,jclass) { + try {auto state=std::make_shared();auto handle=nextHandle.fetch_add(1);std::lock_guard lock(handlesMutex);handles.emplace(handle,std::move(state));return handle;} + catch(const std::exception& e){fail(env,e);return 0;} +} +extern "C" JNIEXPORT void JNICALL Java__1DeLib_FiniNative_destroy(JNIEnv*,jclass,jlong handle) { + std::lock_guard lock(handlesMutex);handles.erase(handle); +} +extern "C" JNIEXPORT jstring JNICALL Java__1DeLib_FiniNative_request(JNIEnv* env,jclass,jlong handle,jstring op,jstring args) { + try { + auto state=retain(handle); + // All requests run on Fusion's runtime thread; no callbacks occur while native state is in use. + auto result=state->request(text(env,op),fini::Json::parse(text(env,args))); + auto units=to_utf16_string(result.dump()); + return env->NewString(reinterpret_cast(units.data()),units.size()); + }catch(const std::exception& e){fail(env,e);return nullptr;} +} diff --git a/Extensions/Android_Gradle/app/src/main/java/Extensions/CExtLoad.java b/Extensions/Android_Gradle/app/src/main/java/Extensions/CExtLoad.java index 665871aa7..6ddf6b472 100644 --- a/Extensions/Android_Gradle/app/src/main/java/Extensions/CExtLoad.java +++ b/Extensions/Android_Gradle/app/src/main/java/Extensions/CExtLoad.java @@ -84,6 +84,10 @@ else if (name.compareToIgnoreCase("OUYA")==0) // STARTCUT // STARTCUT // STARTCUT + if (name.compareToIgnoreCase("Fini")==0) + { + object=new CRunFini(); + } if (name.compareToIgnoreCase("FindTheWay")==0) { object=new CRunFindTheWay(); diff --git a/Extensions/Android_Gradle/app/src/main/java/Extensions/CRunFini.java b/Extensions/Android_Gradle/app/src/main/java/Extensions/CRunFini.java new file mode 100644 index 000000000..0745f16d1 --- /dev/null +++ b/Extensions/Android_Gradle/app/src/main/java/Extensions/CRunFini.java @@ -0,0 +1,121 @@ +package Extensions; + +import Actions.CActExtension; +import Conditions.CCndExtension; +import Expressions.CValue; +import Objects.CObject; +import Objects.CExtension; +import RunLoop.CCreateObjectInfo; +import Services.CBinaryFile; +import _DeLib.FiniNative; +import org.json.*; +import java.util.*; +import java.io.*; +import android.util.Base64; +import Runtime.MMFRuntime; +import Application.CRunApp; + +/** Fini's original ACE IDs. Runtime objects and events are accessed only in Java. */ +public class CRunFini extends CRunExtension { + private long state; + private boolean cf25p, allowRVforCS, keep; + private int localizationFixed; + private boolean linked; + private String section="", item="", sectionLoop="", itemLoop=""; + private static final Map> saved = new WeakHashMap<>(); + public String lastError=""; + + private Object call(String op, Object... args) { + if(state==0) throw new IllegalStateException("Closed Fini object"); + try { return new JSONTokener(FiniNative.request(state,op,new JSONArray(Arrays.asList(args)).toString())).nextValue(); } + catch(JSONException e) { throw new IllegalArgumentException(e); } + } + private String str(String op,Object... a) { return (String)call(op,a); } + private String writePath(String path) {return new File(path).isAbsolute()?path:new File(MMFRuntime.inst.getFilesDir(),path).getPath();} + private String readBytes(String path) throws IOException { + String privatePath=writePath(path); + CRunApp.HFile file=rh.rhApp.openHFile(new File(privatePath).isFile()?privatePath:path); + if(file==null||file.stream==null)throw new IOException("Open INI input"); + try {ByteArrayOutputStream bytes=new ByteArrayOutputStream();byte[] buffer=new byte[16384];int n;while((n=file.stream.read(buffer))!=-1)bytes.write(buffer,0,n);return Base64.encodeToString(bytes.toByteArray(),Base64.NO_WRAP);} + finally {file.close();} + } + private String objectName() {return ho.hoOiList.oilName;} + @Override public int getNumberOfConditions(){return 4;} + @Override public boolean createRunObject(CBinaryFile file,CCreateObjectInfo cob,int version) { + cf25p=version>=2&&file.readByte()!=0; allowRVforCS=version>=2&&file.readByte()!=0; keep=version>=3&&file.readByte()!=0; + state=FiniNative.create(); + if(keep) {Map data=saved.get(rh.rhApp);if(data!=null){String value=data.remove(objectName());if(value!=null)call("load",value);}} + return state!=0; + } + @Override public void destroyRunObject(boolean fast) { + if(state==0)return; + try {call("flush");}catch(RuntimeException e){lastError=e.getMessage();} + try {if(keep&&(Boolean)call("initialized")) saved.computeIfAbsent(rh.rhApp,k->new HashMap<>()).putIfAbsent(objectName(),str("save"));} + finally {FiniNative.destroy(state);state=0;} + } + private CObject object(int fixed) {CObject o=ho.getObjectFromFixed(fixed);return o!=null&&(o.hoFlags&CObject.HOF_DESTROYED)==0?o:null;} + private CRunFini linkedObject() {CObject o=object(localizationFixed);if(o instanceof CExtension&&((CExtension)o).ext instanceof CRunFini)return (CRunFini)((CExtension)o).ext;return null;} + @Override public int handleRunObject() {return 0;} + private void updateLocalization() {if(linked){CRunFini o=linkedObject();call("localData",o!=null&&o.state!=0?o.str("save"):"");if(o==null)linked=false;}} + @Override public boolean condition(int n,CCndExtension c) { + try {switch(n){ + case 0:return sectionLoop.equals(c.getParamExpString(rh,0)); + case 1:return itemLoop.equals(c.getParamExpString(rh,0)); + case 2:return (Boolean)call("hasItem",c.getParamExpString(rh,0),c.getParamExpString(rh,1)); + case 3:return (Boolean)call("hasSection",c.getParamExpString(rh,0)); + }}catch(RuntimeException e){lastError=e.getMessage();}return false; + } + private boolean valuesAllowed(CObject o) {return o!=null&&o.rov!=null&&(o.hoType>=32||o.hoType==2||((o.hoType==3||o.hoType==7)&&cf25p&&allowRVforCS));} + private int valueCount(CObject o){return cf25p?o.rov.rvNumberOfValues:Math.min(26,o.rov.rvNumberOfValues);} + private int stringCount(CObject o){return cf25p?o.rov.rvNumberOfStrings:Math.min(10,o.rov.rvNumberOfStrings);} + private String packValues(CObject o) throws JSONException { + if(!valuesAllowed(o))return ""; + JSONObject data=new JSONObject();data.put("flags",o.rov.rvValueFlags&0xffffffffL); + JSONArray values=new JSONArray(),strings=new JSONArray(); + for(int i=0;i` and is reused by plugin publishers. +The supplied Fini publisher builds/imports this common dependency, then links its +own libfini.so against it. It does not statically link encryption or libcrypto. + +`Android_Gradle/cmake/Encryption/android_runtime.py` creates its own +`ToInstall/OFEEncryption.zip`, containing one +common library and matching libc++_shared.so per ABI, shared source/license +material, ABI version and SHA256 manifest. Plugin ZIPs must not contain these +common runtime binaries or their implementations. The client header is allowed. + +For an exported Android project, stage the shared package **once**: + +```sh +python3 Extensions/Android_Gradle/cmake/Encryption/android_runtime.py \ + --archive Extensions/Android_Gradle/cmake/Encryption/ToInstall/OFEEncryption.zip \ + --project /path/to/exported-android-project +``` + +Or pass `--android-project /path/to/project` to the Fini publisher. This only +adds the common native libraries; it never replaces Gradle or Runtime templates. +Repeated installation of the identical runtime is a no-op. Conflicting libraries +are rejected before copying anything. Every dependent plugin uses the same +SONAME and the APK must contain one copy per ABI. A separate common ZIP is not +automatically selected by Fusion merely by being placed in Data/Runtime/Android; +use this staging step for the exported project. Automatic common-dependency +selection by the proprietary Fusion exporter has not been established here. + +The C++ implementation remains in `_DeLib/Encryption`. Dependency preparation +uses `Extensions/_Script/Encryption/prepare_openssl.py`; no cipher is rewritten. diff --git a/Extensions/Android_Gradle/cmake/Encryption/android_runtime.py b/Extensions/Android_Gradle/cmake/Encryption/android_runtime.py new file mode 100644 index 000000000..bf916b040 --- /dev/null +++ b/Extensions/Android_Gradle/cmake/Encryption/android_runtime.py @@ -0,0 +1,91 @@ +"""Shared Android encryption build/package/install functions for any plugin. +Consumers link libofe_encryption.so; common binaries never belong in plugin ZIPs. +""" +import hashlib,json,os,platform,shutil,struct,zipfile +from pathlib import Path +EXT=Path(__file__).resolve().parents[3] +ABIS={'armeabi-v7a':(1,40,'arm-linux-androideabi'),'arm64-v8a':(2,183,'aarch64-linux-android'),'x86':(1,3,'i686-linux-android'),'x86_64':(2,62,'x86_64-linux-android')} +def validate_elf(path,abi): + data=Path(path).read_bytes();bits,machine,_=ABIS[abi] + if len(data)<64 or data[:4]!=b'\x7fELF' or data[4]!=bits or data[5]!=1 or struct.unpack_from('len(data):raise RuntimeError(f'Truncated ELF: {path}') + if struct.unpack_from(' actions=new HashSet<>(),expressions=new HashSet<>(),conditions=new HashSet<>(); + static void check(boolean value,String message){if(!value)throw new AssertionError(message);} + static void act(CRunFini o,int n,Object... p){actions.add(n);o.action(n,new CActExtension(p));check(o.lastError.isEmpty(),"Action "+n+": "+o.lastError);} + static CValue exp(CRunFini o,int n,Object... p){expressions.add(n);for(Object v:p)o.ho.params.add(v instanceof String?new CValue((String)v):new CValue(((Number)v).intValue()));return o.expression(n);} + static boolean cnd(CRunFini o,int n,Object... p){conditions.add(n);return o.condition(n,new CCndExtension(p));} + static CRunFini create(){CRunFini o=new CRunFini();check(o.createRunObject(new CBinaryFile(1,1,1),null,3),"Create");return o;} + public static void main(String[] args)throws Exception { + long h=FiniNative.create();String text="中文 😀\u0000end";String q=org.json.JSONObject.quote(text); + check(FiniNative.request(h,"decode","["+FiniNative.request(h,"encode","["+q+",false]")+",false]").equals(q),"JNI UTF16/NUL"); + FiniNative.destroy(h);FiniNative.destroy(h);boolean rejected=false;try{FiniNative.request(h,"save","[]");}catch(IllegalArgumentException e){rejected=true;}check(rejected,"Validated closed handle"); + CRunFini o=create();check(o.getNumberOfConditions()==4,"Conditions count");check(!cnd(o,3,"A"),"Uninitialized"); + act(o,0);act(o,5,"A","key","你好 😀");act(o,4,"A","number",15.2);check(exp(o,1,"a","KEY").getString().equals("你好 😀"),"String");check(Math.abs(exp(o,0,"A","number").getDouble()-15.2)<0.0001,"Number"); + check(cnd(o,2,"A","key"),"Has item");act(o,12,"A","B",0);act(o,6,"B","key");check(!cnd(o,2,"B","key"),"Delete"); + List seen=new ArrayList<>();o.ho.listener=n->{if(n==0){check(cnd(o,0,"sections"),"Section event");seen.add(exp(o,2).getString());}else{check(cnd(o,1,"items"),"Item event");seen.add(exp(o,3).getString());}}; + act(o,7,"sections");check(seen.contains("A")&&seen.contains("B"),"Sections");seen.clear();act(o,8,"A","items");check(seen.contains("key")&&seen.contains("number"),"Items"); + String ini=exp(o,4).getString(),b64=exp(o,8).getString(),compressed=exp(o,7).getString(); + act(o,1);act(o,19,b64);check(exp(o,4).getString().equals(ini),"Base64 load");act(o,15,compressed);check(exp(o,4).getString().equals(ini),"Compressed load");act(o,9,ini); + for(int zip=0;zip<2;zip++){String value=exp(o,9,"text 中文 😀",zip).getString();check(exp(o,10,value,zip).getString().equals("text 中文 😀"),"String codec");} + CObject object=new CObject();object.hoX=-42;object.hoY=12;object.rov.rvValueFlags=0x80000001;object.rov.rvValues[0].forceDouble(15.25);object.rov.rvValues[1].forceInt(-2);object.rov.rvStrings[0]="你好 😀";o.ho.objects.put(123,object); + String values=exp(o,5,123).getString(),position=exp(o,6,123).getString();object.hoX=0;object.hoY=0;object.rov.rvStrings[0]="changed";act(o,13,123,values);act(o,14,123,position);check(object.hoX==-42&&object.hoY==12&&object.rov.rvStrings[0].equals("你好 😀")&&object.rov.rvValueFlags==0x80000001,"Object restore"); + Path file=Files.createTempFile("fini-jni",".ini");Path local=Files.createTempFile("fini-local",".ini"); + act(o,3,file.toString(),"密码 😀");act(o,2,file.toString(),"密码 😀");check(exp(o,1,"A","key").getString().equals("你好 😀"),"Encrypted file");act(o,10,file.toString(),"密码 😀");act(o,11); + Files.write(local,"[zh]\n你好 😀=翻译\n".getBytes("UTF-8"));act(o,16,local.toString(),"");act(o,17,"zh");check(exp(o,1,"A","key").getString().equals("翻译"),"Localization file"); + CRunFini other=create();act(other,5,"zh","你好 😀","链接");o.ho.objects.put(456,other.ho);act(o,18,456);check(exp(o,1,"A","key").getString().equals("链接"),"Localization link");other.destroyRunObject(false);check(exp(o,1,"A","key").getString().equals("你好 😀"),"Destroyed link"); + o.action(15,new CActExtension("invalid"));check(!o.lastError.isEmpty(),"Bad input recorded");act(o,5,"A","recovery","OK"); + Object app=o.rh.rhApp;o.destroyRunObject(false);o.destroyRunObject(false);CRunFini next=new CRunFini();next.rh.rhApp=(Application.CRunApp)app;next.createRunObject(new CBinaryFile(1,1,1),null,3);check(exp(next,1,"A","recovery").getString().equals("OK"),"Cross-frame state");next.destroyRunObject(false); + // Reentrant event may destroy its own runtime object safely. + CRunFini destroyed=create();act(destroyed,5,"A","B","C");destroyed.ho.listener=n->destroyed.destroyRunObject(false);act(destroyed,7,"destroy"); + check(actions.size()==20&&expressions.size()==11&&conditions.size()==4,"Complete ACE coverage"); + Files.delete(file);Files.delete(local); + System.out.println("PASS: real JNI/Java adapter; all 20 actions,11 expressions,4 conditions; SDK doubles only"); + } +} diff --git a/Extensions/Android_Gradle/tests/fini/audit_native.py b/Extensions/Android_Gradle/tests/fini/audit_native.py new file mode 100644 index 000000000..368105cd0 --- /dev/null +++ b/Extensions/Android_Gradle/tests/fini/audit_native.py @@ -0,0 +1,59 @@ +#!/usr/bin/env python3 +"""Audit real Android binaries and package ownership; not an APK/device test.""" +import importlib.util +import json +from pathlib import Path +import subprocess +import tempfile +import zipfile + +repo=Path(__file__).resolve().parents[4] +root=repo/'Extensions/Android_Gradle/cmake/Encryption' +spec=importlib.util.spec_from_file_location('common_encryption',root/'android_runtime.py') +common=importlib.util.module_from_spec(spec);spec.loader.exec_module(common) +tools=json.loads((repo/'work/fini-android/toolchain.json').read_text()) +llvm=next((Path(tools['ndk'])/'toolchains/llvm/prebuilt').iterdir())/'bin' +fini=repo/'Extensions/Fini/ToInstall/Files/Data/Runtime/Android/Fini.zip' +archive=root/'ToInstall/OFEEncryption.zip' + +def inspect(tool,*args): + return subprocess.check_output([str(llvm/tool),*map(str,args)],text=True) + +def check(value,message): + if not value:raise AssertionError(message) + +with zipfile.ZipFile(fini) as plugin,zipfile.ZipFile(archive) as shared: + names=plugin.namelist();shared_names=shared.namelist() + check(not any(name.endswith(('libofe_encryption.so','libc++_shared.so','libcrypto.a','libcrypto.so')) for name in names),'Plugin contains common runtime binaries') + check(not any('/Encryption/' in name and name.endswith(('CAPI.cpp','Core.h','KeyGen.h')) for name in names),'Plugin contains a private encryption implementation') + check(json.loads(plugin.read('Fini-manifest.json'))['requires']==['OFEEncryption/1'],'Missing runtime dependency') + for abi in common.ABIS: + client=repo/'work/fini-android'/abi/'libfini.so' + library=repo/'work/mobile-encryption/android'/abi/'libofe_encryption.so' + common.validate_elf(client,abi);common.validate_elf(library,abi) + dynamic=inspect('llvm-readelf','-d',client) + check('libofe_encryption.so' in dynamic,'Client does not reference common SONAME: '+abi) + defined=inspect('llvm-nm','-D','--defined-only',client) + check('EVP_' not in defined and 'Encryption_EncryptBuffer' not in defined,'Crypto embedded in client: '+abi) + exported=inspect('llvm-nm','-D','--defined-only',library) + check('Encryption_EncryptBuffer' in exported and 'Encryption_DeriveKeyAndIv' in exported,'Missing shared C ABI: '+abi) + check('EVP_' not in exported,'EVP leaked from shared ABI: '+abi) + key=f'app/src/main/jniLibs/{abi}/libofe_encryption.so' + check(shared.read(key)==library.read_bytes(),'Packaged shared runtime differs: '+abi) + key=f'app/src/main/jniLibs/{abi}/libfini.so' + check(plugin.read(key)==client.read_bytes(),'Packaged client differs: '+abi) + manifest=json.loads(shared.read('OFEEncryption-manifest.json')) + check(manifest['abiVersion']==1 and len(manifest['abis'])==4,'Common runtime ABI manifest') + +with tempfile.TemporaryDirectory(prefix='fini-shared-audit-',dir=repo/'work') as temp: + project=Path(temp);(project/'app').mkdir() + common.install_once(archive,project) + common.install_once(archive,project) + files=list((project/'app/src/main/jniLibs').rglob('*.so')) + check(len(files)==8,'Expected one encryption and one STL per ABI') + conflict=project/'app/src/main/jniLibs/arm64-v8a/libofe_encryption.so';conflict.write_bytes(b'conflicting runtime') + try:common.install_once(archive,project) + except RuntimeError:pass + else:raise AssertionError('Conflicting common runtime was silently overwritten') + check(conflict.read_bytes()==b'conflicting runtime','Existing runtime was overwritten') +print('PASS: four real ABI builds, 64-bit 16 KiB alignment, common SONAME/C exports, no crypto in client, separate packages and one-time staging') diff --git a/Extensions/Android_Gradle/tests/fini/core.cpp b/Extensions/Android_Gradle/tests/fini/core.cpp new file mode 100644 index 000000000..fc76c5d29 --- /dev/null +++ b/Extensions/Android_Gradle/tests/fini/core.cpp @@ -0,0 +1,57 @@ +#include "IniCore.h" +#include +using fini::check; +using fini::Json; +template void rejects(F f){bool caught=false;try{f();}catch(const std::exception&){caught=true;}check(caught,"Expected failure");} +int main(){ + fini::State state; + check(!state.request("hasSection",{"A"}).get(),"Uninitialized section"); + check(!state.request("hasItem",{"A","B"}).get(),"Uninitialized item"); + check(state.save().empty(),"Uninitialized save"); + state.request("set",{"Section","Item","你好 🌍"}); + check(state.get("section","ITEM")=="你好 🌍","INI case/UTF8"); + check(state.modified,"Set modified"); + state.request("set",{"Section","float","-15.2"}); + check(std::abs(state.request("number",{"SECTION","float"}).get()+15.2)<0.00001,"Float"); + state.request("set",{"Section","text","nonnumeric"}); + check(state.request("number",{"Section","text"})==0,"Nonnumeric"); + check(state.request("number",{"missing","none"})==0,"Missing number"); + state.request("copy",{"Section","Copy",false}); + check(state.get("Copy","Item")=="你好 🌍","Copy"); + state.request("delete",{"Copy","Item"});check(!state.request("hasItem",{"Copy","Item"}).get(),"Delete item"); + state.request("delete",{"Copy",""});check(!state.request("hasSection",{"Copy"}).get(),"Delete section"); + auto snapshot=state.request("sections");state.release();check(!snapshot.empty(),"Owned iteration snapshot"); + auto original=std::string("[Mixed]\r\nvalue = 中文 😀\r\nnum=1.00000\r\n");state.load(original); + fini::State other;other.load(state.save());check(other.get("mixed","VALUE")=="中文 😀","Save/load"); + for(bool zipped:{false,true}){auto encoded=state.request("encode",{state.save(),zipped});auto decoded=state.request("decode",{encoded,zipped}).get();check(decoded==state.save(),"Base64 roundtrip");} + check(fini::base64("hello")=="aGVsbG8=","Base64 known answer"); + check(fini::unbase64("AAECA/8=")==std::string("\0\1\2\3\xff",5),"Base64 binary known answer"); + rejects([]{fini::unbase64("bad");});rejects([]{fini::unbase64("ab==");});rejects([]{fini::decompress("");});rejects([]{fini::decompress("invalid");}); + auto zipped=fini::compress(std::string(100000,'a'));check(fini::decompress(zipped)==std::string(100000,'a'),"High compression ratio"); + rejects([&]{fini::decompress(zipped.substr(0,zipped.size()-1));}); + check(state.request("packPosition",{-1,42})=="/////yoAAAA=","Win32 position vector"); + check(state.request("unpackPosition",{"/////yoAAAA="})==Json::array({-1,42}),"Win32 position decode");rejects([&]{state.request("unpackPosition",{"AA=="});}); + Json values={{"flags",0x80000001u},{"values",Json::array({{{"type",0},{"value",-42}},{{"type",2},{"value",15.25}}})},{"strings",Json::array({"你好 😀",""})}}; + auto encoded=fini::packValues(values);check(fini::unpackValues(encoded,2,2)==values,"Win32 CValue/UTF16 object state"); + auto bytes=fini::unbase64(encoded);check(bytes.size()==50,"Win32 4-byte flags,16-byte values,UTF16LE");check(static_cast(bytes[0])==1&&static_cast(bytes[3])==128,"Flags LE"); + rejects([&]{fini::unpackValues(encoded,3,2);});rejects([&]{fini::unpackValues(encoded.substr(4),2,2);}); + state.localizeFrom("[zh]\n中文 😀 = 本地文本\n");state.language="zh";check(state.get("mixed","value")=="本地文本","Localization");state.language="missing";check(state.get("mixed","value")=="中文 😀","Localization fallback"); + std::string plain="[数据]\r\n值=你好 😀\r\n",key="密码/😀/long-key"; + check(fini::crypt(fini::crypt(plain,key,true),key,false)==plain,"Shared encryption Unicode password"); + // Independent existing asset fixture verifies exact historical ciphertext reuse. + auto root=std::filesystem::path(__FILE__).parent_path()/"../../.."; + auto asset=root/"NeoPic/ToInstall/Files/Examples/NeoPic"; + auto encrypted=asset/"Test_Encrypt.dat",jpeg=asset/"Test_Origin.jpg"; + if(std::filesystem::exists(encrypted)&&std::filesystem::exists(jpeg)){ + auto cipher=fini::read(encrypted.string(),"");auto clear=fini::read(jpeg.string(),""); + check(fini::crypt(cipher,"12345678ABCDEFGH",false)==clear,"Existing Windows encrypted asset");check(fini::crypt(clear,"12345678ABCDEFGH",true)==cipher,"Exact Windows ciphertext"); + }else throw std::runtime_error("Missing existing asset fixture"); + auto path=std::filesystem::temp_directory_path()/"fini-port-test.ini"; + struct Cleanup{std::filesystem::path p;~Cleanup(){std::filesystem::remove(p);}} cleanup{path}; + fini::State file;file.request("autoOn",{path.string(),key});file.request("set",{"A","B","first"});file.load("[Other]\nitem=value\n"); + fini::State saved;saved.loadFile(path.string(),key);check(saved.get("A","B")=="first","Auto save before replacement"); + saved.request("set",{"A","B","updated"});saved.loadFile(path.string(),key);check(saved.get("A","B")=="updated","Auto save before reading same file"); + auto before=saved.save();rejects([&]{saved.loadFile(path.string()+".missing",key);});check(saved.save()==before,"Failed input preserves state"); + saved.request("set",{"A","B","second"});saved.release();saved.flush();check(fini::read(path.string(),key).find("updated")!=std::string::npos,"Release suppresses auto save"); + std::cout<<"PASS: shared INI, codecs, Win32 formats, existing ciphertext, lifecycle and errors\n"; +} diff --git a/Extensions/Android_Gradle/tests/fini/default_ini.cpp b/Extensions/Android_Gradle/tests/fini/default_ini.cpp new file mode 100644 index 000000000..fd784156f --- /dev/null +++ b/Extensions/Android_Gradle/tests/fini/default_ini.cpp @@ -0,0 +1,14 @@ +#include "IniCore.h" +#include + +#if defined(FINI_PORTABLE_RUNTIME) || defined(NLOHMANN_JSON_VERSION_MAJOR) +#error The default INI API must not enable the mobile Runtime or JSON dependencies. +#endif + +int main() { + std::unique_ptr ini(FiniIni::Create()); + if (!FiniIni::Set(ini.get(), "section", "key", "value")) return 1; + if (!FiniIni::Copy(ini.get(), "section", "copy", false)) return 2; + if (FiniIni::Save(ini.get()).find("value") == std::string::npos) return 3; + return FiniIni::Delete(ini.get(), "copy", "key") ? 0 : 4; +} diff --git a/Extensions/Android_Gradle/tests/fini/fixture.py b/Extensions/Android_Gradle/tests/fini/fixture.py new file mode 100644 index 000000000..ea5cb6ab1 --- /dev/null +++ b/Extensions/Android_Gradle/tests/fini/fixture.py @@ -0,0 +1,48 @@ +"""Small SDK doubles for ACE behavior; not an Android/exporter replacement.""" +from pathlib import Path + +def create(root): + files={ + 'RunLoop/CRun.java': 'package RunLoop; public class CRun {public Application.CRunApp rhApp=new Application.CRunApp();}', + 'RunLoop/CCreateObjectInfo.java': 'package RunLoop; public class CCreateObjectInfo {}', + 'RunLoop/CObjInfo.java':'package RunLoop; public class CObjInfo {public String oilName="Fini";}', + 'Application/CRunApp.java': '''package Application; public class CRunApp { + public class HFile {public java.io.InputStream stream;public void close(){try{stream.close();}catch(java.io.IOException e){throw new RuntimeException(e);}}} + public HFile openHFile(String path){try{HFile f=new HFile();f.stream=new java.io.FileInputStream(path);return f;}catch(java.io.IOException e){return null;}} + }''', + 'Services/CBinaryFile.java':'''package Services; public class CBinaryFile {private int p;private byte[] data;public CBinaryFile(int... values){data=new byte[values.length];for(int i=0;i objects=new java.util.HashMap<>(); + public java.util.ArrayDeque params=new java.util.ArrayDeque<>(); + public java.util.function.IntConsumer listener=n->{}; + public CObject getObjectFromFixed(int fixed){return objects.get(fixed);} + public Expressions.CValue getExpParam(){return params.remove();} + public void generateEvent(int n,int p){listener.accept(n);} + }''', + 'Values/CRVal.java':'''package Values; public class CRVal { + public int rvValueFlags,rvNumberOfValues=2,rvNumberOfStrings=2; + public Expressions.CValue[] rvValues={new Expressions.CValue(0),new Expressions.CValue(0)};public String[] rvStrings={"",""}; + public Expressions.CValue getValue(int n){return rvValues[n];}public String getString(int n){return rvStrings[n];}public void setString(int n,String s){rvStrings[n]=s;} + }''', + 'Actions/CActExtension.java':'''package Actions; public class CActExtension { + Object[] p;public CActExtension(Object... p){this.p=p;} + public String getParamExpString(RunLoop.CRun rh,int n){return (String)p[n];}public String getParamFilename2(RunLoop.CRun rh,int n){return (String)p[n];} + public int getParamExpression(RunLoop.CRun rh,int n){return ((Number)p[n]).intValue();}public float getParamExpFloat(RunLoop.CRun rh,int n){return ((Number)p[n]).floatValue();} + }''', + 'Conditions/CCndExtension.java':'''package Conditions; public class CCndExtension {Object[] p;public CCndExtension(Object... p){this.p=p;}public String getParamExpString(RunLoop.CRun rh,int n){return (String)p[n];}}''', + 'Extensions/CRunExtension.java':'''package Extensions; public abstract class CRunExtension { + public RunLoop.CRun rh=new RunLoop.CRun();public Objects.CExtension ho=new Objects.CExtension(); + public CRunExtension(){ho.ext=this;}public abstract int getNumberOfConditions(); + public abstract boolean createRunObject(Services.CBinaryFile file,RunLoop.CCreateObjectInfo cob,int version); + public abstract void destroyRunObject(boolean fast);public abstract int handleRunObject(); + public abstract boolean condition(int n,Conditions.CCndExtension c);public abstract void action(int n,Actions.CActExtension a);public abstract Expressions.CValue expression(int n); + }''', + 'android/util/Base64.java':'''package android.util; public class Base64 {public static final int NO_WRAP=2;public static String encodeToString(byte[] b,int flags){return java.util.Base64.getEncoder().encodeToString(b);}}''', + 'Runtime/MMFRuntime.java':'''package Runtime; public class MMFRuntime {public static MMFRuntime inst=new MMFRuntime();public java.io.File getFilesDir(){return new java.io.File(System.getProperty("java.io.tmpdir"));}}''' + } + for name,text in files.items(): + p=root/name;p.parent.mkdir(parents=True,exist_ok=True);p.write_text(text) diff --git a/Extensions/Android_Gradle/tests/fini/run.py b/Extensions/Android_Gradle/tests/fini/run.py new file mode 100644 index 000000000..c037320e9 --- /dev/null +++ b/Extensions/Android_Gradle/tests/fini/run.py @@ -0,0 +1,20 @@ +#!/usr/bin/env python3 +"""Host core/JNI/ACE checks. Requires JDK and JSON-java 20240303 source or JAR. +SDK doubles validate boundary behavior, not Android APK or device integration. +""" +import argparse,os,subprocess +from pathlib import Path +from fixture import create +p=argparse.ArgumentParser(description=__doc__) +p.add_argument('--cmake',default='cmake');p.add_argument('--json',type=Path,required=True,help='JSON-java source root or JAR (test-only)') +p.add_argument('--build',type=Path,default=Path('work/fini-host')) +a=p.parse_args();repo=Path(__file__).resolve().parents[4];tests=Path(__file__).resolve().parent;build=a.build.resolve();java=repo/'Extensions/Android_Gradle/app/src/main/java' +def run(cmd):subprocess.run(list(map(str,cmd)),cwd=repo,check=True) +run([a.cmake,'-S',repo/'Extensions/Android_Gradle/cmake/Fini','-B',build,'-DFINI_BUILD_TESTS=ON']) +run([a.cmake,'--build',build,'--parallel','2']);run([build/'core/fini_default_ini']);run([build/'core/fini_tests']) +fixture=build/'fixture';create(fixture);classes=build/'classes';classes.mkdir(exist_ok=True) +source=list(fixture.rglob('*.java'))+[java/'Expressions/CValue.java',java/'Extensions/CRunFini.java',java/'_DeLib/FiniNative.java',tests/'IntegrationTest.java'] +cp=str(a.json.resolve()) if a.json.is_file() else str(classes) +if a.json.is_dir():source+=list(a.json.resolve().rglob('*.java')) +run(['javac','--release','8','-cp',cp,'-d',classes,*source]) +run(['java','-Xcheck:jni','-Djava.library.path='+str(build),'-cp',str(classes)+os.pathsep+cp,'Extensions.IntegrationTest']) diff --git a/Extensions/Android_Gradle/tests/fini/test_native_tools.py b/Extensions/Android_Gradle/tests/fini/test_native_tools.py new file mode 100644 index 000000000..bb7b3a67f --- /dev/null +++ b/Extensions/Android_Gradle/tests/fini/test_native_tools.py @@ -0,0 +1,61 @@ +"""Exercise discovery/installation decisions without downloading an SDK in tests.""" +import importlib.util +import json +import os +from pathlib import Path +import tempfile +import unittest +from unittest.mock import patch,Mock + +repo=Path(__file__).resolve().parents[4] +spec=importlib.util.spec_from_file_location('fini_tools',repo/'Extensions/_Script/FiniNativeTools.py') +tools=importlib.util.module_from_spec(spec);spec.loader.exec_module(tools) +config=json.loads((repo/'Extensions/Android_Gradle/cmake/Fini/toolchain.json').read_text()) + +def touch(path): + path.parent.mkdir(parents=True,exist_ok=True);path.write_text('fixture');return path + +class NativeToolsTest(unittest.TestCase): + def test_ready_sdk_never_installs_or_downloads(self): + with tempfile.TemporaryDirectory() as temp: + sdk=Path(temp);ndk=sdk/'ndk'/config['ndk'];touch(ndk/'build/cmake/android.toolchain.cmake') + run=Mock() + with patch.object(tools,'find_build_tools',return_value=(sdk/'cmake',sdk/'ninja')),patch.object(tools,'bootstrap_sdkmanager') as bootstrap: + result=tools.prepare(repo,config,run,sdk=sdk) + self.assertEqual(result['ndk'],ndk);run.assert_not_called();bootstrap.assert_not_called() + def test_missing_ndk_uses_sdkmanager_and_rechecks_install(self): + with tempfile.TemporaryDirectory() as temp: + sdk=Path(temp);manager=touch(sdk/'cmdline-tools/latest/bin/sdkmanager') + commands=[] + def run(command): + commands.append(list(map(str,command))) + touch(sdk/'ndk'/config['ndk']/'build/cmake/android.toolchain.cmake') + with patch.object(tools,'find_build_tools',return_value=(sdk/'cmake',sdk/'ninja')),patch.object(tools,'find_sdkmanager',return_value=manager): + result=tools.prepare(repo,config,run,sdk=sdk) + self.assertIn('ndk;'+config['ndk'],commands[0]);self.assertNotIn('cmake;'+config['cmake'],commands[0]);self.assertTrue(result['ndk'].is_dir()) + def test_success_exit_without_actual_install_is_failure(self): + with tempfile.TemporaryDirectory() as temp: + sdk=Path(temp) + with patch.object(tools,'find_build_tools',return_value=(sdk/'cmake',sdk/'ninja')),patch.object(tools,'find_sdkmanager',return_value=sdk/'sdkmanager'): + with self.assertRaisesRegex(RuntimeError,'incomplete'): + tools.prepare(repo,config,lambda _:None,sdk=sdk) + def test_vs_sibling_ninja_is_found(self): + with tempfile.TemporaryDirectory() as temp: + root=Path(temp);cmake=touch(root/'Microsoft/CMake/CMake/bin/cmake');ninja=touch(root/'Microsoft/CMake/Ninja/ninja') + with patch.object(tools,'executable',side_effect=lambda n:n),patch.object(tools,'capture',side_effect=lambda cmd:'cmake version 3.31.0' if str(cmd[0])==str(cmake) else '1.12'): + self.assertEqual(tools.find_build_tools(root,cmake=cmake),(cmake,ninja)) + def test_no_install_reports_exact_missing_packages(self): + with tempfile.TemporaryDirectory() as temp: + with patch.object(tools,'find_build_tools',return_value=None): + with self.assertRaisesRegex(RuntimeError,'ndk;'+config['ndk']): + tools.prepare(repo,config,Mock(),sdk=Path(temp),install=False) + def test_selects_pinned_commandline_tools_for_host(self): + xml='linuxcommandlinetools-linux-13114758_latest.zipabc' + self.assertEqual(tools.command_line_archive(xml,'19.0','linux'),('commandlinetools-linux-13114758_latest.zip','sha1','abc')) + with self.assertRaises(RuntimeError):tools.command_line_archive(xml,'19.0','macosx') + def test_explicit_invalid_ndk_is_not_silently_replaced(self): + with tempfile.TemporaryDirectory() as temp: + with self.assertRaisesRegex(RuntimeError,'Explicit Android NDK'): + tools.prepare(repo,config,Mock(),sdk=Path(temp),ndk=Path(temp)/'typo') + +if __name__=='__main__':unittest.main() diff --git a/Extensions/Android_Gradle/tests/fini/test_publisher.py b/Extensions/Android_Gradle/tests/fini/test_publisher.py new file mode 100644 index 000000000..dbfcb6fd1 --- /dev/null +++ b/Extensions/Android_Gradle/tests/fini/test_publisher.py @@ -0,0 +1,33 @@ +"""Synthetic ELF fixtures test validation/ownership, never serve as release binaries.""" +import importlib.util,struct,tempfile,unittest,zipfile,io +from pathlib import Path +from unittest.mock import patch +repo=Path(__file__).resolve().parents[4] +spec=importlib.util.spec_from_file_location('fini_publish',repo/'Extensions/_Script/Fini.py');publisher=importlib.util.module_from_spec(spec);spec.loader.exec_module(publisher) +def elf(machine=183,align=16384): + data=bytearray(120);data[:6]=b'\x7fELF\x02\x01';struct.pack_into(' + + true + + + + \ No newline at end of file diff --git a/Extensions/Fini/Func.h b/Extensions/Fini/Func.h index 014a56e56..53337b386 100644 --- a/Extensions/Fini/Func.h +++ b/Extensions/Fini/Func.h @@ -1,4 +1,5 @@ #pragma once +#include "IniCore.h" // ------------------------ // forward declaration @@ -21,8 +22,7 @@ inline void ReleaseIni(LPRDATA rdPtr); inline void InitIni(LPRDATA rdPtr) { ReleaseIni(rdPtr); - rdPtr->ini = new INI; - rdPtr->ini->SetUnicode(); + rdPtr->ini = FiniIni::Create(); } inline void ReleaseIni(LPRDATA rdPtr) { diff --git a/Extensions/Fini/IniCore.h b/Extensions/Fini/IniCore.h new file mode 100644 index 000000000..3cbbad663 --- /dev/null +++ b/Extensions/Fini/IniCore.h @@ -0,0 +1,172 @@ +#pragma once +#if defined(FINI_PORTABLE_RUNTIME) +#define SI_NO_CONVERSION +#endif +#include +#include "../_3rdLib/simpleini/SimpleIni.h" + +// Runtime-independent operations extracted from Fini/Main.cpp and Func.h. +// Windows uses wide-character SimpleIni; mobile uses its UTF-8 interface. +namespace FiniIni { +template Ini* Create() {auto* ini=new Ini;ini->SetUnicode();return ini;} +template std::string Save(const Ini* ini) {std::string out;if(ini)ini->Save(out);return out;} +template bool Set(Ini* ini,const Char* section,const Char* item,const Char* value) { + if(!ini||!section||!*section||!item||!*item)return false; + const auto result=ini->SetValue(section,item,value); + return result==SI_UPDATED||result==SI_INSERTED; +} +template bool Delete(Ini* ini,const Char* section,const Char* item) { + return ini&§ion&&*section&&ini->Delete(section,item,true); +} +template bool Copy(Ini* ini,const Char* source,const Char* destination,bool removeSource) { + if(!ini||!source||!*source||!destination||!*destination)return false; + typename Ini::TNamesDepend names;ini->GetAllKeys(source,names); + for(const auto& name:names)ini->SetValue(destination,name.pItem,ini->GetValue(source,name.pItem)); + if(removeSource)ini->Delete(source,nullptr,true); + return !names.empty(); +} +} + +#if defined(FINI_PORTABLE_RUNTIME) +// Computational core only; Fusion objects and event dispatch stay in each runtime. +#include "../_3rdLib/nlohmann/single_include/nlohmann/json.hpp" +#include "../_DeLib/Encryption/CAPI.h" +#include +#include +#include +#include +#include +#include +#include +#include "../_DeLib/UnicodeConversion.h" +#include "../_3rdLib/Base64.h" +#include "../_3rdLib/Compress/zlibInterface.h" +#include +#include +#include "../_DeLib/StrNum.h" +#include +#include + +namespace fini { +using Json = nlohmann::json; +inline void check(bool ok, const char* message) { if (!ok) throw std::runtime_error(message); } +// Reuse the Windows extension's C++ codecs, not platform copies. +inline std::string base64(const std::string& data) { + check(data.size()<=UINT_MAX,"Base64 input too large"); + return Base64{}.base64_encode(reinterpret_cast(data.data()),data.size()); +} +inline std::string unbase64(const std::string& text) { + auto bytes=Base64{}.base64_decode(text); + return std::string(bytes.begin(),bytes.end()); +} +inline std::string compress(const std::string& data) { + std::string out;check(ZLIBI_Compress(data,out),"Compress failed");return out; +} +inline std::string decompress(const std::string& data) { + std::string out;check(ZLIBI_UnCompress(data.data(),data.size(),out,4,64*1024*1024),"Invalid or oversized zlib data");return out; +} +inline std::string crypt(const std::string& data,const std::string& password,bool encrypt) { + // Call the common runtime C ABI; no EVP/core cipher implementation in Fini. + const auto units=to_utf16_string(password); + std::vector encoded; + for(auto unit:units){encoded.push_back(static_cast(unit));encoded.push_back(static_cast(unit>>8));} + std::array key{},iv{}; + check(Encryption_DeriveKeyAndIv(encoded.data(),encoded.size(),key.data(),iv.data())==ENCRYPTION_OK,"Derive encryption material"); + check(data.size()<=(std::numeric_limits::max)()-16,"Encryption input too large"); + std::string result(data.size()+16,'\0');size_t written=0; + auto transform=encrypt?Encryption_EncryptBuffer:Encryption_DecryptBuffer; + check(transform(key.data(),iv.data(),reinterpret_cast(data.data()),data.size(),reinterpret_cast(result.data()),result.size(),&written)==ENCRYPTION_OK,"Shared encryption operation failed"); + result.resize(written);return result; +} +inline std::string read(const std::string& path,const std::string& key) { + std::ifstream in(std::filesystem::path(std::u8string(reinterpret_cast(path.data()),path.size())),std::ios::binary); check(bool(in),"Open INI file"); + std::string data((std::istreambuf_iterator(in)),{}); check(!in.bad(),"Read INI file"); + return key.empty()?data:crypt(data,key,false); +} +inline void write(const std::string& path,const std::string& key,const std::string& data) { + check(!path.empty(),"Empty INI path"); auto bytes=key.empty()?data:crypt(data,key,true); + std::ofstream out(std::filesystem::path(std::u8string(reinterpret_cast(path.data()),path.size())),std::ios::binary|std::ios::trunc); check(bool(out),"Open INI output"); + out.write(bytes.data(),bytes.size()); out.close(); check(bool(out),"Write INI output"); +} +inline void put32(std::string& out,uint32_t n) { for(int i=0;i<4;++i) out+=static_cast(n>>(8*i)); } +inline uint32_t get32(const std::string& in,size_t& p) { check(p+4<=in.size(),"Truncated object data"); uint32_t n=0; for(int i=0;i<4;++i) n|=uint32_t(static_cast(in[p++]))<<(8*i); return n; } +inline std::string packValues(const Json& data) { + std::string out; put32(out,data.at("flags").get()); + for(const auto& v:data.at("values")) { + int type=v.at("type"); check(type==0||type==2,"Invalid numeric value type"); put32(out,type); put32(out,0); + if(type==0) { put32(out,v.at("value").get()); put32(out,0); } + else { double d=v.at("value"); uint64_t bits; std::memcpy(&bits,&d,8); put32(out,bits); put32(out,bits>>32); } + } + for(const auto& s:data.at("strings")) { + auto units=to_utf16_string(s.get()); + for(auto c:units) { out+=static_cast(c); out+=static_cast(c>>8); } out.append(2,'\0'); + } + return base64(out); +} +inline Json unpackValues(const std::string& text,size_t values,size_t strings) { + auto in=unbase64(text); size_t p=0; Json result={{"flags",get32(in,p)},{"values",Json::array()},{"strings",Json::array()}}; + check(values<=4096 && strings<=4096,"Too many alterable values"); + for(size_t i=0;i(low)}}); + else { uint64_t bits=uint64_t(low)|(uint64_t(high)<<32); double d; std::memcpy(&d,&bits,8); check(std::isfinite(d),"Nonfinite value"); result["values"].push_back({{"type",2},{"value",d}}); } + } + for(size_t i=0;i(in[p])|(static_cast(in[p+1])<<8); p+=2; if(!c) break; units+=static_cast(c); } + result["strings"].push_back(std::wstring_convert,char16_t>{}.to_bytes(units)); + } + check(p==in.size(),"Object data count mismatch"); return result; +} +class State { + std::unique_ptr ini_,localization_; +public: + bool autoSave=true,modified=false; + std::string path,key,language; + static std::unique_ptr makeIni() { return std::unique_ptr(FiniIni::Create()); } + bool initialized() const { return bool(ini_); } + void reset() { ini_=makeIni(); modified=false; } + void release() { ini_.reset(); modified=false; } + std::string save() const { return FiniIni::Save(ini_.get()); } + void flush() { if(ini_&&autoSave&&modified&&!path.empty()) {write(path,key,save());modified=false;} } + void load(const std::string& data) { auto next=makeIni(); check(next->LoadData(data.data(),data.size())>=0,"Parse INI"); flush(); ini_=std::move(next); modified=false; } + void loadFile(const std::string& file,const std::string& password) { flush(); auto bytes=read(file,password); load(bytes); if(autoSave){path=file;key=password;} } + void loadLocalization(const std::string& file,const std::string& password) { localization_=makeIni(); if(!file.empty()) {auto bytes=read(file,password); check(localization_->LoadData(bytes.data(),bytes.size())>=0,"Parse localization");} } + void localizeFrom(const std::string& data) { localization_=makeIni(); localization_->LoadData(data.data(),data.size()); } + std::string raw(const std::string& section,const std::string& item) const { return ini_&&!section.empty()?ini_->GetValue(section.c_str(),item.c_str(),""):""; } + std::string get(const std::string& section,const std::string& item) const { auto value=raw(section,item); if(localization_&&!language.empty()) {std::string translated=localization_->GetValue(language.c_str(),value.c_str(),""); if(!translated.empty()) return translated;} return value; } + Json names(const std::string* section=nullptr) const { Json result=Json::array(); if(!ini_)return result; CSimpleIniA::TNamesDepend names; if(section)ini_->GetAllKeys(section->c_str(),names);else ini_->GetAllSections(names); for(const auto& n:names)result.push_back(n.pItem); return result; } + Json request(const std::string& op,const Json& a=Json::array()) { + auto s=[&](size_t i){return a.at(i).get();}; + if(op=="initialized")return initialized(); + if(op=="fileData"||op=="localBytes"){auto data=unbase64(s(0));auto password=s(1);if(!password.empty())data=crypt(data,password,false);if(op=="fileData"){load(data);if(autoSave){path=s(2);key=password;}}else localizeFrom(data);return nullptr;} + if(op=="new") { reset(); return nullptr; } if(op=="release") { release(); return nullptr; } + if(op=="save")return save(); if(op=="load"){load(s(0));return nullptr;} + if(op=="fileLoad"){loadFile(s(0),s(1));return nullptr;} + if(op=="fileSave"){if(ini_)write(s(0),s(1),save());return nullptr;} + if(op=="flush"){flush();return nullptr;} + if(op=="autoOn"){autoSave=true;path=s(0);key=s(1);return nullptr;} + if(op=="autoOff"){autoSave=false;return nullptr;} + if(op=="setNumber")return request("set",{s(0),s(1),std::format("{}",a.at(2).get())}); + if(op=="set") {if(!ini_)reset();auto sec=s(0),item=s(1),val=s(2);if(!sec.empty()&&!item.empty())modified=FiniIni::Set(ini_.get(),sec.c_str(),item.c_str(),val.c_str())||modified;return nullptr;} + if(op=="delete") {auto sec=s(0),item=s(1);if(ini_&&!sec.empty())modified=FiniIni::Delete(ini_.get(),sec.c_str(),item.empty()?nullptr:item.c_str())||modified;return nullptr;} + if(op=="copy") {auto src=s(0),dst=s(1);if(ini_&&!src.empty()&&!dst.empty()) {modified=FiniIni::Copy(ini_.get(),src.c_str(),dst.c_str(),a.at(2).get())||modified;}return nullptr;} + if(op=="get")return get(s(0),s(1)); + if(op=="number") {auto v=_stof(raw(s(0),s(1)));return std::isfinite(v)?Json(v):Json(0);} + if(op=="hasSection") {auto sec=s(0);return ini_&&!sec.empty()&&ini_->SectionExists(sec.c_str());} + if(op=="hasItem") {auto sec=s(0),item=s(1);return ini_&&!sec.empty()&&!item.empty()&&ini_->KeyExists(sec.c_str(),item.c_str());} + if(op=="sections")return names(); if(op=="items"){auto sec=s(0);return names(&sec);} + if(op=="localFile"){loadLocalization(s(0),s(1));return nullptr;} + if(op=="localData"){localizeFrom(s(0));return nullptr;} if(op=="language"){language=s(0);return nullptr;} + if(op=="encode")return base64(a.at(1).get()?compress(s(0)):s(0)); + if(op=="decode"){auto data=unbase64(s(0));return a.at(1).get()?decompress(data):data;} + if(op=="packValues")return packValues(a.at(0)); + if(op=="unpackValues")return unpackValues(s(0),a.at(1),a.at(2)); + if(op=="packPosition"){std::string bytes;put32(bytes,a.at(0).get());put32(bytes,a.at(1).get());return base64(bytes);} + if(op=="unpackPosition"){auto bytes=unbase64(s(0));check(bytes.size()==8,"Invalid position data");size_t p=0;auto x=static_cast(get32(bytes,p)),y=static_cast(get32(bytes,p));return Json::array({x,y});} + throw std::runtime_error("Unknown Fini operation"); + } +}; +} // namespace fini + +#endif // FINI_PORTABLE_RUNTIME diff --git a/Extensions/Fini/Main.cpp b/Extensions/Fini/Main.cpp index fb6eb22f5..a2a0859fb 100644 --- a/Extensions/Fini/Main.cpp +++ b/Extensions/Fini/Main.cpp @@ -248,8 +248,8 @@ short WINAPI DLLExport Action_SetSecItem_Value(LPRDATA rdPtr, long param1, long if (rdPtr->ini == nullptr) { InitIni(rdPtr); } if (StrEmpty(Section) || StrEmpty(Item)) { return 0; } - rdPtr->Modified = Modified(rdPtr->ini->SetValue(Section, Item, - std::format(L"{}", Value).c_str())) + rdPtr->Modified = FiniIni::Set(rdPtr->ini, Section, Item, + std::format(L"{}", Value).c_str()) || rdPtr->Modified; return 0; @@ -263,7 +263,7 @@ short WINAPI DLLExport Action_SetSecItem_String(LPRDATA rdPtr, long param1, long if (rdPtr->ini == nullptr) { InitIni(rdPtr); } if (StrEmpty(Section) || StrEmpty(Item)) { return 0; } - rdPtr->Modified = Modified(rdPtr->ini->SetValue(Section, Item, String)) + rdPtr->Modified = FiniIni::Set(rdPtr->ini, Section, Item, String) || rdPtr->Modified; return 0; @@ -281,19 +281,7 @@ short WINAPI DLLExport Action_CopySection(LPRDATA rdPtr, long param1, long param return 0; } - INILIST items; - rdPtr->ini->GetAllKeys(Src, items); - - rdPtr->Modified = !items.empty() || rdPtr->Modified; - - for (const auto & it: items) { - //need not to check if the value is string or number to set default value, cause src is valid - rdPtr->ini->SetValue(Des, it.pItem, rdPtr->ini->GetValue(Src, it.pItem)); - } - - if (DeleteSrc) { - rdPtr->ini->Delete(Src, nullptr, true); - } + rdPtr->Modified = FiniIni::Copy(rdPtr->ini, Src, Des, DeleteSrc) || rdPtr->Modified; return 0; } @@ -307,7 +295,7 @@ short WINAPI DLLExport Action_DeleteSecItem(LPRDATA rdPtr, long param1, long par if (StrEqu(Item, Empty_Str)) { Item = nullptr; } - rdPtr->Modified = rdPtr->ini->Delete(Section, Item, true) + rdPtr->Modified = FiniIni::Delete(rdPtr->ini, Section, Item) || rdPtr->Modified; return 0; diff --git a/Extensions/_3rdLib/Compress/zlibInterface.h b/Extensions/_3rdLib/Compress/zlibInterface.h index cdc1ec095..823b8b61a 100644 --- a/Extensions/_3rdLib/Compress/zlibInterface.h +++ b/Extensions/_3rdLib/Compress/zlibInterface.h @@ -1,14 +1,19 @@ #pragma once #include "zlib/zlib.h" -#include "GeneralDefinition.h" // for macro RELATIVE_PATH +#include "PathUtilities.h" // only dependency: RELATIVE_PATH +#include +#include +#ifdef _MSC_VER #ifdef _WIN64 #pragma comment(lib, RELATIVE_PATH("zlibstatic_x64.lib")) #else #pragma comment(lib, RELATIVE_PATH("zlibstatic.lib")) #endif +#endif + #include inline bool ZLIBI_Compress(const char* pSrc, const size_t srcSz, @@ -54,8 +59,14 @@ inline std::string ZLIBI_Compress(const std::string& in, // 0 -> decompress buffer is 8 times larger than input // other -> decompress buffer is input size * estimateRatio inline bool ZLIBI_UnCompress(const char* pSrc, const size_t srcSz, - std::string& out, const unsigned char estimateRatio = 4) { - size_t blockSz = !estimateRatio ? (srcSz << 3) : srcSz * estimateRatio; + std::string& out, const unsigned char estimateRatio = 4, + const size_t maxOutputSize = (std::numeric_limits::max)()) { + if (!pSrc || !srcSz || !maxOutputSize || srcSz > (std::numeric_limits::max)()) { + out.clear(); return false; + } + const size_t ratio = !estimateRatio ? 8 : estimateRatio; + size_t blockSz = srcSz > maxOutputSize / ratio ? maxOutputSize : srcSz * ratio; + blockSz = (std::max)(size_t{1}, (std::min)(blockSz, maxOutputSize)); while (true) { out.clear(); @@ -68,7 +79,10 @@ inline bool ZLIBI_UnCompress(const char* pSrc, const size_t srcSz, static_cast(srcSz)); if (ret == Z_BUF_ERROR) { - blockSz = blockSz << 1; + if (blockSz >= maxOutputSize || blockSz > (std::numeric_limits::max)() / 2) { + out.clear(); return false; + } + blockSz = blockSz > maxOutputSize / 2 ? maxOutputSize : blockSz * 2; continue; } diff --git a/Extensions/_DeLib/Encryption/CAPI.h b/Extensions/_DeLib/Encryption/CAPI.h index 7e5a364b3..11dfcd658 100644 --- a/Extensions/_DeLib/Encryption/CAPI.h +++ b/Extensions/_DeLib/Encryption/CAPI.h @@ -11,6 +11,8 @@ # else # define ENCRYPTION_API __declspec(dllimport) # endif +#elif defined(ENCRYPTION_SHARED) && (defined(__GNUC__) || defined(__clang__)) +# define ENCRYPTION_API __attribute__((visibility("default"))) #else # define ENCRYPTION_API #endif diff --git a/Extensions/_DeLib/StrNum.h b/Extensions/_DeLib/StrNum.h index f570ea7fd..3b91bc664 100644 --- a/Extensions/_DeLib/StrNum.h +++ b/Extensions/_DeLib/StrNum.h @@ -7,7 +7,7 @@ #include #include -#include "GeneralDefinition.h" +#include "StringUtilities.h" #include "StringTraits.h" #include "ArithmeticTraits.h" diff --git a/Extensions/_Script/Encryption/README.md b/Extensions/_Script/Encryption/README.md new file mode 100644 index 000000000..0f5fcce37 --- /dev/null +++ b/Extensions/_Script/Encryption/README.md @@ -0,0 +1,19 @@ +# Shared mobile encryption runtime + +The existing `_DeLib/Encryption/CAPI.cpp`, Core.h and KeyGen.h implement the +algorithms. This directory contains shared dependency-preparation tooling; +Android integration belongs to Android_Gradle and Apple integration to iOS. +Clients use CAPI.h; only this runtime links libcrypto. C ABI version is 1. + +## Dependency preparation + +Source preparation is shared too: prepare_openssl.py pins OpenSSL 3.6.5 and its +SHA512 to the repository's vcpkg revision +9a5dfef5d1a0438a7d24856a85bd60f309d355e9. It prepares target dependencies, never a +second encryption algorithm. Automatic source builds need Linux/macOS, Perl and +make. Windows publishers can use prebuilt target OpenSSL prefixes via --openssl. + +Platform build/package integration lives in its own project: + +- Android: `Extensions/Android_Gradle/cmake/Encryption/README.md`. +- iOS: `Extensions/iOS/Encryption/README.md`. diff --git a/Extensions/_Script/Encryption/prepare_openssl.py b/Extensions/_Script/Encryption/prepare_openssl.py new file mode 100644 index 000000000..6fbd38ac0 --- /dev/null +++ b/Extensions/_Script/Encryption/prepare_openssl.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +"""Prepare target OpenSSL for the existing encryption core, never a new cipher. +Source version/hash match the repository's pinned vcpkg OpenSSL port. +""" +import argparse +import hashlib +import json +import os +from pathlib import Path +import platform +import shutil +import subprocess +import tarfile +import urllib.request + +VERSION='3.6.5' +SOURCE_URL='https://github.com/openssl/openssl/archive/refs/tags/openssl-'+VERSION+'.tar.gz' +SOURCE_SHA512='40f0b346a9a5cf4cb37f1ecebdc74750c39a95783710061bfac58964883b0831543ecee0aee9f747ea093781b13c58a543b0b935d222519e3cb19648d77a0276' +ANDROID={'armeabi-v7a':('android-arm','armv7a-linux-androideabi'),'arm64-v8a':('android-arm64','aarch64-linux-android'),'x86':('android-x86','i686-linux-android'),'x86_64':('android-x86_64','x86_64-linux-android')} +IOS={'iphoneos-arm64':('iphoneos','arm64','ios64-xcrun'),'iphonesimulator-arm64':('iphonesimulator','arm64','iossimulator-arm64-xcrun'),'iphonesimulator-x86_64':('iphonesimulator','x86_64','iossimulator-x86_64-xcrun')} + +def source_tree(work,archive=None): + work.mkdir(parents=True,exist_ok=True) + archive=Path(archive) if archive else work/f'openssl-{VERSION}.tar.gz' + if not archive.is_file(): + temporary=archive.with_suffix('.download') + try: + with urllib.request.urlopen(SOURCE_URL,timeout=30) as src,temporary.open('wb') as out: + shutil.copyfileobj(src,out) + if hashlib.sha512(temporary.read_bytes()).hexdigest()!=SOURCE_SHA512: + raise RuntimeError('Downloaded OpenSSL source SHA512 mismatch') + temporary.replace(archive) + finally: + temporary.unlink(missing_ok=True) + if hashlib.sha512(archive.read_bytes()).hexdigest()!=SOURCE_SHA512: + raise RuntimeError('OpenSSL source SHA512 mismatch: '+str(archive)) + source=work/f'openssl-{VERSION}' + marker=source/'.fini-source-sha512' + if not marker.is_file() or marker.read_text()!=SOURCE_SHA512: + if source.exists():shutil.rmtree(source) + # Python's data filter rejects escaping links and special files. + with tarfile.open(archive) as tar: + names={member.name.split('/')[0] for member in tar.getmembers()} + if len(names)!=1:raise RuntimeError('Unexpected OpenSSL source archive layout') + top=work/next(iter(names)) + for member in tar.getmembers(): + if Path(member.name).is_absolute() or '..' in Path(member.name).parts: + raise RuntimeError('Unsafe OpenSSL archive path') + if not member.isdir() and not member.isfile(): + raise RuntimeError('Unexpected link/special file in OpenSSL source') + if top!=source and top.exists():shutil.rmtree(top) + tar.extractall(work) + if top!=source:top.rename(source) + marker.write_text(SOURCE_SHA512) + return source + + +def prepare(kind,output,work,ndk=None,abis=None,api=26,archive=None): + if os.name=='nt': + raise RuntimeError('For native Windows publishing supply prebuilt ABI OpenSSL via --openssl; automatic source preparation currently requires a Unix build host (Linux/macOS).') + if not shutil.which('perl') or not shutil.which('make'): + raise RuntimeError('OpenSSL source build requires Perl and GNU make (Apple make is supported).') + output=Path(output).resolve();work=Path(work).resolve();source=source_tree(work,archive) + output.mkdir(parents=True,exist_ok=True);shutil.copy2(source/'LICENSE.txt',output/'LICENSE.txt') + selected=list(abis or ANDROID) if kind=='android' else list(IOS) + with (work/'openssl-build.log').open('a') as log: + def run(cmd,cwd,environment): + command=list(map(str,cmd));print('+',' '.join(command),flush=True) + log.write('+ '+' '.join(command)+'\n');log.flush() + subprocess.run(command,cwd=cwd,env=environment,stdout=log,stderr=subprocess.STDOUT,check=True) + for name in selected: + prefix=output/name;environment=os.environ.copy();options=['no-shared','no-module','no-tests','no-apps','no-docs','-fPIC','--libdir=lib','--prefix='+str(prefix)] + if kind=='android': + if ndk is None:raise RuntimeError('Android OpenSSL requires --ndk') + ndk=Path(ndk).resolve();host='darwin-x86_64' if platform.system()=='Darwin' else 'linux-x86_64' + tools=ndk/'toolchains/llvm/prebuilt'/host/'bin';target,triple=ANDROID[name] + environment.update({'ANDROID_NDK_ROOT':str(ndk),'PATH':str(tools)+os.pathsep+environment['PATH'],'CC':str(tools/(triple+str(api)+'-clang')),'AR':str(tools/'llvm-ar'),'RANLIB':str(tools/'llvm-ranlib')}) + options+=['-D__ANDROID_API__='+str(api)] + identity=str(ndk)+'|'+str(api) + else: + if platform.system()!='Darwin':raise RuntimeError('iOS OpenSSL preparation requires macOS/Xcode') + sdk,arch,target=IOS[name] + sysroot=subprocess.check_output(['xcrun','--sdk',sdk,'--show-sdk-path'],text=True).strip() + compiler=subprocess.check_output(['xcrun','--sdk',sdk,'--find','clang'],text=True).strip() + environment.update({'CC':compiler,'CROSS_TOP':str(Path(sysroot).parent.parent),'CROSS_SDK':Path(sysroot).name,'SDKROOT':sysroot}) + options+=['-isysroot',sysroot,'-arch',arch,'-mios-version-min=17.0' if sdk=='iphoneos' else '-mios-simulator-version-min=17.0'] + identity=sysroot+'|17.0' + signature=json.dumps({'source_sha512':SOURCE_SHA512,'target':target,'options':options,'toolchain':identity},sort_keys=True) + marker=prefix/'fini-openssl.json' + if marker.is_file() and marker.read_text()==signature and (prefix/'lib/libcrypto.a').is_file() and (prefix/'include/openssl/evp.h').is_file(): + print('OpenSSL up to date:',name,flush=True);continue + build=work/(kind+'-'+name);build.mkdir(parents=True,exist_ok=True) + run(['perl',source/'Configure',target,*options],build,environment) + run(['make','-j'+str(min(os.cpu_count() or 2,8))],build,environment) + run(['make','install_dev'],build,environment) + marker.write_text(signature) + return output + + +def main(): + p=argparse.ArgumentParser(description=__doc__) + p.add_argument('--platform',choices=['android','ios'],required=True);p.add_argument('--ndk',type=Path) + p.add_argument('--abis',nargs='+',choices=ANDROID);p.add_argument('--api',type=int,default=26) + p.add_argument('--output',type=Path,required=True);p.add_argument('--work',type=Path,required=True);p.add_argument('--source-archive',type=Path) + a=p.parse_args();prepare(a.platform,a.output,a.work,a.ndk,a.abis,a.api,a.source_archive) +if __name__=='__main__':main() diff --git a/Extensions/_Script/Fini.bat b/Extensions/_Script/Fini.bat index 7a70070fd..0938c47cb 100644 --- a/Extensions/_Script/Fini.bat +++ b/Extensions/_Script/Fini.bat @@ -1,99 +1,9 @@ -Set ProjectName=Fini -Set CompressPath="C:\Program Files\7-Zip\7z.exe" - -Set ParentPath=%~dp0\..\ -Set ToInstallPath=%~dp0\..\%ProjectName%\ToInstall\Files\ - -@REM @echo Android... - -@REM Set AndroidPath=%~dp0\..\Android_Gradle - -@REM Set SrcPath=%AndroidPath%\app - -@REM Set ContentPath=%AndroidPath%\_Release - -@REM Set ReleasePath=%ToInstallPath%\Data\Runtime\Android - -@REM Set FusionPath=C:\Steam\steamapps\common\Clickteam Fusion 2.5\Data\Runtime\Android - -@REM @echo Delete... - -@REM rem Clean Content Path -@REM for /r "%ContentPath%" %%F in (*.*) do ( -@REM del "%%F" /f /s /q -@REM ) - -@REM rmdir /s /q "%ContentPath%\build" -@REM rmdir /s /q "%ContentPath%\app" -@REM rmdir /s /q "%ContentPath%\libs" -@REM rmdir /s /q "%ContentPath%\src" - -@REM @echo Copy... - -@REM @REM Add content here -@REM echo f | xcopy "%SrcPath%\src\main\java\Extensions\CRun%ProjectName%.java" "%ContentPath%\app\src\main\java\Extensions\CRun%ProjectName%.java" /s /e - -@REM @REM Use these commands if extra lib & build script are needed - -@REM @REM xcopy "%SrcPath%\libs\*.*" "%ContentPath%\app\libs\" /s /e - -@REM @REM copy "%AndroidPath%\release.gradle" "%ContentPath%\app\build.gradle" -@REM @REM copy "%AndroidPath%\ReleaseSettings.gradle" "%ContentPath%\settings.gradle" - -@REM @REM del "%ContentPath%\app\libs\gson-2.8.6.jar" /f /s /q - -@REM @echo Compress... - -@REM del %AndroidPath%\%ProjectName%.zip /f /s /q - -@REM %CompressPath% a -tzip %AndroidPath%\%ProjectName%.zip %ContentPath%\* - -@REM @echo Install... - -@REM del "%FusionPath%\%ProjectName%.zip" /f /s /q -@REM copy "%AndroidPath%\%ProjectName%.zip" "%FusionPath%" - -@REM del "%ReleasePath%\%ProjectName%.zip" /f /s /q -@REM echo f | xcopy "%AndroidPath%\%ProjectName%.zip" "%ReleasePath%\%ProjectName%.zip" /s /e - -@REM @echo Remove Temp... - -@REM del %AndroidPath%\%ProjectName%.zip /f /s /q - -@REM @echo HTML5 - -@REM Set HTML5Path=%~dp0\..\HTML5 - -@REM Set ReleasePath=%ToInstallPath%\Data\Runtime\Html5 - -@REM Set FusionPath=C:\Steam\steamapps\common\Clickteam Fusion 2.5\Data\Runtime\Html5 - -@REM del "%ReleasePath%\%ProjectName%.js" /f /s /q -@REM echo f | xcopy "%HTML5Path%\%ProjectName%.js" "%ReleasePath%\%ProjectName%.js" /s /e - -@REM @echo Install... - -@REM del "%FusionPath%\%ProjectName%.js" /f /s /q -@REM copy "%HTML5Path%\%ProjectName%.js" "%FusionPath%" - -@echo Make Release Package - -set Date=%date:~2,8% - -set Date=%Date:/=% - -%CompressPath% a -tzip %ParentPath%\%ProjectName%_B%Date%.zip %ToInstallPath%\* -%CompressPath% d %ParentPath%\%ProjectName%_B%Date%.zip *.exe -r -%CompressPath% d %ParentPath%\%ProjectName%_B%Date%.zip *.001 -r -%CompressPath% d %ParentPath%\%ProjectName%_B%Date%.zip *.keep -r -%CompressPath% d %ParentPath%\%ProjectName%_B%Date%.zip *.DS_Store -r - -copy "%ParentPath%\%ProjectName%_B%Date%.zip" "%ParentPath%\%ProjectName%_Release.zip" - -@echo Stop JDK... - -Powershell.exe -executionpolicy remotesigned -File "F:\DEV\OpenFusionExamples\Tools\JDK Terminator\JDK_Terminator.ps1" - -@REM @echo Wait... - -@REM pause \ No newline at end of file +@echo off +setlocal +where py >nul 2>nul +if errorlevel 1 ( + python "%~dp0Fini.py" %* +) else ( + py -3 "%~dp0Fini.py" %* +) +exit /b %errorlevel% diff --git a/Extensions/_Script/Fini.py b/Extensions/_Script/Fini.py new file mode 100644 index 000000000..bc5e81d1f --- /dev/null +++ b/Extensions/_Script/Fini.py @@ -0,0 +1,171 @@ +#!/usr/bin/env python3 +"""Single-entry Fini Android publisher. Fusion exports use default Gradle templates.""" +import argparse,hashlib,io,json,os,platform,shutil,struct,subprocess,tempfile,zipfile,sys,traceback +from pathlib import Path +REPO=Path(__file__).resolve().parents[2] +EXT=REPO/'Extensions';JAVA=EXT/'Android_Gradle/app/src/main/java' +sys.path.insert(0,str(Path(__file__).resolve().parent)) +import FiniNativeTools +TOOLCHAIN=json.loads((EXT/'Android_Gradle/cmake/Fini/toolchain.json').read_text()) +sys.path.insert(0,str(EXT/'Android_Gradle/cmake/Encryption')) +sys.path.insert(0,str(EXT/'iOS/Tools')) +import android_runtime as encryption_runtime +import apple_artifacts +import FiniIOS +ABIS=encryption_runtime.ABIS +validate_elf=encryption_runtime.validate_elf +OWNED_JAVA=['Extensions/CRunFini.java','_DeLib/FiniNative.java'] +RULES='-keep class Extensions.CRunFini { *; }\n-keep class _DeLib.FiniNative { *; }\n' + + +def source_files(): + changed=['Fini/IniCore.h','Fini/Func.h','Fini/Main.cpp','Android_Gradle/app/src/main/cpp/Fini/CMakeLists.txt','Android_Gradle/Fini-JNI.md','Android_Gradle/tests/fini/ACE.csv','Android_Gradle/cmake/Fini/toolchain.json','iOS/Fini/build.py','_Script/Encryption/prepare_openssl.py','Android_Gradle/app/src/main/cpp/Encryption/CMakeLists.txt','iOS/Encryption/build.py','Android_Gradle/cmake/Encryption/android_runtime.py','_Script/Encryption/README.md','Android_Gradle/cmake/Encryption/README.md','iOS/Encryption/README.md','iOS/Tools/apple_artifacts.py', + '_3rdLib/Base64.h','_3rdLib/Compress/zlibInterface.h','_3rdLib/Compress/zlib/zlib.h','_3rdLib/Compress/zlib/zconf.h','_3rdLib/simpleini/SimpleIni.h','_3rdLib/nlohmann/single_include/nlohmann/json.hpp', + '_DeLib/PathUtilities.h','_DeLib/GeneralMacros.h','_DeLib/StringTraits.h','_DeLib/StrNum.h','_DeLib/StringUtilities.h','_DeLib/ArithmeticTraits.h','_DeLib/UnicodeConversion.h','_DeLib/Encryption/Core.h','_DeLib/Encryption/KeyGen.h','_DeLib/Encryption/CAPI.h','_DeLib/Encryption/CAPI.cpp', + 'Android_Gradle/cmake/Fini/CMakeLists.txt','Android_Gradle/app/src/main/cpp/Fini/FiniJNI.cpp', + 'iOS/Fini/Runtime/CRunFini.h','iOS/Fini/Runtime/CRunFini.mm','_Script/Fini.py','_Script/Fini.bat','_Script/FiniNativeTools.py','_Script/FiniIOS.py','iOS/Tools/make_ext.py','iOS/Tools/xcode_project.py'] + return [EXT/p for p in changed]+[JAVA/p for p in OWNED_JAVA]+[p for p in (EXT/'Android_Gradle/tests/fini').glob('*') if p.is_file() and p.suffix in ('.py','.cpp','.java')]+[p for p in (EXT/'iOS/Fini').rglob('*') if p.is_file() and 'xcuserdata' not in p.parts and '__pycache__' not in p.parts and p.name not in ('Local.xcconfig','.gitignore') and not any(name in p.parts for name in ('Prebuilt','Runtime','Dependencies','SDK')) and p.suffix in ('.py','.md','.xcconfig','.pbxproj','.xcscheme','.xcworkspacedata')] + +def package(destination,abi_dirs,include_sources=True,licenses=None): + payload={f'app/src/main/java/{p}':(JAVA/p).read_bytes() for p in OWNED_JAVA} + for abi,directory in abi_dirs.items(): + for name in ('libfini.so',): + path=directory/name;validate_elf(path,abi);payload[f'app/src/main/jniLibs/{abi}/{name}']=path.read_bytes() + rules=io.BytesIO() + with zipfile.ZipFile(rules,'w',zipfile.ZIP_DEFLATED) as z:z.writestr('META-INF/proguard/Fini.pro',RULES) + payload['app/libs/Fini-JNI-rules.jar']=rules.getvalue() + if include_sources: + for path in source_files(): + # Runtime packages carry only the shared encryption API, not its implementation. + if abi_dirs and (EXT/'_DeLib/Encryption') in path.parents and path.name!='CAPI.h':continue + payload['source/'+str(path.relative_to(REPO))]=path.read_bytes() + # Owned code and third-party license material only, never the exporter SDK. + for relative in ('LICENSE','LICENSE.md','LICENSE.txt'): + if (REPO/relative).is_file():payload['source/'+relative]=(REPO/relative).read_bytes() + for folder in ('_3rdLib/simpleini','_3rdLib/nlohmann','_3rdLib/Compress/zlib'): + for path in (EXT/folder).glob('*'): + if path.is_file() and ('license' in path.name.lower() or path.name=='README'): + payload['source/Extensions/'+str(path.relative_to(EXT))]=path.read_bytes() + payload.update(licenses or {}) + payload['Fini-manifest.json']=json.dumps({'extension':'Fini','requires':['OFEEncryption/1'],'artifactKind':'runtime' if abi_dirs else 'source-review','abis':list(abi_dirs),'nativeAPI':26,'format':'Win32 assets; UTF-8 INI; AES-128-CBC legacy KeyGen', + 'sha256':{name:hashlib.sha256(data).hexdigest() for name,data in payload.items()}},indent=2).encode() + destination.parent.mkdir(parents=True,exist_ok=True) + fd,temporary=tempfile.mkstemp(prefix='Fini-',suffix='.zip',dir=destination.parent);os.close(fd) + try: + with zipfile.ZipFile(temporary,'w',zipfile.ZIP_DEFLATED) as z: + for name,data in sorted(payload.items()):z.writestr(name,data) + os.replace(temporary,destination) + finally: + if Path(temporary).exists():Path(temporary).unlink() + + +def sync_apple(fusion=None): + plugin=apple_artifacts.validate(EXT/'iOS/Fini/Prebuilt','Fini') + common=apple_artifacts.validate(EXT/'iOS/Encryption/Prebuilt','OFEEncryption') + if plugin and not common:raise ValueError('Fini Apple handoff requires the separately committed common encryption artifact') + if not plugin and not common: + print('Apple libraries not staged yet; build/push on Mac, then pull and rebuild on Windows.') + return [] + if fusion and not (fusion/'Data/Runtime/iPhone').is_dir():raise ValueError('Fusion iPhone runtime directory is missing; set FusionRoot to the installed Fusion directory') + scratch=REPO/'work';scratch.mkdir(exist_ok=True) + prepared=[] + # Validate/generate every result before replacing any known-good output. + with tempfile.TemporaryDirectory(prefix='fini-apple-package-',dir=scratch) as temp: + temp=Path(temp) + if common: + path=EXT/'iOS/Encryption/ToInstall/OFEEncryption-iOS.zip' + staged=temp/path.name;apple_artifacts.package(common,staged);prepared.append((path,staged.read_bytes())) + if plugin: + path=EXT/'Fini/ToInstall/Files/Data/Runtime/iPhone/Fini-iOS.zip' + staged=temp/path.name + apple_artifacts.package(plugin,staged,{'FiniConsumer.xcconfig':EXT/'iOS/Fini/Config/FiniConsumer.xcconfig','README.md':EXT/'iOS/Fini/README.md'}) + prepared.append((path,staged.read_bytes())) + extension=path.with_name('Fini.ext');staged=temp/extension.name + FiniIOS.write(plugin,staged);prepared.append((extension,staged.read_bytes())) + for path,data in prepared: + apple_artifacts.atomic_write(path,data) + if fusion:apple_artifacts.atomic_write(fusion/'Data/Runtime/iPhone'/path.name,data) + print('Updated pulled Apple package:',path) + return [path for path,_ in prepared] + + +def main(): + p=argparse.ArgumentParser(description=__doc__) + p.add_argument('--sdk',type=Path);p.add_argument('--ndk',type=Path);p.add_argument('--openssl',type=Path,help='Target OpenSSL prefixes: //{include,lib/libcrypto.a}') + p.add_argument('--android-project',type=Path,help='Stage the common runtime once in this exported Android project; does not alter its Gradle files') + p.add_argument('--openssl-license',type=Path,help='OpenSSL distribution license if not stored under its prefix') + p.add_argument('--cmake');p.add_argument('--ninja');p.add_argument('--abis',nargs='+',choices=ABIS,default=list(ABIS));p.add_argument('--fusion',type=Path) + p.add_argument('--source-bundle',type=Path,help='Write a review source bundle; not an installable runtime') + p.add_argument('--ios-only',action='store_true',help='Package pulled Mac artifacts and archive/install without Android tools or Xcode') + p.add_argument('--prepare-only',action='store_true',help='Discover/prepare SDK, NDK, CMake and Ninja without compiling or packaging') + p.add_argument('--no-install-tools',action='store_true',help='Only use installed Android tools') + p.add_argument('--build',type=Path,default=REPO/'work/fini-android');a=p.parse_args() + if a.source_bundle: + package(a.source_bundle,{});print('Source review bundle:',a.source_bundle);return + fusion=a.fusion or Path(os.environ.get('FusionRoot',r'C:\Steam\steamapps\common\Clickteam Fusion 2.5')) + if not fusion.is_dir():fusion=None + sync_apple(fusion) + if a.ios_only: + archive_release();return + a.build=a.build.resolve();a.build.mkdir(parents=True,exist_ok=True) + log_path=a.build/'publisher.log' + def run(cmd): + command=list(map(str,cmd));print('+',' '.join(command),flush=True) + with log_path.open('a',encoding='utf-8') as log: + log.write('+ '+' '.join(command)+'\n');log.flush() + environment=os.environ.copy() + environment.setdefault('ANDROID_USER_HOME',str(a.build/'android-user')) + Path(environment['ANDROID_USER_HOME']).mkdir(parents=True,exist_ok=True) + result=subprocess.run(command,stdin=subprocess.DEVNULL,stdout=log,stderr=subprocess.STDOUT,env=environment) + if result.returncode:raise RuntimeError(f'Native tool/build command failed ({result.returncode}); see {log_path}') + log_path.write_text('Fini publisher: native tool preparation\n',encoding='utf-8') + try: + tools=FiniNativeTools.prepare(REPO,TOOLCHAIN,run,sdk=a.sdk,ndk=a.ndk,cmake=a.cmake,ninja=a.ninja,install=not a.no_install_tools) + except Exception: + with log_path.open('a',encoding='utf-8') as log:traceback.print_exc(file=log) + raise + sdk,ndk,cmake,ninja=(tools[k] for k in ('sdk','ndk','cmake','ninja')) + (a.build/'toolchain.json').write_text(json.dumps({k:str(v) for k,v in tools.items()},indent=2)) + if a.prepare_only: + print('Native tools ready:',a.build/'toolchain.json');return + openssl=a.openssl or (Path(os.environ['FINI_OPENSSL_ROOT']) if os.environ.get('FINI_OPENSSL_ROOT') else None) + if openssl is None: + openssl=REPO/'work/mobile-encryption/openssl/android' + run([sys.executable,EXT/'_Script/Encryption/prepare_openssl.py','--platform','android','--ndk',ndk,'--abis',*a.abis,'--api',str(TOOLCHAIN['native_api']),'--output',openssl,'--work',REPO/'work/mobile-encryption/openssl-build']) + license_path=a.openssl_license or next((candidate for candidate in [openssl/'LICENSE.txt',openssl/'LICENSE',openssl/'LICENSE.md'] if candidate.is_file()),None) + if license_path is None:raise RuntimeError('OpenSSL license missing; pass --openssl-license for redistribution') + licenses={'licenses/OpenSSL.txt':license_path.read_bytes()} + ndk_notice=ndk/'NOTICE' + if not ndk_notice.is_file():raise RuntimeError(f'Missing NDK redistribution notice: {ndk_notice}') + licenses['licenses/NDK-NOTICE.txt']=ndk_notice.read_bytes() + if (ndk/'NOTICE.toolchain').is_file():licenses['licenses/NDK-TOOLCHAIN-NOTICE.txt']=(ndk/'NOTICE.toolchain').read_bytes() + host={'Windows':'windows-x86_64','Linux':'linux-x86_64','Darwin':'darwin-x86_64'}[platform.system()] + common_stages=encryption_runtime.build(a.abis,REPO/'work/mobile-encryption/android',openssl,ndk,cmake,ninja,TOOLCHAIN['native_api'],run) + abi_dirs={} + for abi in a.abis: + prefix=openssl/abi + if not (prefix/'include/openssl/evp.h').is_file() or not (prefix/'lib/libcrypto.a').is_file():raise RuntimeError(f'Missing target OpenSSL: {prefix}') + build=a.build/abi + run([cmake,'-S',EXT/'Android_Gradle/cmake/Fini','-B',build,'-G','Ninja',f'-DCMAKE_MAKE_PROGRAM={ninja}',f'-DCMAKE_TOOLCHAIN_FILE={ndk}/build/cmake/android.toolchain.cmake',f'-DANDROID_ABI={abi}',f'-DANDROID_PLATFORM=android-{TOOLCHAIN["native_api"]}','-DANDROID_STL=c++_shared','-DANDROID_SUPPORT_FLEXIBLE_PAGE_SIZES=ON','-DCMAKE_BUILD_TYPE=Release',f'-DOFE_ENCRYPTION_LIBRARY={(common_stages[abi]/"libofe_encryption.so").resolve()}']) + run([cmake,'--build',build,'--parallel']) + stage=a.build/'stage'/abi;stage.mkdir(parents=True,exist_ok=True) + shutil.copy2(build/'libfini.so',stage/'libfini.so') + abi_dirs[abi]=stage + destination=EXT/'Fini/ToInstall/Files/Data/Runtime/Android/Fini.zip' + package(destination,abi_dirs) + common=encryption_runtime.publish(common_stages,licenses) + if a.android_project:encryption_runtime.install_once(common,a.android_project) + if fusion: + install=fusion/'Data/Runtime/Android/Fini.zip';install.parent.mkdir(parents=True,exist_ok=True);shutil.copy2(destination,install) + archive_release() + print('Published:',destination) + +def archive_release(): + archive=EXT/'Fini_Release.zip';temp=archive.with_suffix('.tmp') + with zipfile.ZipFile(temp,'w',zipfile.ZIP_DEFLATED) as z: + for path in (EXT/'Fini/ToInstall/Files').rglob('*'): + if path.is_file():z.write(path,path.relative_to(EXT/'Fini/ToInstall/Files')) + os.replace(temp,archive);print('Release archive:',archive) +if __name__=='__main__': + try:main() + except (RuntimeError,OSError,ValueError) as e:raise SystemExit('Fini publish failed: '+str(e)) diff --git a/Extensions/_Script/FiniIOS.py b/Extensions/_Script/FiniIOS.py new file mode 100644 index 000000000..782c1bb7f --- /dev/null +++ b/Extensions/_Script/FiniIOS.py @@ -0,0 +1,59 @@ +"""Package Fini.ext on Windows from the author's verified Mac native library. +Uses the independently verified legacy .ext codec from PR #100. No SDK source. +""" +import importlib.util +import io +from pathlib import Path +import plistlib +import zipfile + +ROOT=Path(__file__).resolve().parents[1] + + +def codec(): + spec=importlib.util.spec_from_file_location('fusion_ext',ROOT/'iOS/Tools/make_ext.py') + module=importlib.util.module_from_spec(spec);spec.loader.exec_module(module);return module + + +def contents(validated): + manifest,data=validated + if manifest['owner']!='Fini':raise ValueError('Expected Fini native artifacts') + payload={ + 'Classes/Extensions/CRunFini.m':b'''// The implementation is precompiled in the author's native archive. +#import "CRunFini.h" +#import +#if TARGET_OS_SIMULATOR +__asm__(".linker_option \\\"-lFini_iphonesimulator\\\""); +#else +__asm__(".linker_option \\\"-lFini_iphoneos\\\""); +#endif +__asm__(".linker_option \\\"-lc++\\\""); +__asm__(".linker_option \\\"-lz\\\""); +''', + 'configure_fini_export.py':(ROOT/'iOS/Fini/configure_export.py').read_bytes(), + 'xcode_project.py':(ROOT/'iOS/Tools/xcode_project.py').read_bytes(), + } + with zipfile.ZipFile(io.BytesIO(data)) as archive: + info=plistlib.loads(archive.read('Fini.xcframework/Info.plist'));seen=set();header=None + for entry in info['AvailableLibraries']: + if entry.get('SupportedPlatform')!='ios' or 'arm64' not in entry.get('SupportedArchitectures',[]):continue + variant=entry.get('SupportedPlatformVariant','device') + if variant not in ('device','simulator') or variant in seen:raise ValueError('Duplicate/unexpected Fini platform slice') + seen.add(variant);prefix='Fini.xcframework/'+entry['LibraryIdentifier']+'/' + binary=archive.read(prefix+entry['LibraryPath']) + if binary[:8]!=b'!\n' and binary[:4] not in (b'\xca\xfe\xba\xbe',b'\xbe\xba\xfe\xca',b'\xca\xfe\xba\xbf',b'\xbf\xba\xfe\xca'):raise ValueError('Expected a native static archive') + sdk='iphonesimulator' if variant=='simulator' else 'iphoneos' + payload['libFini_'+sdk+'.a']=binary + current=archive.read(prefix+entry.get('HeadersPath','Headers')+'/CRunFini.h').replace(b'\r\n',b'\n') + if header is not None and current!=header:raise ValueError('Device/simulator public headers differ') + header=current + if seen!={'device','simulator'}:raise ValueError('Both Fini platforms are required') + payload['Classes/Extensions/CRunFini.h']=header + return payload + + +def write(validated,output): + module=codec();payload=contents(validated);module.write_extension('Fini',payload,output) + decoded=module.read_extension(output.read_bytes()) + if any(decoded.get(name)!=data for name,data in payload.items()):raise ValueError('Fini .ext verification failed') + return output diff --git a/Extensions/_Script/FiniNativeTools.py b/Extensions/_Script/FiniNativeTools.py new file mode 100644 index 000000000..df76e8cac --- /dev/null +++ b/Extensions/_Script/FiniNativeTools.py @@ -0,0 +1,190 @@ +"""Fini native tool preparation, following FTW's installed-tool/sdkmanager flow. +No dependency on FindTheWay.py, its core, package names, or build outputs. +""" +import hashlib +import os +from pathlib import Path +import platform +import re +import shutil +import subprocess +import tempfile +import urllib.request +from urllib.parse import urlsplit +import xml.etree.ElementTree as ET +import zipfile + +REPOSITORY = 'https://dl.google.com/android/repository/' + + +def sdk_root(explicit, repo): + if explicit: + return Path(explicit).expanduser().resolve() + candidates = [Path(os.environ[k]).expanduser() for k in ('ANDROID_SDK_ROOT', 'ANDROID_HOME') if os.environ.get(k)] + local = repo / 'Extensions/Android_Gradle/local.properties' + if local.is_file(): + match = re.search(r'^\s*sdk\.dir\s*=\s*(.+)$', local.read_text(), re.M) + if match: + candidates.append(Path(match[1].replace('\\:', ':').replace('\\\\', '\\'))) + if os.environ.get('LOCALAPPDATA'): + candidates.append(Path(os.environ['LOCALAPPDATA']) / 'Android/Sdk') + candidates += [Path.home() / 'Library/Android/sdk', Path.home() / 'Android/Sdk'] + return next((p.resolve() for p in candidates if p.is_dir()), repo / 'work/toolchains/android-sdk') + + +def executable(name): + return name + ('.exe' if os.name == 'nt' else '') + + +def capture(command): + return subprocess.check_output(list(map(str, command)), text=True, encoding='utf-8', errors='replace').strip() + + +def find_build_tools(sdk, cmake=None, ninja=None): + """Reuse SDK, VS-bundled or PATH tools; VS Ninja may be in a sibling tree.""" + if cmake and not Path(cmake).is_file(): + raise RuntimeError('Explicit --cmake does not exist: ' + str(cmake)) + if ninja and not Path(ninja).is_file(): + raise RuntimeError('Explicit --ninja does not exist: ' + str(ninja)) + candidates = [Path(cmake)] if cmake else [] + if not cmake: + candidates += sorted((sdk / 'cmake').glob('*/bin/' + executable('cmake')), reverse=True) + if os.name == 'nt': + vswhere = Path(os.environ.get('ProgramFiles(x86)', r'C:\Program Files (x86)')) / 'Microsoft Visual Studio/Installer/vswhere.exe' + if vswhere.is_file(): + for line in capture([vswhere, '-all', '-products', '*', '-property', 'installationPath', '-utf8']).splitlines(): + if line.strip(): + candidates.append(Path(line.strip()) / 'Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/bin/cmake.exe') + if shutil.which('cmake'): + candidates.append(Path(shutil.which('cmake'))) + for candidate in candidates: + if not candidate.is_file(): + continue + try: + version = re.search(r'cmake version (\d+)\.(\d+)', capture([candidate, '--version'])) + except (OSError, subprocess.CalledProcessError): + continue + if not version or tuple(map(int, version.groups())) < (3, 22): + continue + ninjas = [Path(ninja)] if ninja else [candidate.parent / executable('ninja'), candidate.parent.parent.parent / 'Ninja' / executable('ninja')] + if not ninja: + ninjas += sorted((sdk/'cmake').glob('*/bin/'+executable('ninja')),reverse=True) + if shutil.which('ninja'):ninjas.append(Path(shutil.which('ninja'))) + for build in ninjas: + if build.is_file(): + try: + capture([build, '--version']) + return candidate.resolve(), build.resolve() + except (OSError, subprocess.CalledProcessError): + continue + return None + + +def find_sdkmanager(sdk): + name = 'sdkmanager.bat' if os.name == 'nt' else 'sdkmanager' + managers = list((sdk / 'cmdline-tools').glob('*/bin/' + name)) + managers.sort(key=lambda p: (p.parent.parent.name != 'latest', str(p))) + managers += [sdk / 'tools/bin' / name] + if shutil.which(name): + managers.append(Path(shutil.which(name))) + return next((p for p in managers if p.is_file()), None) + + +def command_line_archive(xml, version, host): + """Select an exact tools revision and its Google-published archive checksum.""" + root = ET.fromstring(xml) + for package in root.iter(): + if package.tag.rsplit('}', 1)[-1] != 'remotePackage' or package.attrib.get('path') != 'cmdline-tools;' + version: + continue + for archive in package.findall('./archives/archive'): + if archive.findtext('host-os') != host: + continue + complete = archive.find('complete') + checksum = complete.find('checksum') + url = complete.findtext('url') + algorithm = checksum.attrib.get('type', 'sha1') + if algorithm not in ('sha1', 'sha256') or not re.fullmatch(r'commandlinetools-[\w.-]+\.zip', url or ''): + raise RuntimeError('Unsupported SDK tools archive/checksum in Google metadata') + return url, algorithm, checksum.text.strip() + raise RuntimeError('Google repository metadata has no cmdline-tools;' + version + ' for ' + host) + + +def bootstrap_sdkmanager(sdk, version): + host = {'Windows': 'windows', 'Linux': 'linux', 'Darwin': 'macosx'}[platform.system()] + print('Preparing Android Command-line Tools ' + version + ' from Google repository metadata', flush=True) + try: + with urllib.request.urlopen(REPOSITORY + 'repository2-3.xml', timeout=30) as response: + metadata = response.read() + url, algorithm, expected = command_line_archive(metadata, version, host) + sdk.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory(prefix='fini-sdk-', dir=sdk) as temp: + temp = Path(temp) + archive = temp / 'tools.zip' + digest = hashlib.new(algorithm) + with urllib.request.urlopen(REPOSITORY + url, timeout=30) as source, archive.open('wb') as output: + while True: + block = source.read(1024 * 1024) + if not block: + break + digest.update(block) + output.write(block) + if digest.hexdigest().lower() != expected.lower(): + raise RuntimeError('Android Command-line Tools checksum mismatch') + unpacked = temp / 'unpacked' + with zipfile.ZipFile(archive) as zipped: + for entry in zipped.infolist(): + path = Path(entry.filename) + if path.is_absolute() or '..' in path.parts or '\\' in entry.filename: + raise RuntimeError('Unsafe SDK archive path') + zipped.extractall(unpacked) + target = sdk / 'cmdline-tools' / version + if target.exists(): + raise RuntimeError('Incomplete tools already exist at ' + str(target) + '; repair that directory explicitly') + target.parent.mkdir(parents=True, exist_ok=True) + shutil.move(str(unpacked / 'cmdline-tools'), target) + if os.name != 'nt': + for path in (target / 'bin').iterdir(): + path.chmod(path.stat().st_mode | 0o111) + manager = find_sdkmanager(sdk) + if not manager: + raise RuntimeError('Command-line Tools extraction did not produce sdkmanager') + return manager + except OSError as error: + raise RuntimeError('Cannot prepare Android Command-line Tools: ' + str(error) + + '. Use an existing SDK via --sdk/ANDROID_SDK_ROOT or repair network access; no NDK build was performed.') from error + + +def prepare(repo, config, run, sdk=None, ndk=None, cmake=None, ninja=None, install=True): + sdk = sdk_root(sdk, repo) + explicit_ndk = ndk or os.environ.get('ANDROID_NDK_HOME') + ndk = Path(explicit_ndk).expanduser().resolve() if explicit_ndk else sdk / 'ndk' / config['ndk'] + ready_ndk = (ndk / 'build/cmake/android.toolchain.cmake').is_file() + if explicit_ndk and not ready_ndk: + raise RuntimeError('Explicit Android NDK is invalid: ' + str(ndk)) + tools = find_build_tools(sdk, cmake, ninja) + packages = [] + if not ready_ndk: + packages.append('ndk;' + config['ndk']) + if not tools: + packages.append('cmake;' + config['cmake']) + if packages: + if not install: + raise RuntimeError('Missing Android packages: ' + ', '.join(packages) + '; allow preparation or install them with sdkmanager') + manager = find_sdkmanager(sdk) or bootstrap_sdkmanager(sdk, config['command_line_tools']) + # Existing accepted licenses are respected. Never feed automatic license acceptance. + proxy=[] + proxy_url=os.environ.get('HTTPS_PROXY') or os.environ.get('https_proxy') or os.environ.get('HTTP_PROXY') or os.environ.get('http_proxy') + if proxy_url: + address=urlsplit(proxy_url) + if address.hostname: + proxy=['--proxy=http','--proxy_host='+address.hostname,'--proxy_port='+str(address.port or 80)] + run([manager, '--sdk_root=' + str(sdk), *proxy, '--install', *packages]) + tools = find_build_tools(sdk, cmake, ninja) + if not (ndk / 'build/cmake/android.toolchain.cmake').is_file() or not tools: + raise RuntimeError('Native tools remain incomplete after sdkmanager. Inspect publisher.log, JAVA_HOME, and sdkmanager --licenses for the selected SDK.') + properties = ndk / 'source.properties' + if properties.is_file(): + match = re.search(r'Pkg.Revision\s*=\s*(\S+)', properties.read_text()) + if not match or int(match[1].split('.')[0]) < 28: + raise RuntimeError('Fini requires NDK 28+ for its native/STL build contract: ' + str(ndk)) + return {'sdk': sdk, 'ndk': ndk, 'cmake': tools[0], 'ninja': tools[1]} diff --git a/Extensions/iOS/Encryption/.gitignore b/Extensions/iOS/Encryption/.gitignore new file mode 100644 index 000000000..c7b33616f --- /dev/null +++ b/Extensions/iOS/Encryption/.gitignore @@ -0,0 +1,2 @@ +ToInstall/ +**/__pycache__/ diff --git a/Extensions/iOS/Encryption/Prebuilt/README.md b/Extensions/iOS/Encryption/Prebuilt/README.md new file mode 100644 index 000000000..fe521ba04 --- /dev/null +++ b/Extensions/iOS/Encryption/Prebuilt/README.md @@ -0,0 +1,6 @@ +# Common author-built Apple runtime + +The Mac common build stages OFEEncryption.xcframework.zip and manifest.json here. +Commit/push both once for all dependent plugins. Windows packaging checks hashes, +source freshness and required slices before updating OFEEncryption-iOS.zip. +Do not duplicate these binaries in each plugin's Prebuilt directory. diff --git a/Extensions/iOS/Encryption/README.md b/Extensions/iOS/Encryption/README.md new file mode 100644 index 000000000..a06004e9f --- /dev/null +++ b/Extensions/iOS/Encryption/README.md @@ -0,0 +1,21 @@ +# Shared iOS encryption runtime + +`iOS/Encryption/build.py` builds a **dynamic** OFEEncryption.framework for device arm64 and +simulator arm64/x86_64, then combines them into OFEEncryption.xcframework. +Only it links libcrypto; Fini.xcframework contains the adapter without crypto. +Add and embed/sign the shared framework once in the final app; link all plugins +against that same framework. Do not merge libcrypto into each plugin archive. +The common library exposes the stable Encryption_* C ABI and keeps EVP symbols +private through an export list. Its install name is +`@rpath/OFEEncryption.framework/OFEEncryption`. + +```sh +python3 Extensions/iOS/Encryption/build.py \ + --openssl work/mobile-encryption/openssl/ios \ + --output work/mobile-encryption/ios --prepare-openssl +``` + +The build stages native handoff inputs into this directory's `Prebuilt` folder. +The author commits/pushes them; Windows packages them through Fini.py. The C++ +implementation remains in `_DeLib/Encryption`. Shared dependency preparation uses +`Extensions/_Script/Encryption/prepare_openssl.py`. diff --git a/Extensions/iOS/Encryption/build.py b/Extensions/iOS/Encryption/build.py new file mode 100644 index 000000000..88ca41261 --- /dev/null +++ b/Extensions/iOS/Encryption/build.py @@ -0,0 +1,62 @@ +#!/usr/bin/env python3 +"""Build one shared OFEEncryption XCFramework for all Fusion plugins.""" +import argparse +import os +from pathlib import Path +import plistlib +import re +import shutil +import subprocess +import sys + + +def main(): + p=argparse.ArgumentParser(description=__doc__) + p.add_argument('--openssl',type=Path,required=True) + p.add_argument('--output',type=Path,required=True) + p.add_argument('--prepare-openssl',action='store_true') + a=p.parse_args();root=Path(__file__).resolve().parent;extensions=root.parents[1];out=a.output.resolve() + sys.path.insert(0,str(extensions/'_Script/Encryption')) + sys.path.insert(0,str(root.parent/'Tools')) + if not shutil.which('xcrun') or not shutil.which('xcodebuild'):raise SystemExit('Shared iOS encryption requires Xcode on macOS') + prefixes=[('iphoneos','arm64'),('iphonesimulator','arm64'),('iphonesimulator','x86_64')] + if any(not (a.openssl/f'{sdk}-{arch}/lib/libcrypto.a').is_file() for sdk,arch in prefixes): + if not a.prepare_openssl:raise SystemExit('Missing target OpenSSL; use --prepare-openssl or supply prepared prefixes') + from prepare_openssl import prepare + prepare('ios',a.openssl,out/'openssl-build') + out.mkdir(parents=True,exist_ok=True) + # Export the stable C ABI only; keep EVP and C++ implementation symbols private. + capi=extensions/'_DeLib/Encryption/CAPI.h';symbols=re.findall(r'ENCRYPTION_API\s+(?:\w+\s+)+(Encryption_\w+)\s*\(',capi.read_text()) + export_list=out/'exports.txt';export_list.write_text(''.join('_'+name+'\n' for name in sorted(set(symbols)))) + with (out/'publisher.log').open('w') as log: + def run(cmd): + print('+',' '.join(map(str,cmd)),flush=True);subprocess.run(list(map(str,cmd)),stdout=log,stderr=subprocess.STDOUT,check=True) + frameworks={} + for sdk,arch in prefixes: + prefix=a.openssl.resolve()/f'{sdk}-{arch}';framework=out/f'{sdk}-{arch}/OFEEncryption.framework' + headers=framework/'Headers';headers.mkdir(parents=True,exist_ok=True);shutil.copy2(capi,headers/'CAPI.h') + modules=framework/'Modules';modules.mkdir(exist_ok=True) + (modules/'module.modulemap').write_text('framework module OFEEncryption { umbrella header "CAPI.h" export * }\n') + plist={'CFBundleExecutable':'OFEEncryption','CFBundleIdentifier':'org.openfusionexamples.encryption','CFBundleName':'OFEEncryption','CFBundlePackageType':'FMWK','CFBundleShortVersionString':'1.0','CFBundleVersion':'1','MinimumOSVersion':'17.0','CFBundleSupportedPlatforms':['iPhoneOS' if sdk=='iphoneos' else 'iPhoneSimulator']} + (framework/'Info.plist').write_bytes(plistlib.dumps(plist)) + sysroot=subprocess.check_output(['xcrun','--sdk',sdk,'--show-sdk-path'],text=True).strip() + target=f'{arch}-apple-ios17.0'+('-simulator' if sdk=='iphonesimulator' else '') + run(['xcrun','--sdk',sdk,'clang++','-std=c++17','-O2','-fPIC','-fvisibility=hidden','-fvisibility-inlines-hidden','-DENCRYPTION_SHARED','-DENCRYPTION_BUILD','-D_FILE_OFFSET_BITS=64','-isysroot',sysroot,'-target',target,'-dynamiclib',f'-I{prefix}/include',capi.with_suffix('.cpp'),prefix/'lib/libcrypto.a','-Wl,-install_name,@rpath/OFEEncryption.framework/OFEEncryption','-Wl,-exported_symbols_list,'+str(export_list),'-o',framework/'OFEEncryption']) + frameworks[(sdk,arch)]=framework + sim=out/'simulator/OFEEncryption.framework' + if sim.exists():shutil.rmtree(sim) + shutil.copytree(frameworks[('iphonesimulator','arm64')],sim) + run(['xcrun','lipo','-create',frameworks[('iphonesimulator','arm64')]/'OFEEncryption',frameworks[('iphonesimulator','x86_64')]/'OFEEncryption','-output',sim/'OFEEncryption']) + pending=out/'pending.xcframework';final=out/'OFEEncryption.xcframework' + if pending.exists():shutil.rmtree(pending) + run(['xcodebuild','-create-xcframework','-framework',frameworks[('iphoneos','arm64')],'-framework',sim,'-output',pending]) + if final.exists():shutil.rmtree(final) + pending.rename(final) + if (a.openssl/'LICENSE.txt').is_file():shutil.copy2(a.openssl/'LICENSE.txt',out/'OpenSSL-LICENSE.txt') + from apple_artifacts import stage,REPO + sources=['Extensions/_DeLib/Encryption/'+name for name in ('CAPI.h','CAPI.cpp','Core.h','KeyGen.h')]+['Extensions/_DeLib/UnicodeConversion.h','Extensions/iOS/Encryption/build.py','Extensions/_Script/Encryption/prepare_openssl.py'] + licenses={'OpenSSL-LICENSE.txt':a.openssl/'LICENSE.txt'} if (a.openssl/'LICENSE.txt').is_file() else {} + stage(final,'OFEEncryption',sources,root/'Prebuilt',extra_files=licenses) + print('Shared runtime for all plugins:',final) + +if __name__=='__main__':main() diff --git a/Extensions/iOS/Fini/.gitignore b/Extensions/iOS/Fini/.gitignore new file mode 100644 index 000000000..0a415baae --- /dev/null +++ b/Extensions/iOS/Fini/.gitignore @@ -0,0 +1,5 @@ +Config/Local.xcconfig +Dependencies/ +SDK/ +**/xcuserdata/ +**/__pycache__/ diff --git a/Extensions/iOS/Fini/Config/Fini.xcconfig b/Extensions/iOS/Fini/Config/Fini.xcconfig new file mode 100644 index 000000000..6f99333e7 --- /dev/null +++ b/Extensions/iOS/Fini/Config/Fini.xcconfig @@ -0,0 +1,22 @@ +// Fini-owned standalone project. SDK code stays in the user's licensed export. +FINI_RUNTIME_ROOT = $(SRCROOT)/SDK +FINI_OPENSSL_BASE = $(SRCROOT)/../../../work/mobile-encryption/openssl/ios +FINI_OUTPUT_ROOT = $(SRCROOT)/../../../work/fini-ios +#include? "Local.xcconfig" + +SDKROOT = iphoneos +SUPPORTED_PLATFORMS = iphoneos iphonesimulator +IPHONEOS_DEPLOYMENT_TARGET = 17.0 +CLANG_CXX_LANGUAGE_STANDARD = c++20 +CLANG_CXX_LIBRARY = libc++ +CLANG_ENABLE_OBJC_ARC = NO +GCC_ENABLE_CPP_EXCEPTIONS = YES +GCC_ENABLE_OBJC_EXCEPTIONS = YES +GCC_PREPROCESSOR_DEFINITIONS = $(inherited) _FILE_OFFSET_BITS=64 FINI_PORTABLE_RUNTIME +GCC_PRECOMPILE_PREFIX_HEADER = NO +CODE_SIGNING_ALLOWED = NO +ENABLE_USER_SCRIPT_SANDBOXING = NO + +FINI_OPENSSL_ROOT[sdk=iphoneos*] = $(FINI_OPENSSL_BASE)/iphoneos-$(CURRENT_ARCH) +FINI_OPENSSL_ROOT[sdk=iphonesimulator*] = $(FINI_OPENSSL_BASE)/iphonesimulator-$(CURRENT_ARCH) +HEADER_SEARCH_PATHS = $(inherited) "$(FINI_RUNTIME_ROOT)/Classes/**" "$(SRCROOT)/../../_DeLib" diff --git a/Extensions/iOS/Fini/Config/FiniConsumer.xcconfig b/Extensions/iOS/Fini/Config/FiniConsumer.xcconfig new file mode 100644 index 000000000..369805463 --- /dev/null +++ b/Extensions/iOS/Fini/Config/FiniConsumer.xcconfig @@ -0,0 +1,4 @@ +// Add Fini.xcframework to the exporter's Frameworks/Libraries list separately. +// Compile the adapter source OR link the XCFramework, never both. +OTHER_LDFLAGS = $(inherited) -ObjC -lc++ -lz -framework Foundation -framework OFEEncryption +IPHONEOS_DEPLOYMENT_TARGET = 17.0 diff --git a/Extensions/iOS/Fini/Fini.xcodeproj/project.pbxproj b/Extensions/iOS/Fini/Fini.xcodeproj/project.pbxproj new file mode 100644 index 000000000..806dff986 --- /dev/null +++ b/Extensions/iOS/Fini/Fini.xcodeproj/project.pbxproj @@ -0,0 +1,231 @@ +// !$*UTF8*$! +{ + "archiveVersion" = "1"; + "classes" = { + }; + "objectVersion" = "56"; + "objects" = { + "25619D1565A8049216978411" = { + "isa" = "PBXFileReference"; + "lastKnownFileType" = "sourcecode.cpp.objcpp"; + "path" = "Runtime/CRunFini.mm"; + "sourceTree" = "SOURCE_ROOT"; + }; + "DE505F91BAFC9AEDB3C97C1E" = { + "isa" = "PBXFileReference"; + "lastKnownFileType" = "sourcecode.c.h"; + "path" = "Runtime/CRunFini.h"; + "sourceTree" = "SOURCE_ROOT"; + }; + "986EDD7755335BC4166424BD" = { + "isa" = "PBXFileReference"; + "lastKnownFileType" = "text.xcconfig"; + "path" = "Config/Fini.xcconfig"; + "sourceTree" = "SOURCE_ROOT"; + }; + "02D8EB259C919A7CD52E6F42" = { + "isa" = "PBXFileReference"; + "explicitFileType" = "archive.ar"; + "path" = "libFini.a"; + "sourceTree" = "BUILT_PRODUCTS_DIR"; + "includeInIndex" = "0"; + }; + "A4863FFF55AB28D8C945DF63" = { + "isa" = "PBXBuildFile"; + "fileRef" = "25619D1565A8049216978411"; + }; + "209E99E9F50150AC040E6977" = { + "isa" = "PBXSourcesBuildPhase"; + "buildActionMask" = "2147483647"; + "files" = ( + "A4863FFF55AB28D8C945DF63", + ); + "runOnlyForDeploymentPostprocessing" = "0"; + }; + "0744B1CB3EA3A62A0AD63F85" = { + "isa" = "PBXFrameworksBuildPhase"; + "buildActionMask" = "2147483647"; + "files" = ( + ); + "runOnlyForDeploymentPostprocessing" = "0"; + }; + "244A1E8C4E751E420CC600EA" = { + "isa" = "PBXShellScriptBuildPhase"; + "buildActionMask" = "2147483647"; + "files" = ( + ); + "inputPaths" = ( + ); + "outputPaths" = ( + ); + "alwaysOutOfDate" = "1"; + "runOnlyForDeploymentPostprocessing" = "0"; + "shellPath" = "/bin/sh"; + "shellScript" = "set -eu\npython3 \"$SRCROOT/build.py\" --runtime \"$FINI_RUNTIME_ROOT\" --openssl \"$FINI_OPENSSL_BASE\" --output \"$FINI_OUTPUT_ROOT\" --prepare-openssl\n"; + }; + "302074DAA8E78094C2F9FEEB" = { + "isa" = "XCBuildConfiguration"; + "baseConfigurationReference" = "986EDD7755335BC4166424BD"; + "buildSettings" = { + "GCC_OPTIMIZATION_LEVEL" = "0"; + "DEBUG_INFORMATION_FORMAT" = "dwarf"; + }; + "name" = "Debug"; + }; + "111A78A4176F1B619ED5D5DB" = { + "isa" = "XCBuildConfiguration"; + "baseConfigurationReference" = "986EDD7755335BC4166424BD"; + "buildSettings" = { + "GCC_OPTIMIZATION_LEVEL" = "s"; + "DEBUG_INFORMATION_FORMAT" = "dwarf-with-dsym"; + }; + "name" = "Release"; + }; + "985152BA51FAF2358CF57152" = { + "isa" = "XCConfigurationList"; + "buildConfigurations" = ( + "302074DAA8E78094C2F9FEEB", + "111A78A4176F1B619ED5D5DB", + ); + "defaultConfigurationIsVisible" = "0"; + "defaultConfigurationName" = "Release"; + }; + "499A4CD32584D92F1FB7CE82" = { + "isa" = "XCBuildConfiguration"; + "baseConfigurationReference" = "986EDD7755335BC4166424BD"; + "buildSettings" = { + "GCC_OPTIMIZATION_LEVEL" = "0"; + "DEBUG_INFORMATION_FORMAT" = "dwarf"; + "PRODUCT_NAME" = "Fini"; + "SKIP_INSTALL" = "NO"; + "PUBLIC_HEADERS_FOLDER_PATH" = "include/Fini"; + }; + "name" = "Debug"; + }; + "E5963FE4EDF8005EFFD37C54" = { + "isa" = "XCBuildConfiguration"; + "baseConfigurationReference" = "986EDD7755335BC4166424BD"; + "buildSettings" = { + "GCC_OPTIMIZATION_LEVEL" = "s"; + "DEBUG_INFORMATION_FORMAT" = "dwarf-with-dsym"; + "PRODUCT_NAME" = "Fini"; + "SKIP_INSTALL" = "NO"; + "PUBLIC_HEADERS_FOLDER_PATH" = "include/Fini"; + }; + "name" = "Release"; + }; + "590D829C40C13DADB675CB2D" = { + "isa" = "XCConfigurationList"; + "buildConfigurations" = ( + "499A4CD32584D92F1FB7CE82", + "E5963FE4EDF8005EFFD37C54", + ); + "defaultConfigurationIsVisible" = "0"; + "defaultConfigurationName" = "Release"; + }; + "D6661A475B5DE2A5A4BB2F3B" = { + "isa" = "XCBuildConfiguration"; + "baseConfigurationReference" = "986EDD7755335BC4166424BD"; + "buildSettings" = { + "GCC_OPTIMIZATION_LEVEL" = "0"; + "DEBUG_INFORMATION_FORMAT" = "dwarf"; + }; + "name" = "Debug"; + }; + "C34244C6B3095FED8CA8915B" = { + "isa" = "XCBuildConfiguration"; + "baseConfigurationReference" = "986EDD7755335BC4166424BD"; + "buildSettings" = { + "GCC_OPTIMIZATION_LEVEL" = "s"; + "DEBUG_INFORMATION_FORMAT" = "dwarf-with-dsym"; + }; + "name" = "Release"; + }; + "D64414A6F62FFF6E6EFB5244" = { + "isa" = "XCConfigurationList"; + "buildConfigurations" = ( + "D6661A475B5DE2A5A4BB2F3B", + "C34244C6B3095FED8CA8915B", + ); + "defaultConfigurationIsVisible" = "0"; + "defaultConfigurationName" = "Release"; + }; + "3AAA2DA11429F0AA47388A7B" = { + "isa" = "PBXNativeTarget"; + "buildConfigurationList" = "590D829C40C13DADB675CB2D"; + "buildPhases" = ( + "209E99E9F50150AC040E6977", + "0744B1CB3EA3A62A0AD63F85", + ); + "buildRules" = ( + ); + "dependencies" = ( + ); + "name" = "Fini"; + "productName" = "Fini"; + "productReference" = "02D8EB259C919A7CD52E6F42"; + "productType" = "com.apple.product-type.library.static"; + }; + "1B738AB4CDE142147586E965" = { + "isa" = "PBXAggregateTarget"; + "buildConfigurationList" = "D64414A6F62FFF6E6EFB5244"; + "buildPhases" = ( + "244A1E8C4E751E420CC600EA", + ); + "dependencies" = ( + ); + "name" = "FiniXCFramework"; + "productName" = "FiniXCFramework"; + }; + "16E27E772C412D419512CC72" = { + "isa" = "PBXGroup"; + "children" = ( + "02D8EB259C919A7CD52E6F42", + ); + "name" = "Products"; + "sourceTree" = ""; + }; + "44E61B7163DE074E6E7F2252" = { + "isa" = "PBXGroup"; + "children" = ( + "25619D1565A8049216978411", + "DE505F91BAFC9AEDB3C97C1E", + "986EDD7755335BC4166424BD", + ); + "name" = "Fini sources and configuration"; + "sourceTree" = ""; + }; + "C437995D56617478DA23F48D" = { + "isa" = "PBXGroup"; + "children" = ( + "44E61B7163DE074E6E7F2252", + "16E27E772C412D419512CC72", + ); + "sourceTree" = ""; + }; + "103D3A095E4B3D68173566BF" = { + "isa" = "PBXProject"; + "attributes" = { + "BuildIndependentTargetsInParallel" = "YES"; + "LastUpgradeCheck" = "1500"; + }; + "buildConfigurationList" = "985152BA51FAF2358CF57152"; + "compatibilityVersion" = "Xcode 14.0"; + "developmentRegion" = "en"; + "hasScannedForEncodings" = "0"; + "knownRegions" = ( + "en", + "Base", + ); + "mainGroup" = "C437995D56617478DA23F48D"; + "productRefGroup" = "16E27E772C412D419512CC72"; + "projectDirPath" = ""; + "projectRoot" = ""; + "targets" = ( + "3AAA2DA11429F0AA47388A7B", + "1B738AB4CDE142147586E965", + ); + }; + }; + "rootObject" = "103D3A095E4B3D68173566BF"; +} diff --git a/Extensions/iOS/Fini/Fini.xcodeproj/project.xcworkspace/contents.xcworkspacedata b/Extensions/iOS/Fini/Fini.xcodeproj/project.xcworkspace/contents.xcworkspacedata new file mode 100644 index 000000000..7337895b0 --- /dev/null +++ b/Extensions/iOS/Fini/Fini.xcodeproj/project.xcworkspace/contents.xcworkspacedata @@ -0,0 +1,2 @@ + + diff --git a/Extensions/iOS/Fini/Fini.xcodeproj/xcshareddata/xcschemes/Fini.xcscheme b/Extensions/iOS/Fini/Fini.xcodeproj/xcshareddata/xcschemes/Fini.xcscheme new file mode 100644 index 000000000..5aef04041 --- /dev/null +++ b/Extensions/iOS/Fini/Fini.xcodeproj/xcshareddata/xcschemes/Fini.xcscheme @@ -0,0 +1,12 @@ + + + + + + + + + + + + diff --git a/Extensions/iOS/Fini/Fini.xcodeproj/xcshareddata/xcschemes/FiniXCFramework.xcscheme b/Extensions/iOS/Fini/Fini.xcodeproj/xcshareddata/xcschemes/FiniXCFramework.xcscheme new file mode 100644 index 000000000..13e833f28 --- /dev/null +++ b/Extensions/iOS/Fini/Fini.xcodeproj/xcshareddata/xcschemes/FiniXCFramework.xcscheme @@ -0,0 +1,12 @@ + + + + + + + + + + + + diff --git a/Extensions/iOS/Fini/Prebuilt/README.md b/Extensions/iOS/Fini/Prebuilt/README.md new file mode 100644 index 000000000..672182e26 --- /dev/null +++ b/Extensions/iOS/Fini/Prebuilt/README.md @@ -0,0 +1,7 @@ +# Author-built Mac artifacts + +FiniXCFramework stages Fini.xcframework.zip and manifest.json here after a successful +Mac build. Commit/push both files together; do not add placeholders. Windows pull +and VS RunOnly's Fini.py validate them and generate/update Fini-iOS.zip automatically. +The shared runtime is committed separately in Extensions/iOS/Encryption/Prebuilt. +No exporter SDK source belongs here. diff --git a/Extensions/iOS/Fini/README.md b/Extensions/iOS/Fini/README.md new file mode 100644 index 000000000..2121c19fb --- /dev/null +++ b/Extensions/iOS/Fini/README.md @@ -0,0 +1,107 @@ +# Build Fini in Xcode + +This is an owned, standalone Xcode project. Generation does not require Xcode. +It reuses the existing C++ INI/encryption/codecs through CRunFini.mm and contains +no licensed exporter Runtime sources or Windows crypto binaries. + +1. Keep this directory in its repository location: `Extensions/iOS/Fini`. +2. On your Mac, configure your licensed Fusion export and target OpenSSL: + + ```sh + python3 Extensions/iOS/Fini/generate_project.py \ + --runtime "/path/to/Fusion-export" + open Extensions/iOS/Fini/Fini.xcodeproj + ``` + + The export must contain `Classes/Extensions/CRunExtension.h`. OpenSSL prefixes + must be `iphoneos-arm64`, `iphonesimulator-arm64`, `iphonesimulator-x86_64`, + each containing `include/openssl/evp.h` and `lib/libcrypto.a`. If omitted, the + common runtime builds these prefixes from pinned source. Supply iOS-built + OpenSSL 3, not a macOS/Homebrew or Windows library. The shared runtime + dependency builder can download and verify the pinned source; SDK sources are + always supplied locally. + Paths are saved in ignored `Config/Local.xcconfig`. You can edit that file + directly instead. No signing identity is needed to build the library. +3. Select scheme **FiniXCFramework**, then Product > Build. It compiles all three + slices, combines the simulator architectures and produces + `work/fini-ios/Fini.xcframework` containing only the adapter. The common module + independently produces `work/mobile-encryption/ios/OFEEncryption.xcframework`, + a dynamic framework containing the shared encryption implementation. + Output and errors are recorded in `work/fini-ios/publisher.log`. +4. Add both XCFrameworks to your exported app. Embed & Sign **OFEEncryption once** + for all dependent plugins; do not embed copies per plugin. Link Foundation, libc++ and + libz. Use `-ObjC` to retain the CRunFini class. The supplied + `Config/FiniConsumer.xcconfig` records the consumer flags; merge them with the + app's existing settings. Select your signing team in the app project when + running a device build. + +The **Fini** scheme is an ordinary static-library target useful for compiling +and debugging just the adapter for the selected device/simulator architecture. +Its `libFini.a` does not include encryption or libcrypto; **FiniXCFramework** +builds the client library and the separate common runtime. The project targets iOS 17+ for the reused C++20 +library facilities. It is intended for Xcode 15+ with the corresponding SDK. + +## Windows .ext and exported-app integration + +Windows packaging now reuses the independently verified legacy `.ext` codec from +PR #100 (`Extensions/iOS/Tools/make_ext.py`). It generates and decodes Fini.ext, +checks all payload/CRC data and six PBX insertion blocks, and packages a thin +loader plus the author's precompiled platform archives. The implementation is +already in those archives; no second Runtime implementation is compiled on Windows. +The separately owned OFEEncryption framework is not embedded in each plugin .ext. + +After Fusion exports on Mac, close Xcode and run the bundled configurator: + +```sh +python3 configure_fini_export.py . --common /path/to/OFEEncryption.xcframework +``` + +It links device/simulator archives, adds one shared framework reference and +Embed & Sign phase, preserves device signing/unrelated flags, fixes stale scheme +references and keeps a higher deployment minimum. Use --dry-run to preview or +--target for an export with multiple apps. Repeating it is a no-op; it refuses a +conflicting shared framework. The source form is available at +`Extensions/iOS/Fini/configure_export.py` for an existing export. Replace your +Application.cci with the Fini test application's export where appropriate. + +This verifies encoding and project edits, not a Fusion export/Apple final link or +device run. Those remain the author's platform checks. Unlike FTW's separately +compiled algorithm core, Fini's present archive includes the Objective-C++ adapter; +changes to that compiled adapter legitimately require a Mac rebuild. Export-only +configurator/loader edits are packaged on Windows without rebuilding the library. + +The generated project and shared schemes have been checked on Linux for stable +IDs, valid object references, correct source/configuration paths and scheme +references. Actual Xcode compilation and runtime validation must run on your Mac. +Regenerate the checked-in project after changing generate_project.py: + +```sh +python3 Extensions/iOS/Fini/generate_project.py +python3 Extensions/iOS/Fini/tests/test_project.py +``` + +## Mac build/push -> Windows pull/VS packaging + +After a successful FiniXCFramework build, the scripts stage these owned artifacts: + +- `Extensions/iOS/Fini/Prebuilt/Fini.xcframework.zip` and `manifest.json`. +- `Extensions/iOS/Encryption/Prebuilt/OFEEncryption.xcframework.zip` + and `manifest.json`, shared once by all dependent plugins. + +The author commits/pushes those four files after building on Mac. No script pushes +Git automatically. On Windows, git pull followed by VS Run_Only Unicode invokes +the existing Fini.bat/Fini.py entry point. It validates SHA256, required slices and +source freshness, then updates Fini-iOS.zip and the separate OFEEncryption-iOS.zip, +installs them when FusionRoot/the normal Fusion directory exists, and rebuilds +Fini_Release.zip. Windows does not need Xcode. To run just this handoff step: + +```sh +python Extensions/_Script/Fini.py --ios-only +``` + +No Mac binaries are included until the author builds them. Before that first +build the publisher reports the missing handoff and leaves Apple outputs alone; +an incomplete, corrupt or source-stale committed pair fails instead of replacing +a known-good package. The handoff creates native XCFramework distribution ZIPs; +Windows also generates a format-verified Fini.ext with the shared PR #100 codec. +An actual exporter round trip, Apple final link and devices still require validation. diff --git a/Extensions/iOS/Fini/Runtime/CRunFini.h b/Extensions/iOS/Fini/Runtime/CRunFini.h new file mode 100644 index 000000000..afc99f157 --- /dev/null +++ b/Extensions/iOS/Fini/Runtime/CRunFini.h @@ -0,0 +1,10 @@ +#import "CRunExtension.h" + +// Native pointer stays opaque to Objective-C importers and the exporter loader. +@interface CRunFini : CRunExtension { + void* finiState; + BOOL cf25p, allowRVforCS, keep, linked; + int localizationFixed; + NSString *currentSection, *currentItem, *sectionLoop, *itemLoop; +} +@end diff --git a/Extensions/iOS/Fini/Runtime/CRunFini.mm b/Extensions/iOS/Fini/Runtime/CRunFini.mm new file mode 100644 index 000000000..6027f3f1b --- /dev/null +++ b/Extensions/iOS/Fini/Runtime/CRunFini.mm @@ -0,0 +1,125 @@ +#import "CRunFini.h" +#import "CRun.h" +#import "CRunApp.h" +#import "CExtension.h" +#import "CActExtension.h" +#import "CCndExtension.h" +#import "CFile.h" +#import "CValue.h" +#import "CObject.h" +#import "CObjInfo.h" +#import "CRVal.h" +#import "CRMvt.h" +#import "CRCom.h" +#include "../../../Fini/IniCore.h" + +using fini::Json; +static std::string utf8(NSString* s) {if(!s)return {};NSData* d=[s dataUsingEncoding:NSUTF8StringEncoding];fini::check(d!=nil,"Invalid Unicode string");return std::string((const char*)[d bytes],[d length]);} +static NSString* ns(const std::string& s) {return [[[NSString alloc] initWithBytes:s.data() length:s.size() encoding:NSUTF8StringEncoding] autorelease];} +static void replace(NSString*& dst,NSString* src) {[src retain];[dst release];dst=src;} +static NSMapTable* stashed; + +@implementation CRunFini +-(fini::State&)state {return *static_cast(finiState);} +-(int)getNumberOfConditions {return 4;} +-(BOOL)createRunObject:(CFile*)file withCOB:(CCreateObjectInfo*)cob andVersion:(int)version { + cf25p=version>=2&&[file readAByte]!=0;allowRVforCS=version>=2&&[file readAByte]!=0;keep=version>=3&&[file readAByte]!=0; + currentSection=[@"" retain];currentItem=[@"" retain];sectionLoop=[@"" retain];itemLoop=[@"" retain]; + try {finiState=new fini::State; + if(!stashed)stashed=[[NSMapTable weakToStrongObjectsMapTable] retain]; + if(keep){NSMutableDictionary* data=[stashed objectForKey:rh->rhApp];NSString* value=[data objectForKey:ho->hoOiList->oilName];if(value){[self state].load(utf8(value));[data removeObjectForKey:ho->hoOiList->oilName];}} + return YES; + }catch(const std::exception& e){NSLog(@"Fini create: %s",e.what());return NO;} +} +-(void)destroyRunObject:(BOOL)fast { + if(!finiState)return; + try {[self state].flush();}catch(const std::exception& e){NSLog(@"Fini auto save: %s",e.what());} + if(keep&&[self state].initialized()) { + NSMutableDictionary* data=[stashed objectForKey:rh->rhApp]; + if(!data){data=[NSMutableDictionary dictionary];[stashed setObject:data forKey:rh->rhApp];} + NSString* name=ho->hoOiList->oilName;if(![data objectForKey:name])[data setObject:ns([self state].save()) forKey:name]; + } + delete static_cast(finiState);finiState=nullptr; + [currentSection release];[currentItem release];[sectionLoop release];[itemLoop release]; +} +-(CObject*)object:(int)fixed {CObject* o=[ho getObjectFromFixed:fixed];return o&&[o fixedValue]==fixed&&!(o->hoFlags&HOF_DESTROYED)?o:nil;} +-(void)updateLocalization { + if(!linked)return;CObject* o=[self object:localizationFixed]; + if([o isKindOfClass:[CExtension class]]&&[((CExtension*)o)->ext isKindOfClass:[CRunFini class]]) { + CRunFini* other=(CRunFini*)((CExtension*)o)->ext; + [self state].localizeFrom(other->finiState?[other state].save():""); + }else {linked=NO;[self state].localizeFrom("");} +} +-(int)handleRunObject {return 0;} +-(std::string)writePath:(NSString*)path {return utf8([rh->rhApp getPathForWriting:path]);} +-(std::string)readBytes:(NSString*)path { + // Exporter resolver handles bundled resources as well as sandbox files. + NSData* bytes=[rh->rhApp loadResourceData:path];fini::check(bytes!=nil,"Open INI input"); + return fini::base64(std::string((const char*)[bytes bytes],[bytes length])); +} +-(BOOL)condition:(int)n withCndExtension:(CCndExtension*)c { + if(!finiState)return NO; + try {switch(n){ + case 0:return [sectionLoop isEqualToString:[c getParamExpString:rh withNum:0]]; + case 1:return [itemLoop isEqualToString:[c getParamExpString:rh withNum:0]]; + case 2:return [self state].request("hasItem",{utf8([c getParamExpString:rh withNum:0]),utf8([c getParamExpString:rh withNum:1])}).get(); + case 3:return [self state].request("hasSection",{utf8([c getParamExpString:rh withNum:0])}).get(); + }}catch(const std::exception& e){NSLog(@"Fini condition: %s",e.what());}return NO; +} +-(BOOL)valuesAllowed:(CObject*)o {return o&&o->rov&&(o->hoType>=32||o->hoType==2||((o->hoType==3||o->hoType==7)&&cf25p&&allowRVforCS));} +-(int)valueCount:(CObject*)o {return cf25p?o->rov->rvNumberOfValues:std::min(26,o->rov->rvNumberOfValues);} +-(int)stringCount:(CObject*)o {return cf25p?o->rov->rvNumberOfStrings:std::min(10,o->rov->rvNumberOfStrings);} +-(std::string)packValues:(CObject*)o { + if(![self valuesAllowed:o])return ""; + Json data={{"flags",static_cast(o->rov->rvValueFlags)},{"values",Json::array()},{"strings",Json::array()}}; + for(int i=0;i<[self valueCount:o];++i){CValue* v=[o->rov getValue:i];if([v getType]==TYPE_INT)data["values"].push_back({{"type",0},{"value",[v getInt]}});else data["values"].push_back({{"type",2},{"value",[v getDouble]}});} + for(int i=0;i<[self stringCount:o];++i)data["strings"].push_back(utf8([o->rov getString:i])); + return fini::packValues(data); +} +-(void)unpackValues:(CObject*)o text:(const std::string&)text { + if(![self valuesAllowed:o])return; + auto data=fini::unpackValues(text,[self valueCount:o],[self stringCount:o]); + o->rov->rvValueFlags=data["flags"].get(); + for(int i=0;i<[self valueCount:o];++i){auto v=data["values"][i];CValue* dst=[o->rov getValue:i];if(v["type"]==0)[dst forceInt:v["value"].get()];else [dst forceDouble:v["value"].get()];} + for(int i=0;i<[self stringCount:o];++i)[o->rov setString:i withString:ns(data["strings"][i].get())]; +} +-(void)action:(int)n withActExtension:(CActExtension*)a { + if(!finiState)return; + auto s=[&](int i){return utf8([a getParamExpString:rh withNum:i]);}; + auto f=[&](int i){return [a getParamFilename2:rh withNum:i];}; + auto integer=[&](int i){return [a getParamExpression:rh withNum:i];}; + try {auto& state=[self state];switch(n){ + case 0:state.reset();break;case 1:state.release();break; + case 2:{NSString* file=f(0);auto key=s(1);state.flush();state.request("fileData",{[self readBytes:file],key,[self writePath:file]});break;} + case 3:{NSString* file=f(0);auto key=s(1);state.request("fileSave",{[self writePath:file],key});break;} + case 4:{auto sec=s(0),item=s(1);float value=[a getParamExpDouble:rh withNum:2];state.request("setNumber",{sec,item,value});break;} + case 5:state.request("set",{s(0),s(1),s(2)});break; + case 6:state.request("delete",{s(0),s(1)});break; + case 7:{auto loop=s(0);auto names=state.names();for(const auto& name:names){if(!finiState)break;replace(sectionLoop,ns(loop));replace(currentSection,ns(name.get()));[ho generateEvent:0 withParam:0];}break;} + case 8:{auto sec=s(0),loop=s(1);if(!sec.empty())replace(currentSection,ns(sec));sec=utf8(currentSection);auto names=state.names(&sec);for(const auto& name:names){if(!finiState)break;replace(currentSection,ns(sec));replace(itemLoop,ns(loop));replace(currentItem,ns(name.get()));[ho generateEvent:1 withParam:0];}break;} + case 9:state.load(s(0));break; + case 10:{NSString* file=f(0);state.request("autoOn",{[self writePath:file],s(1)});break;} + case 11:state.autoSave=false;break; + case 12:state.request("copy",{s(0),s(1),integer(2)!=0});break; + case 13:{int fixed=integer(0);auto text=s(1);[self unpackValues:[self object:fixed] text:text];break;} + case 14:{int fixed=integer(0);auto text=s(1);CObject* o=[self object:fixed];if(o){auto xy=state.request("unpackPosition",{text});o->hoX=xy[0];o->hoY=xy[1];if(o->roc)o->roc->rcChanged=YES;}break;} + case 15:case 19:{auto data=fini::unbase64(s(0));state.load(n==15?fini::decompress(data):data);break;} + case 16:{linked=NO;NSString* file=f(0);auto key=s(1);if([file length])state.request("localBytes",{[self readBytes:file],key});else state.localizeFrom("");break;} + case 17:state.language=s(0);break; + case 18:localizationFixed=integer(0);linked=YES;[self updateLocalization];break; + }}catch(const std::exception& e){NSLog(@"Fini action %d: %s",n,e.what());} +} +-(CValue*)expression:(int)n { + if(!finiState)return n==0?[rh getTempValue:0]:[rh getTempString:@""]; + try {auto& state=[self state];switch(n){ + case 0:{auto sec=utf8([[ho getExpParam] getString]),key=utf8([[ho getExpParam] getString]);return [rh getTempDouble:state.request("number",{sec,key}).get()];} + case 1:{auto sec=utf8([[ho getExpParam] getString]),key=utf8([[ho getExpParam] getString]);[self updateLocalization];return [rh getTempString:ns(state.get(sec,key))];} + case 2:return [rh getTempString:currentSection];case 3:return [rh getTempString:currentItem];case 4:return [rh getTempString:ns(state.save())]; + case 5:return [rh getTempString:ns([self packValues:[self object:[[ho getExpParam] getInt]]])]; + case 6:{CObject* o=[self object:[[ho getExpParam] getInt]];return [rh getTempString:o?ns(state.request("packPosition",{o->hoX,o->hoY}).get()):@""];} + case 7:case 8:return [rh getTempString:ns(fini::base64(n==7?fini::compress(state.save()):state.save()))]; + case 9:case 10:{auto text=utf8([[ho getExpParam] getString]);BOOL compressed=[[ho getExpParam] getInt]!=0;return [rh getTempString:ns(state.request(n==9?"encode":"decode",{text,bool(compressed)}).get())];} + }}catch(const std::exception& e){NSLog(@"Fini expression %d: %s",n,e.what());} + return n==0?[rh getTempValue:0]:[rh getTempString:@""]; +} +@end diff --git a/Extensions/iOS/Fini/build.py b/Extensions/iOS/Fini/build.py new file mode 100644 index 000000000..fbcbe60a3 --- /dev/null +++ b/Extensions/iOS/Fini/build.py @@ -0,0 +1,54 @@ +#!/usr/bin/env python3 +"""Build owned Objective-C++ adapter + reused C++ core as an iOS XCFramework. +Requires macOS, Xcode, a licensed exporter Runtime, and target OpenSSL prefixes. +Does not redistribute or modify the exporter SDK. iOS has no JNI. +""" +import argparse,subprocess,shutil,os,sys +from pathlib import Path + +def main(): + p=argparse.ArgumentParser(description=__doc__) + p.add_argument('--runtime',type=Path,required=True,help='Exporter project root containing Classes/') + p.add_argument('--openssl',type=Path,required=True,help='Prefixes: iphoneos-arm64, iphonesimulator-arm64, iphonesimulator-x86_64') + p.add_argument('--encryption-framework',type=Path,help='Use an already built shared OFEEncryption.xcframework') + p.add_argument('--prepare-openssl',action='store_true',help='Build missing iOS OpenSSL slices from pinned source') + p.add_argument('--output',type=Path,default=Path('work/fini-ios'));a=p.parse_args() + if not shutil.which('xcrun') or not shutil.which('xcodebuild'):raise SystemExit('Fini iOS requires macOS with Xcode/xcrun; no iOS binary was produced.') + root=Path(__file__).resolve().parent;extensions=root.parents[1];classes=a.runtime.resolve()/'Classes' + if not (classes/'Extensions/CRunExtension.h').is_file():raise SystemExit('Missing licensed exporter Runtime headers') + # Shared encryption is built once, outside the plugin archive. + common_output=extensions.parent/'work/mobile-encryption/ios' + if a.encryption_framework: + if not (a.encryption_framework/'Info.plist').is_file():raise SystemExit('Invalid shared encryption XCFramework') + else: + command=[sys.executable,extensions/'iOS/Encryption/build.py','--openssl',a.openssl,'--output',common_output] + if a.prepare_openssl:command.append('--prepare-openssl') + subprocess.run(list(map(str,command)),check=True) + out=a.output.resolve();out.mkdir(parents=True,exist_ok=True) + includes=[f'-I{classes}']+[f'-I{d}' for d in classes.iterdir() if d.is_dir()]+[f'-I{extensions}/_DeLib'] + source=root/'Runtime/CRunFini.mm' + headers=out/'Headers';headers.mkdir(exist_ok=True);shutil.copy2(source.with_suffix('.h'),headers/'CRunFini.h') + with (out/'publisher.log').open('w') as log: + def run(cmd): + print('+',' '.join(map(str,cmd)),flush=True);subprocess.run(list(map(str,cmd)),stdout=log,stderr=subprocess.STDOUT,check=True) + slices={} + for sdk,arch in [('iphoneos','arm64'),('iphonesimulator','arm64'),('iphonesimulator','x86_64')]: + directory=out/f'{sdk}-{arch}';directory.mkdir(exist_ok=True) + sysroot=subprocess.check_output(['xcrun','--sdk',sdk,'--show-sdk-path'],text=True).strip() + target=f'{arch}-apple-ios17.0'+('-simulator' if sdk=='iphonesimulator' else '') + run(['xcrun','--sdk',sdk,'clang++','-std=c++20','-DFINI_PORTABLE_RUNTIME','-O2','-fno-objc-arc','-fobjc-exceptions','-fexceptions','-isysroot',sysroot,'-target',target,*includes,'-c',source,'-o',directory/'Fini.o']) + run(['xcrun','libtool','-static','-o',directory/'libFini.a',directory/'Fini.o']) + slices[(sdk,arch)]=directory/'libFini.a' + simulator=out/'libFini-simulator.a' + run(['xcrun','lipo','-create',slices[('iphonesimulator','arm64')],slices[('iphonesimulator','x86_64')],'-output',simulator]) + final=out/'Fini.xcframework';pending=out/'pending.xcframework' + if pending.exists():shutil.rmtree(pending) + run(['xcodebuild','-create-xcframework','-library',slices[('iphoneos','arm64')],'-headers',headers,'-library',simulator,'-headers',headers,'-output',pending]) + if final.exists():shutil.rmtree(final) + pending.rename(final) + sys.path.insert(0,str(extensions/'iOS/Tools')) + from apple_artifacts import stage + sources=['Extensions/iOS/Fini/Runtime/CRunFini.mm','Extensions/iOS/Fini/Runtime/CRunFini.h','Extensions/Fini/IniCore.h','Extensions/iOS/Fini/build.py','Extensions/_DeLib/Encryption/CAPI.h','Extensions/_DeLib/StrNum.h','Extensions/_DeLib/UnicodeConversion.h','Extensions/_3rdLib/Base64.h','Extensions/_3rdLib/Compress/zlibInterface.h','Extensions/_3rdLib/simpleini/SimpleIni.h','Extensions/_3rdLib/nlohmann/single_include/nlohmann/json.hpp','Extensions/_DeLib/StringUtilities.h','Extensions/_DeLib/StringTraits.h','Extensions/_DeLib/ArithmeticTraits.h','Extensions/_DeLib/PathUtilities.h','Extensions/_DeLib/GeneralMacros.h','Extensions/_3rdLib/Compress/zlib/zlib.h','Extensions/_3rdLib/Compress/zlib/zconf.h'] + stage(final,'Fini',sources,root/'Prebuilt',requires=['OFEEncryption/1']) + print('Built',final,'Link the common OFEEncryption.xcframework once, plus libc++, Foundation and libz. No crypto implementation is included in Fini.') +if __name__=='__main__':main() diff --git a/Extensions/iOS/Fini/configure_export.py b/Extensions/iOS/Fini/configure_export.py new file mode 100644 index 000000000..5f6e44da2 --- /dev/null +++ b/Extensions/iOS/Fini/configure_export.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python3 +"""Idempotently link Fini and embed one common encryption framework in an iOS export.""" +import argparse +import difflib +import hashlib +from pathlib import Path +import plistlib +import shutil +import sys +import xml.etree.ElementTree as ET + +# Exported packages carry this generic parser next to the configurator. +try: + from xcode_project import Project,render,words,version,payload_digest +except ImportError: + sys.path.insert(0,str(Path(__file__).resolve().parents[1]/'Tools')) + from xcode_project import Project,render,words,version,payload_digest + + +def configure(bundle,common=None,target=None,minimum='17.0',dry_run=False): + bundle=Path(bundle).resolve() + if bundle.suffix!='.xcodeproj': + found=list(bundle.glob('*.xcodeproj')) + if len(found)!=1:raise ValueError('Specify one exported .xcodeproj') + bundle=found[0] + root=bundle.parent;path=bundle/'project.pbxproj';text=path.read_text(encoding='utf-8-sig');project=Project(text) + objects={key:project.plain(key) for key in project.objects} + apps=[(key,obj) for key,obj in objects.items() if obj.get('isa')=='PBXNativeTarget' and obj.get('productType')=='com.apple.product-type.application' and (not target or obj.get('name')==target)] + if len(apps)!=1:raise ValueError('Expected one App target; use --target') + app_id,app=apps[0] + for sdk in ('iphoneos','iphonesimulator'): + native=root/f'libFini_{sdk}.a' + if not native.is_file() or (native.read_bytes()[:8]!=b'!\n' and native.read_bytes()[:4] not in (b'\xca\xfe\xba\xbe',b'\xbe\xba\xfe\xca',b'\xca\xfe\xba\xbf',b'\xbf\xba\xfe\xca')):raise ValueError('Missing/invalid exported archive: '+str(native)) + destination=root/'Frameworks/OFEEncryption.xcframework';source=Path(common).resolve() if common else destination + if not (source/'Info.plist').is_file():raise ValueError('Supply the separately built common runtime with --common /path/to/OFEEncryption.xcframework') + info=plistlib.loads((source/'Info.plist').read_bytes()) + variants={entry.get('SupportedPlatformVariant','device') for entry in info.get('AvailableLibraries',[]) if entry.get('SupportedPlatform')=='ios' and 'arm64' in entry.get('SupportedArchitectures',[])} + if not {'device','simulator'}.issubset(variants):raise ValueError('Shared framework requires arm64 device/simulator slices') + changes={};additions={} + def current(key):return changes.get(key,objects[key]) + def stable(name):return hashlib.sha256(('Fini export '+app_id+' '+name).encode()).hexdigest()[:24].upper() + def add(name,obj): + key=stable(name) + if key in objects and objects[key]!=obj:raise ValueError('Project identifier collision: '+key) + if key not in objects:additions[key]=obj + return key + refs=[key for key,obj in objects.items() if obj.get('isa')=='PBXFileReference' and obj.get('path')=='Frameworks/OFEEncryption.xcframework'] + ref=refs[0] if refs else add('shared-ref',dict(isa='PBXFileReference',lastKnownFileType='wrapper.xcframework',path='Frameworks/OFEEncryption.xcframework',sourceTree='SOURCE_ROOT')) + framework_phase=next((phase for phase in app['buildPhases'] if objects[phase]['isa']=='PBXFrameworksBuildPhase'),None) + if not framework_phase:raise ValueError('App has no Frameworks build phase') + if not any(objects[file].get('fileRef')==ref for file in objects[framework_phase].get('files',[])): + build=add('shared-link',dict(isa='PBXBuildFile',fileRef=ref));obj=dict(objects[framework_phase]);obj['files']=[*obj.get('files',[]),build];changes[framework_phase]=obj + if not any(objects.get(file,{}).get('fileRef')==ref for phase in app['buildPhases'] if objects[phase]['isa']=='PBXCopyFilesBuildPhase' for file in objects[phase].get('files',[])): + build=add('shared-embed',dict(isa='PBXBuildFile',fileRef=ref,settings={'ATTRIBUTES':['CodeSignOnCopy','RemoveHeadersOnCopy']})) + phase=add('embed-phase',dict(isa='PBXCopyFilesBuildPhase',buildActionMask='2147483647',dstPath='',dstSubfolderSpec='10',files=[build],name='Embed shared OFEEncryption',runOnlyForDeploymentPostprocessing='0')) + app=dict(app);app['buildPhases']=[*app['buildPhases'],phase];changes[app_id]=app + main=objects[project.root.value['rootObject'].value]['mainGroup'] + if not refs: + obj=dict(current(main));obj['children']=[*obj.get('children',[]),ref];changes[main]=obj + # Static SDK libraries must use required linkage and explicit build dependencies. + products={obj.get('productReference'):key for key,obj in objects.items() if obj.get('isa')=='PBXNativeTarget'} + app=dict(current(app_id));dependencies=list(app.get('dependencies',[])) + known={objects[dep].get('target') for dep in dependencies} + for build_id in objects[framework_phase].get('files',[]): + build=objects[build_id];file=objects[build['fileRef']] + if not file.get('path','').endswith('.a'):continue + if 'Weak' in build.get('settings',{}).get('ATTRIBUTES',[]): + replacement=dict(build);settings=dict(build['settings']);settings['ATTRIBUTES']=[value for value in settings['ATTRIBUTES'] if value!='Weak'];replacement['settings']=settings;changes[build_id]=replacement + dependency=products.get(build['fileRef']) + if dependency and dependency not in known: + proxy=add('proxy-'+dependency,dict(isa='PBXContainerItemProxy',containerPortal=project.root.value['rootObject'].value,proxyType='1',remoteGlobalIDString=dependency,remoteInfo=objects[dependency]['name'])) + item=add('dependency-'+dependency,dict(isa='PBXTargetDependency',target=dependency,targetProxy=proxy));dependencies.append(item);known.add(dependency) + if dependencies!=app.get('dependencies',[]):app['dependencies']=dependencies;changes[app_id]=app + configs=list(objects[app['buildConfigurationList']]['buildConfigurations']) + app_configs=set(configs) + linked={objects[file].get('fileRef') for file in objects[framework_phase].get('files',[])} + for obj in objects.values(): + if obj.get('isa')=='PBXNativeTarget' and obj.get('productType')=='com.apple.product-type.library.static' and obj.get('productReference') in linked: + configs.extend(objects[obj['buildConfigurationList']]['buildConfigurations']) + project_configs=objects[objects[project.root.value['rootObject'].value]['buildConfigurationList']]['buildConfigurations'] + floor=max([minimum]+[objects[key]['buildSettings'].get('IPHONEOS_DEPLOYMENT_TARGET',minimum) for key in set(configs+project_configs)],key=version) + for key in configs: + obj=dict(objects[key]);settings=dict(obj['buildSettings']) + for sdk in ('iphoneos','iphonesimulator'): + name=f'OTHER_LDFLAGS[sdk={sdk}*]';flags=words(settings.get(name));base=words(settings.get('OTHER_LDFLAGS')) + if base and not any(flags[index:index+len(base)]==base for index in range(len(flags)-len(base)+1)):flags=base+flags + for flag in ('$(inherited)',f'-lFini_{sdk}','-ObjC','-lc++','-lz'): + if flag not in flags:flags.append(flag) + settings[name]=flags + search=words(settings.get('LIBRARY_SEARCH_PATHS')) + for item in ('$(inherited)','$(SRCROOT)'): + if item not in search:search.append(item) + settings['LIBRARY_SEARCH_PATHS']=search + for name in list(settings): + if name.split('[',1)[0] in ('ARCHS','VALID_ARCHS','EXCLUDED_ARCHS','ONLY_ACTIVE_ARCH'):del settings[name] + settings['ARCHS']='arm64';settings['ONLY_ACTIVE_ARCH']='NO';settings['EXCLUDED_ARCHS']='' + settings['CODE_SIGNING_ALLOWED[sdk=iphonesimulator*]']='NO' + settings['IPHONEOS_DEPLOYMENT_TARGET']=floor + obj['buildSettings']=settings + if obj!=objects[key]:changes[key]=obj + edits=[] + for key,obj in changes.items(): + node=project.objects[key];edits.append((node.start,node.end,render(obj,2))) + if additions: + at=project.root.value['objects'].end-1 + edits.append((at,at,''.join(key+' = '+render(obj,2)+';\n' for key,obj in additions.items()))) + changed=text + for start,end,replacement in sorted(edits,reverse=True):changed=changed[:start]+replacement+changed[end:] + Project(changed) + updates={path:changed} if changed!=text else {} + native={key:obj for key,obj in objects.items() if obj.get('isa')=='PBXNativeTarget'} + for scheme in bundle.glob('xcshareddata/xcschemes/*.xcscheme'): + old=scheme.read_text();xml=ET.fromstring(old);dirty=False + for node in xml.iter('BuildableReference'): + obj=native.get(node.get('BlueprintIdentifier')) + if not obj:continue + desired={'ReferencedContainer':'container:'+bundle.name,'BlueprintName':obj['name'],'BuildableName':objects[obj['productReference']]['path']} + for key,value in desired.items(): + if node.get(key)!=value:node.set(key,value);dirty=True + if dirty:updates[scheme]=ET.tostring(xml,encoding='unicode')+'\n' + if dry_run: + for file,data in updates.items():print(''.join(difflib.unified_diff(file.read_text().splitlines(True),data.splitlines(True),fromfile=str(file),tofile=str(file)))) + else: + if source!=destination: + if destination.exists(): + left={p.relative_to(source).as_posix():payload_digest(p) for p in source.rglob('*') if p.is_file()} + right={p.relative_to(destination).as_posix():payload_digest(p) for p in destination.rglob('*') if p.is_file()} + if left!=right: + changed=sorted(name for name in set(left)|set(right) if left.get(name)!=right.get(name)) + raise ValueError('Refusing to overwrite a different installed common framework: '+', '.join(changed)) + else: + destination.parent.mkdir(parents=True,exist_ok=True);shutil.copytree(source,destination) + for file,data in updates.items():file.write_text(data) + return len(updates) + +if __name__=='__main__': + parser=argparse.ArgumentParser(description=__doc__);parser.add_argument('project',type=Path);parser.add_argument('--common',type=Path);parser.add_argument('--target');parser.add_argument('--dry-run',action='store_true');parser.add_argument('--minimum-ios',default='17.0') + args=parser.parse_args();print('Updated project files:',configure(args.project,args.common,args.target,args.minimum_ios,args.dry_run)) diff --git a/Extensions/iOS/Fini/generate_project.py b/Extensions/iOS/Fini/generate_project.py new file mode 100644 index 000000000..19795f35f --- /dev/null +++ b/Extensions/iOS/Fini/generate_project.py @@ -0,0 +1,104 @@ +#!/usr/bin/env python3 +"""Generate Fini's owned Xcode project on any host; configure local SDKs on a Mac. +No exporter SDK source is copied into the project or distribution. +""" +import argparse +import hashlib +import json +from pathlib import Path +import xml.etree.ElementTree as ET + +ROOT=Path(__file__).resolve().parent + +def uid(name): + return hashlib.sha1(('Fini.iOS.'+name).encode()).hexdigest()[:24].upper() + + +def build_model(): + objects={} + def add(key,isa,**fields): + objects[uid(key)]={'isa':isa,**fields};return uid(key) + source=add('source','PBXFileReference',lastKnownFileType='sourcecode.cpp.objcpp',path='Runtime/CRunFini.mm',sourceTree='SOURCE_ROOT') + header=add('header','PBXFileReference',lastKnownFileType='sourcecode.c.h',path='Runtime/CRunFini.h',sourceTree='SOURCE_ROOT') + config=add('config','PBXFileReference',lastKnownFileType='text.xcconfig',path='Config/Fini.xcconfig',sourceTree='SOURCE_ROOT') + product=add('product','PBXFileReference',explicitFileType='archive.ar',path='libFini.a',sourceTree='BUILT_PRODUCTS_DIR',includeInIndex='0') + build_file=add('source.build','PBXBuildFile',fileRef=source) + sources=add('sources','PBXSourcesBuildPhase',buildActionMask='2147483647',files=[build_file],runOnlyForDeploymentPostprocessing='0') + frameworks=add('frameworks','PBXFrameworksBuildPhase',buildActionMask='2147483647',files=[],runOnlyForDeploymentPostprocessing='0') + packaging=add('package','PBXShellScriptBuildPhase',buildActionMask='2147483647',files=[],inputPaths=[],outputPaths=[],alwaysOutOfDate='1',runOnlyForDeploymentPostprocessing='0',shellPath='/bin/sh',shellScript='set -eu\npython3 "$SRCROOT/build.py" --runtime "$FINI_RUNTIME_ROOT" --openssl "$FINI_OPENSSL_BASE" --output "$FINI_OUTPUT_ROOT" --prepare-openssl\n') + configurations={} + for target in ('project','Fini','FiniXCFramework'): + configs=[] + for name in ('Debug','Release'): + settings={'GCC_OPTIMIZATION_LEVEL':'0' if name=='Debug' else 's','DEBUG_INFORMATION_FORMAT':'dwarf' if name=='Debug' else 'dwarf-with-dsym'} + if target=='Fini': + settings.update({'PRODUCT_NAME':'Fini','SKIP_INSTALL':'NO','PUBLIC_HEADERS_FOLDER_PATH':'include/Fini'}) + configs.append(add(target+'.'+name,'XCBuildConfiguration',baseConfigurationReference=config,buildSettings=settings,name=name)) + configurations[target]=add(target+'.configs','XCConfigurationList',buildConfigurations=configs,defaultConfigurationIsVisible='0',defaultConfigurationName='Release') + native=add('Fini','PBXNativeTarget',buildConfigurationList=configurations['Fini'],buildPhases=[sources,frameworks],buildRules=[],dependencies=[],name='Fini',productName='Fini',productReference=product,productType='com.apple.product-type.library.static') + aggregate=add('FiniXCFramework','PBXAggregateTarget',buildConfigurationList=configurations['FiniXCFramework'],buildPhases=[packaging],dependencies=[],name='FiniXCFramework',productName='FiniXCFramework') + products=add('products','PBXGroup',children=[product],name='Products',sourceTree='') + files=add('files','PBXGroup',children=[source,header,config],name='Fini sources and configuration',sourceTree='') + main=add('main','PBXGroup',children=[files,products],sourceTree='') + project=add('project','PBXProject',attributes={'BuildIndependentTargetsInParallel':'YES','LastUpgradeCheck':'1500'},buildConfigurationList=configurations['project'],compatibilityVersion='Xcode 14.0',developmentRegion='en',hasScannedForEncodings='0',knownRegions=['en','Base'],mainGroup=main,productRefGroup=products,projectDirPath='',projectRoot='',targets=[native,aggregate]) + return {'archiveVersion':'1','classes':{},'objectVersion':'56','objects':objects,'rootObject':project} + + +def openstep(value,level=0): + indent='\t'*level + if isinstance(value,dict): + return '{\n'+''.join(indent+'\t'+json.dumps(k)+' = '+openstep(v,level+1)+';\n' for k,v in value.items())+indent+'}' + if isinstance(value,list): + return '(\n'+''.join(indent+'\t'+openstep(v,level+1)+',\n' for v in value)+indent+')' + return json.dumps(str(value),ensure_ascii=False) + + +def scheme(target,product): + root=ET.Element('Scheme',LastUpgradeVersion='1500',version='1.3') + action=ET.SubElement(root,'BuildAction',parallelizeBuildables='YES',buildImplicitDependencies='YES') + entries=ET.SubElement(action,'BuildActionEntries') + entry=ET.SubElement(entries,'BuildActionEntry',buildForTesting='NO',buildForRunning='YES',buildForProfiling='NO',buildForArchiving='YES',buildForAnalyzing='YES') + ET.SubElement(entry,'BuildableReference',BuildableIdentifier='primary',BlueprintIdentifier=uid(target),BuildableName=product,BlueprintName=target,ReferencedContainer='container:Fini.xcodeproj') + ET.SubElement(root,'AnalyzeAction',buildConfiguration='Debug') + ET.SubElement(root,'ArchiveAction',buildConfiguration='Release',revealArchiveInOrganizer='NO') + ET.indent(root) + return ET.tostring(root,encoding='unicode',xml_declaration=True)+'\n' + + +def generate(output=ROOT): + project=output/'Fini.xcodeproj';schemes=project/'xcshareddata/xcschemes';schemes.mkdir(parents=True,exist_ok=True) + (project/'project.pbxproj').write_text('// !$*UTF8*$!\n'+openstep(build_model())+'\n') + for name,product in [('Fini','libFini.a'),('FiniXCFramework','FiniXCFramework')]: + (schemes/(name+'.xcscheme')).write_text(scheme(name,product)) + workspace=project/'project.xcworkspace';workspace.mkdir(exist_ok=True) + (workspace/'contents.xcworkspacedata').write_text('\n\n') + return project + + +def xcpath(value): + # Prevent comments, build-setting expansion or newline injection in xcconfig paths. + text=str(value.resolve()) + if any(c in text for c in ('\n','\r','$','"')) or '//' in text: + raise ValueError('Unsupported character in local SDK path: '+text) + return '"'+text+'"' + + +def main(): + parser=argparse.ArgumentParser(description=__doc__) + parser.add_argument('--runtime',type=Path,help='Licensed exported project containing Classes/') + parser.add_argument('--openssl',type=Path,help='OpenSSL prefixes for all three iOS slices') + args=parser.parse_args() + local=[] + if args.runtime: + if not (args.runtime/'Classes/Extensions/CRunExtension.h').is_file(): + parser.error('--runtime must contain Classes/Extensions/CRunExtension.h') + local.append('FINI_RUNTIME_ROOT = '+xcpath(args.runtime)) + if args.openssl: + local.append('FINI_OPENSSL_BASE = '+xcpath(args.openssl)) + if local: + (ROOT/'Config/Local.xcconfig').write_text('// Machine-local SDK locations. Do not commit.\n'+'\n'.join(local)+'\n') + print('Xcode project ready:',generate()) + print('Open Fini.xcodeproj, choose FiniXCFramework, then Product > Build.') + +if __name__=='__main__': + main() diff --git a/Extensions/iOS/Fini/tests/test_ext_publisher.py b/Extensions/iOS/Fini/tests/test_ext_publisher.py new file mode 100644 index 000000000..07775855e --- /dev/null +++ b/Extensions/iOS/Fini/tests/test_ext_publisher.py @@ -0,0 +1,99 @@ +"""Owned native/archive fixtures; validates packaging, never an Apple build.""" +import importlib.util +import contextlib +import io +from pathlib import Path +import plistlib +import sys +import tempfile +import unittest +import zipfile +from unittest.mock import patch + +repo=Path(__file__).resolve().parents[4] +sys.path.insert(0,str(repo/'Extensions/_Script')) +import FiniIOS +import Fini as publisher +spec=importlib.util.spec_from_file_location('fini_configure',repo/'Extensions/iOS/Fini/configure_export.py') +configure=importlib.util.module_from_spec(spec);spec.loader.exec_module(configure) +sys.path.insert(0,str(repo/'Extensions/iOS/Fini')) +import generate_project +from xcode_project import Project,render + + +def artifact(second_header=b'// interface\r\n'): + output=io.BytesIO() + libs=[dict(LibraryIdentifier='ios-arm64',LibraryPath='libFini.a',HeadersPath='Headers',SupportedPlatform='ios',SupportedArchitectures=['arm64']),dict(LibraryIdentifier='ios-arm64-simulator',LibraryPath='libFini.a',HeadersPath='Headers',SupportedPlatform='ios',SupportedPlatformVariant='simulator',SupportedArchitectures=['arm64'])] + with zipfile.ZipFile(output,'w') as zipped: + zipped.writestr('Fini.xcframework/Info.plist',plistlib.dumps({'AvailableLibraries':libs})) + for index,lib in enumerate(libs): + prefix='Fini.xcframework/'+lib['LibraryIdentifier']+'/' + zipped.writestr(prefix+'libFini.a',b'!\nowned binary fixture\r\n') + zipped.writestr(prefix+'Headers/CRunFini.h',second_header if index else b'// interface\n') + return {'owner':'Fini'},output.getvalue() + + +class ExportTests(unittest.TestCase): + def test_real_ext_framing_and_current_configurator_are_packaged(self): + with tempfile.TemporaryDirectory() as temp: + path=Path(temp)/'Fini.ext';FiniIOS.write(artifact(),path) + contents=FiniIOS.codec().read_extension(path.read_bytes()) + self.assertIn('Classes/Extensions/CRunFini.m',contents) + self.assertIn('libFini_iphoneos.a',contents);self.assertIn('libFini_iphonesimulator.a',contents) + self.assertIn('configure_fini_export.py',contents);self.assertIn('xcode_project.py',contents) + self.assertEqual(len(contents['project.txt'].split(b'\0')),7) + self.assertFalse(any('OFEEncryption.framework' in key for key in contents)) + self.assertIn(b'.linker_option \\"-lFini_iphoneos\\"',contents['Classes/Extensions/CRunFini.m']) + def test_changed_header_does_not_replace_previous_ext(self): + with tempfile.TemporaryDirectory() as temp: + output=Path(temp)/'Fini.ext';output.write_bytes(b'previous') + with self.assertRaisesRegex(ValueError,'headers differ'):FiniIOS.write(artifact(b'// changed interface\n'),output) + self.assertEqual(output.read_bytes(),b'previous') + def test_actual_publisher_preserves_all_outputs_on_bad_native_headers(self): + with tempfile.TemporaryDirectory() as temp: + root=Path(temp);extension=root/'Extensions';config=extension/'iOS/Fini';config.mkdir(parents=True) + (config/'Config').mkdir();(config/'Config/FiniConsumer.xcconfig').write_text('// owned fixture') + (config/'README.md').write_text('owned fixture') + runtime=extension/'Fini/ToInstall/Files/Data/Runtime/iPhone';runtime.mkdir(parents=True) + previous=runtime/'Fini-iOS.zip';previous.write_bytes(b'known good client package') + common_output=extension/'Android_Gradle/cmake/Encryption/ToInstall';common_output.mkdir(parents=True) + previous_common=common_output/'OFEEncryption-iOS.zip';previous_common.write_bytes(b'known good common package') + bad=artifact(b'// different interface\n') + manifest=dict(bad[0],file='Fini.xcframework.zip',requires=['OFEEncryption/1']) + common=(dict(owner='OFEEncryption',file='OFEEncryption.xcframework.zip'),b'owned common fixture') + with patch.object(publisher,'EXT',extension),patch.object(publisher,'REPO',root),patch.object(publisher.apple_artifacts,'validate',side_effect=[(manifest,bad[1]),common]): + with self.assertRaisesRegex(ValueError,'headers differ'):publisher.sync_apple() + self.assertEqual(previous.read_bytes(),b'known good client package') + self.assertEqual(previous_common.read_bytes(),b'known good common package') + + def test_export_configuration_dry_run_preservation_and_idempotency(self): + with tempfile.TemporaryDirectory() as temp: + root=Path(temp);bundle=root/'Renamed.xcodeproj';bundle.mkdir() + model=generate_project.build_model();objects=model['objects'];app=objects[generate_project.uid('Fini')] + app.update(name='App',productType='com.apple.product-type.application') + objects[generate_project.uid('product')]['path']='App.app' + for key in objects[app['buildConfigurationList']]['buildConfigurations']: + objects[key]['buildSettings'].update(DEVELOPMENT_TEAM='keep-team',IPHONEOS_DEPLOYMENT_TARGET='18.0',OTHER_LDFLAGS=['-custom']) + project=bundle/'project.pbxproj';project.write_text(render(model));before=project.read_bytes() + for sdk in ('iphoneos','iphonesimulator'):(root/f'libFini_{sdk}.a').write_bytes(b'!\nfixture') + common=root/'Common.xcframework';common.mkdir() + (common/'Info.plist').write_bytes(plistlib.dumps({'AvailableLibraries':[dict(SupportedPlatform='ios',SupportedArchitectures=['arm64']),dict(SupportedPlatform='ios',SupportedPlatformVariant='simulator',SupportedArchitectures=['arm64'])]})) + with contextlib.redirect_stdout(io.StringIO()) as preview: + self.assertGreater(configure.configure(bundle,common,dry_run=True),0) + self.assertIn('OTHER_LDFLAGS',preview.getvalue()) + self.assertEqual(project.read_bytes(),before);self.assertFalse((root/'Frameworks').exists()) + self.assertGreater(configure.configure(bundle,common),0) + first=project.read_bytes();self.assertEqual(configure.configure(bundle,common),0);self.assertEqual(project.read_bytes(),first) + parsed=Project(first.decode());settings=[parsed.plain(key)['buildSettings'] for key in objects[app['buildConfigurationList']]['buildConfigurations']] + for value in settings: + self.assertEqual(value['DEVELOPMENT_TEAM'],'keep-team');self.assertEqual(value['IPHONEOS_DEPLOYMENT_TARGET'],'18.0') + self.assertIn('-custom',value['OTHER_LDFLAGS[sdk=iphoneos*]']) + self.assertEqual(sum(parsed.plain(key).get('isa')=='PBXCopyFilesBuildPhase' for key in parsed.objects),1) + header=common/'public.h';header.write_bytes(b'interface\n') + destination=root/'Frameworks/OFEEncryption.xcframework/public.h';destination.write_bytes(b'interface\r\n') + self.assertEqual(configure.configure(bundle,common),0) + (common/'modified').write_text('different') + with self.assertRaisesRegex(ValueError,'overwrite'):configure.configure(bundle,common) + self.assertEqual(project.read_bytes(),first) + +if __name__=='__main__':unittest.main() diff --git a/Extensions/iOS/Fini/tests/test_handoff.py b/Extensions/iOS/Fini/tests/test_handoff.py new file mode 100644 index 000000000..c4dbe1ac8 --- /dev/null +++ b/Extensions/iOS/Fini/tests/test_handoff.py @@ -0,0 +1,86 @@ +"""Mac/Windows handoff fixtures, not Apple compilation or exporter validation.""" +import importlib.util +import json +import subprocess +from pathlib import Path +import plistlib +import tempfile +import unittest +import zipfile + +repo=Path(__file__).resolve().parents[4] +spec=importlib.util.spec_from_file_location('apple_artifacts',repo/'Extensions/iOS/Tools/apple_artifacts.py') +apple=importlib.util.module_from_spec(spec);spec.loader.exec_module(apple) + +def framework(root,owner): + path=root/(owner+'.xcframework');path.mkdir() + libraries=[{'LibraryIdentifier':'ios-arm64','LibraryPath':'lib'+owner+'.a','SupportedPlatform':'ios','SupportedArchitectures':['arm64']},{'LibraryIdentifier':'ios-arm64_x86_64-simulator','LibraryPath':'lib'+owner+'.a','SupportedPlatform':'ios','SupportedPlatformVariant':'simulator','SupportedArchitectures':['arm64','x86_64']}] + (path/'Info.plist').write_bytes(plistlib.dumps({'AvailableLibraries':libraries})) + for lib in libraries: + folder=path/lib['LibraryIdentifier'];folder.mkdir();(folder/lib['LibraryPath']).write_bytes(b'fixture archive, not a compiled library') + return path + +class HandoffTest(unittest.TestCase): + def test_mac_stage_and_windows_package_without_xcode(self): + with tempfile.TemporaryDirectory() as temp: + root=Path(temp);(root/'source.cpp').write_text('source\n') + native=framework(root,'Fini');prebuilt=root/'Prebuilt' + apple.stage(native,'Fini',['source.cpp'],prebuilt,requires=['OFEEncryption/1'],repo=root) + validated=apple.validate(prebuilt,'Fini',root) + output=root/'Fini-iOS.zip';apple.package(validated,output) + with zipfile.ZipFile(output) as zipped: + self.assertEqual(set(zipped.namelist()),{'Fini.xcframework.zip','manifest.json'}) + self.assertEqual(json.loads(zipped.read('manifest.json'))['requires'],['OFEEncryption/1']) + (root/'source.cpp').write_bytes(b'source\r\n') + self.assertIsNotNone(apple.validate(prebuilt,'Fini',root)) + def test_stale_and_corrupt_artifacts_do_not_replace_output(self): + with tempfile.TemporaryDirectory() as temp: + root=Path(temp);source=root/'source.cpp';source.write_text('version1') + prebuilt=root/'Prebuilt';apple.stage(framework(root,'Fini'),'Fini',['source.cpp'],prebuilt,repo=root) + output=root/'existing.zip';output.write_bytes(b'known good') + source.write_text('version2') + with self.assertRaisesRegex(ValueError,'stale'):apple.validate(prebuilt,'Fini',root) + self.assertEqual(output.read_bytes(),b'known good') + source.write_text('version1');(prebuilt/'Fini.xcframework.zip').write_bytes(b'bad') + with self.assertRaisesRegex(ValueError,'Corrupt'):apple.validate(prebuilt,'Fini',root) + def test_windows_git_conversion_and_binary_newlines(self): + header='Extensions/_DeLib/Encryption/CAPI.h' + converted=subprocess.check_output(['git','-c','core.autocrlf=true','cat-file','--filters','HEAD:'+header],cwd=repo) + original=subprocess.check_output(['git','show','HEAD:'+header],cwd=repo) + self.assertIn(b'\r\n',converted) + self.assertEqual(original.replace(b'\r\n',b'\n'),converted.replace(b'\r\n',b'\n')) + with tempfile.TemporaryDirectory() as temp: + root=Path(temp);(root/'public.h').write_bytes(original) + dependency=root/'library.a';dependency.write_bytes(b'!\n') + before=apple.source_digest(root,['library.a']) + dependency.write_bytes(b'!\r\n') + self.assertNotEqual(apple.source_digest(root,['library.a']),before) + native=framework(root,'Fini') + binary=native/'ios-arm64/libFini.a';binary.write_bytes(b'!\nfixture\n') + prebuilt=root/'Prebuilt';apple.stage(native,'Fini',['public.h'],prebuilt,repo=root) + (root/'public.h').write_bytes(converted) + self.assertIsNotNone(apple.validate(prebuilt,'Fini',root)) + path=prebuilt/'Fini.xcframework.zip' + with zipfile.ZipFile(path) as zipped:payload={name:zipped.read(name) for name in zipped.namelist()} + payload['Fini.xcframework/ios-arm64/libFini.a']=payload['Fini.xcframework/ios-arm64/libFini.a'].replace(b'\n',b'\r\n') + with zipfile.ZipFile(path,'w',zipfile.ZIP_DEFLATED) as zipped: + for name,data in payload.items():zipped.writestr(name,data) + with self.assertRaisesRegex(ValueError,'Corrupt'):apple.validate(prebuilt,'Fini',root) + + def test_missing_and_incomplete_handoff(self): + with tempfile.TemporaryDirectory() as temp: + root=Path(temp);self.assertIsNone(apple.validate(root,'Fini')) + (root/'manifest.json').write_text('{}') + with self.assertRaisesRegex(ValueError,'Incomplete'):apple.validate(root,'Fini') + def test_public_and_client_artifacts_are_separate(self): + with tempfile.TemporaryDirectory() as temp: + root=Path(temp);(root/'source.cpp').write_text('source') + for owner in ('Fini','OFEEncryption'): + apple.stage(framework(root,owner),owner,['source.cpp'],root/owner,repo=root) + output=root/(owner+'.zip');apple.package(apple.validate(root/owner,owner,root),output) + with zipfile.ZipFile(output) as zipped: + self.assertIn(owner+'.xcframework.zip',zipped.namelist()) + other='OFEEncryption' if owner=='Fini' else 'Fini' + self.assertNotIn(other+'.xcframework.zip',zipped.namelist()) + +if __name__=='__main__':unittest.main() diff --git a/Extensions/iOS/Fini/tests/test_project.py b/Extensions/iOS/Fini/tests/test_project.py new file mode 100644 index 000000000..ed9b4092d --- /dev/null +++ b/Extensions/iOS/Fini/tests/test_project.py @@ -0,0 +1,45 @@ +"""Project generation checks; never presented as an Apple compiler/link test.""" +import importlib.util +from pathlib import Path +import tempfile +import unittest +import xml.etree.ElementTree as ET + +root=Path(__file__).resolve().parents[1] +spec=importlib.util.spec_from_file_location('fini_xcode',root/'generate_project.py') +generator=importlib.util.module_from_spec(spec);spec.loader.exec_module(generator) + +class ProjectTest(unittest.TestCase): + def test_model_targets_paths_and_references(self): + model=generator.build_model();objects=model['objects'] + self.assertIn(model['rootObject'],objects) + references={'mainGroup','productRefGroup','buildConfigurationList','productReference','baseConfigurationReference','fileRef'} + reference_lists={'children','targets','buildPhases','buildConfigurations','files'} + for obj in objects.values(): + for key,value in obj.items(): + if key in references:self.assertIn(value,objects) + if key in reference_lists: + for ref in value:self.assertIn(ref,objects) + if obj['isa']=='PBXFileReference' and obj['sourceTree']=='SOURCE_ROOT': + self.assertTrue((root/obj['path']).is_file(),obj['path']) + native=objects[generator.uid('Fini')] + self.assertEqual(native['productType'],'com.apple.product-type.library.static') + self.assertEqual(objects[generator.uid('FiniXCFramework')]['isa'],'PBXAggregateTarget') + script=objects[generator.uid('package')]['shellScript'] + self.assertIn('"$SRCROOT/build.py"',script);self.assertTrue((root/'build.py').is_file()) + self.assertIn('FINI_OPENSSL_BASE',script) + def test_generation_is_deterministic_and_schemes_match(self): + with tempfile.TemporaryDirectory() as temp: + output=Path(temp);project=generator.generate(output) + first=(project/'project.pbxproj').read_bytes();generator.generate(output) + self.assertEqual(first,(project/'project.pbxproj').read_bytes()) + self.assertEqual(first,(root/'Fini.xcodeproj/project.pbxproj').read_bytes()) + for target in ('Fini','FiniXCFramework'): + scheme=ET.parse(project/f'xcshareddata/xcschemes/{target}.xcscheme') + ref=scheme.find('.//BuildableReference') + self.assertEqual(ref.attrib['BlueprintIdentifier'],generator.uid(target)) + self.assertEqual(ref.attrib['ReferencedContainer'],'container:Fini.xcodeproj') + def test_local_path_validation(self): + with self.assertRaises(ValueError):generator.xcpath(Path('/tmp/$(command)')) + +if __name__=='__main__':unittest.main() diff --git a/Extensions/iOS/Tools/apple_artifacts.py b/Extensions/iOS/Tools/apple_artifacts.py new file mode 100644 index 000000000..deea90eb7 --- /dev/null +++ b/Extensions/iOS/Tools/apple_artifacts.py @@ -0,0 +1,110 @@ +"""Owned Mac artifacts -> Git push/pull -> Windows packaging; no Apple tools on Windows.""" +import hashlib +import io +import json +import os +from pathlib import Path +import plistlib +import tempfile +import zipfile + +REPO=Path(__file__).resolve().parents[3] + +def source_bytes(path): + data=path.read_bytes() + if path.suffix in ('.h','.hpp','.c','.cpp','.m','.mm','.py','.txt','.json','.xcconfig'): + data=data.replace(b'\r\n',b'\n') + elif path.name=='Info.plist' and data.startswith(b'= len(self.tokens): + raise ValueError('Unexpected end of project') + token = self.tokens[self.index] + self.index += 1 + if expected is not None and token.group() != expected: + raise ValueError(f'Expected {expected}, found {token.group()}') + return token + + def parse(self): + token = self.take() + word = token.group() + if word == '{': + value = {} + while self.tokens[self.index].group() != '}': + key = self.parse().value + if not isinstance(key, str) or key in value: + raise ValueError(f'Invalid or duplicate project key: {key}') + self.take('=') + value[key] = self.parse() + self.take(';') + end = self.take('}').end() + elif word == '(': + value = [] + while self.tokens[self.index].group() != ')': + value.append(self.parse()) + if self.tokens[self.index].group() != ')': + self.take(',') + end = self.take(')').end() + else: + if word in ('}', ')', '=', ';', ','): + raise ValueError(f'Unexpected token: {word}') + value = re.sub(r'\\(.)', lambda m: {'n': '\n', 'r': '\r', 't': '\t'}.get(m[1], m[1]), word[1:-1]) if word.startswith('"') else word + end = token.end() + return Node(value, token.start(), end) + + def plain(self, identity): + return plain(self.objects[identity]) + + +def plain(node): + value = node.value + if isinstance(value, dict): + return {key: plain(child) for key, child in value.items()} + if isinstance(value, list): + return [plain(child) for child in value] + return value + + +def render(value, depth=0): + tab = '\t' * depth + if isinstance(value, dict): + return '{\n' + ''.join(tab + '\t' + json.dumps(k) + ' = ' + render(v, depth + 1) + ';\n' for k, v in value.items()) + tab + '}' + if isinstance(value, list): + return '(\n' + ''.join(tab + '\t' + render(v, depth + 1) + ',\n' for v in value) + tab + ')' + return json.dumps(value, ensure_ascii=False) + + +def words(value): + return list(value) if isinstance(value, list) else shlex.split(value or '') + + +def version(value): + if not re.fullmatch(r'\d+(?:\.\d+){0,2}', value): + raise ValueError(f'Invalid minimum iOS version: {value}') + return tuple(map(int, value.split('.'))) + (0,) * (3 - len(value.split('.'))) + + +def payload_digest(path): + """Normalize only declared artifact text; binary plists and libraries stay exact.""" + data=path.read_bytes() + if path.suffix in ('.h','.md') or (path.name=='Info.plist' and data.startswith(b'