[defisym]

Modifying existing stuff

Port FFMpeg and SDL_Mixer to Android through JNI while reusing the existing C++ playback/adaptor architecture and master's shared Encryption implementation. Clean up dependencies first, then review each shared-core problem separately; platform integration comes last.

The base remains master 36812170 (portable encryption, #91). Already-merged encryption commits are not repeated. Commit author/committer identities remain associated with defisym.

Review order

Author review checkpoint — 2026-11-04: Items 1–7, through 4685fd99 (Send First/read/error states), have been reviewed. Item 8, 6dab60eb (audio pipeline), remains unreviewed. Item 9, 1895bc01 (clocks/controls), is currently being discussed; it and the following items are not marked approved. Resume substantive review at the audio-pipeline commit. Passing automated tests does not change this checkpoint.

Order	Commit / intent	Scope
1	d8851eda — Cleanup	Use split shared helpers; drop the stale probe comment. No duplicate utility headers.
2	808a2bbd — Portable compilation	Standard exception/types, qualified Lock references and Windows include guards.
3	058e5d39 — Memory view and AVIO ownership	Borrow media data through MemView; manage AVIO buffers independently, use native probing, delegate path constructors and clean up failed initialization.
4	9788a907 — Software conversion	Size output from actual frames and report actual stride; preserve default BGRA/Windows alignment.
5	393def8c — Fractional seek and timestamps	Keep subsecond targets and source-relative best-effort frame timestamps. Cache the source origin once after stream discovery; read timelineOriginSeconds directly without a forwarding getter.
6	9761ce96 — PCM copy bounds	One-line mix-length fix plus a regression that fails before it and passes after it.
7	4685fd99 — Send First and read/error states	Preserve the original branches, queue-wait early return, packet argument and unref placement. Handle EOF, temporary reads and real errors distinctly; reset seek state.
8	6dab60eb — Audio pipeline	Keep receive-frame → submit-filter → collect-output order and early returns; append available PCM, drain EOF tails, resize storage and reset DSP/preroll on seek.
9	1895bc01 — Playback clocks and controls	Monotonic/submitted-audio clock, delayed starts, pause/tempo continuity and seek callback reentry.
10	ca3aa1eb — Shared seek/preview policy	Move Windows wrapper policy into the core; unclocked preview, still images, completion and frame callback reentry.
11	8d6b088c — Generic hardware lifecycle	Extend the existing adaptor contract for context ownership/selection; keep explicit overrides and software fallback.
12	077893c2 — Android integration — last	MediaCodec/GLES adaptors, JNI/Java, shared Mixer X, file/storage ownership, SDK deployment, packaging, crypto API compatibility, platform fixtures and docs.

The first eleven commits contain the C++ core and its focused tests/review notes. No Java, Android adaptor implementation or publishing hook appears in those commits. The final Android commit does not modify FFMpeg.h; its shared playback changes are entirely visible in the preceding problem-specific commits.

Cleanup and shared behavior

• FFMpegUtilities.h and SDLAudioDefinition.h are removed and never introduced anywhere in the rewritten history. FFMpeg directly includes the existing ArithmeticUtilities, StringUtilities and StringConversion helpers. SDLUtilities.h and its buffer constant remain exactly as on master; the unused FFMpeg alias disappears with the audio-pipeline fix.
• MemBuf is replaced by a non-owning MemView: it stores a const source pointer and cursor, without allocating/freeing media or IO buffers. FFMpeg separately allocates the AVIO working buffer and frees its current pointer after FFmpeg probing may replace it. Memory playback/seek continue using native avformat probing. The UTF-8 constructor delegates to the existing wide-path constructor through the shared conversion helper.
• Normal initialized players always have a polymorphic adaptor, including software playback. The destructor guard exists for delegated-constructor failure before the factory is reached (missing input, failed probe/stream discovery or codec initialization); it is not a nullable-adaptor playback policy.
• Keep the original Send First control flow, including bRespond, immediate QUEUE_WAITING return, pending-packet retry, pInputPacket passed to receive wrappers, and packet unref placement. The decoder function's only semantic change against the original is using its already computed nullable input pointer for the EOF send. Source EAGAIN defers the video tick, completion uses native AVERROR_EOF, and nonnegative queue waiting remains distinct from EIO/EPERM. The FFMpeg.h diff in this commit is reduced to 10 added / 4 removed lines.
• The decoder commit retains the original audio timestamp code and packet argument contract. Frame-derived audio timestamp handling stays in the following audio-pipeline commit, where its position before filter submission belongs.
• The audio-pipeline commit retains the original receive-first wrapper structure, KEEP_REF, filter-output loop and early codec/error returns. All available filtered blocks append to the existing audio_buf before returning; codec EOF closes the filter once and drains filter/resampler tails. No separate output queue or pull-first scheduler is introduced. First-frame timing remains anchored before filter submission because filters may buffer input before producing output.
• In the clocks commit, goto_videoPosition still locks the main context, flushes video/audio queues and the audio decoder, forwards to the target frame, then resets audio timing. Only the external display callback is deferred until after unlocking so synchronized controls can reenter. The image protections are retained: invalid frames do not invoke the copy callback, and single-image no-seek/no-revert policy is preserved in the following shared-policy commit.
• At review request, the added forwardFrame receive-EAGAIN av_packet_unref is withdrawn. The existing seek-loop packet-release behavior is retained. Its send-EAGAIN retry issue is not claimed resolved by the separate queued-playback decoder fix.
• Android conversion/build packaging uses the existing DeLib::Helpers module and FTW's pinned iconv dependency, including matching source/notices. There is no second string-conversion implementation or separately selected iconv version.
• Decoder/filter/resampler drain, PCM bounds, seek timestamps, software buffer sizing, clocks and hardware fallback are shared behavior changes affecting Windows too. They are isolated for review rather than described as Android-only compile fixes. Default BGRA, Windows row alignment, state-only pause and the existing video-only fluent-loop policy are retained. Converted frame storage is named pConvertedFrameBuffer to cover the requested pixel formats; the rename is folded into the software-conversion commit.
• Shared regressions now live under Extensions/FFMpeg/tests, independently of JNI/Android; the Android runner calls those same sources. The review ledger is in tests/README.md.

Audio review corrections — 2026-11-05

Folded into 6dab60eb; the first seven reviewed commits through 4685fd99 are unchanged. The later four commits are replayed, and the audio commit remains awaiting author review.

• Restore forwardFrame's frame-grabbing behavior: discard encountered audio packets; do not enqueue them into the main playback queue.
• Keep the default tempo valid. Create an absent initial graph and explicitly rebuild on seek, without assigning atempo = -1.
• Treat byte seek offsets as bytes, disable time-based preroll trimming for those seeks, and anchor synchronization to the first valid decoded PTS. Independent preview contexts cannot consume that pending playback anchor.
• Normalize the negative-flags avformat_seek_file convention before interpreting byte versus time units.
• Preserve the requested fractional tempo in the filter expression, so the DSP and sample clock use the same value.
• Keep receive → submit → collect ordering, EOF filter/resampler draining, PCM capacity management and seek resets. Midstream timestamp-gap handling remains a separate follow-up; this correction does not claim full discontinuity support.

Run Extensions/FFMpeg/tests/run_audio_seek.py with the same four FFmpeg/SDL include/library options as the existing shared runner. It generates real MP4/MPEG-TS fixtures and tests byte/time seeks, EOF restart, default graph construction, grab queue isolation and fractional tempo. Optional --baseline-include accepts a directory containing the original c63af8fd header for negative checks.

The amended audio stage and all four later stages pass these ASan/UBSan tests. The original audio commit fails all four targeted negative checks (seek units, byte-seek PCM, grab queue isolation and tempo). The final existing PCMFragmentTest, DecoderProtocolTest and SharedCoreTest also pass. Tests used host FFmpeg 7.1.5 and SDL2; no Windows/Android build or device playback was performed for this correction. Across the audio-review corrections, changes relative to the original d2a04e75 head are limited to FFMpeg.h and four audio regression files.

The follow-up review is split by responsibility:

• Audio commit 6dab60eb: distinguish sample-clock startup from a valid timestamp anchor. After missing initial audio PTS, use the first later valid timestamp and account for the preceding input duration; reset this state on seek. The new late-pts test fails on the previous 5d903e6f revision and passes after the correction.
• Clocks commit 1895bc01: estimate playback from submitted PCM, clamp to the actual submitted span (including partial/empty callbacks), and snapshot the callback tempo so changing speed cannot retime already submitted PCM. Pause/resume preserves the remaining submitted span.
• run_audio_clock.py takes the same four include/library options as the other host runners. Its deterministic clock tests cover 0.25x/1x/1.23x/4x, 4/512/4096-byte callbacks, EOF tails, empty callbacks and tempo/pause continuity. Optional --baseline-include confirms the previous clocks revision fails the EOF-bound and tempo-continuity cases. The test compiler exposes the internal estimator with -fno-access-control; production headers gain no test hooks.
• The refined audio tests pass at the audio commit and all four later stages. The clock tests pass at the clocks commit and all three later stages; the final PCM, decoder-protocol and shared-core regressions also pass ASan/UBSan. Hardware/device latency is not measured or compensated by this software-only estimate. Windows/Android build and physical playback remain untested for these corrections.

Platform boundaries and dependencies

Android compiles the original FFMpeg class and established adaptor factories. MediaCodec Surface output and GLES copies use platform adaptors beside D3D. Java retains actual Fusion runtime, renderer, storage and file access. Portable playback state, scheduling, queues, seek and clocks remain in C++.

Both plugins use one SDL Mixer X device: FFmpeg's original PCM callback supplies independent looping-chunk/effect channels while SDL_Mixer retains music streams. FFmpeg tempo uses the shared libavfilter chain.

FFmpeg Android deployment reads Windows' FFmpegVersion.cmake; crypto deployment reads EncryptVersion.cmake and its pinned OpenSSL/vcpkg selection. Android SDK version/source/API/ABI checks reject stale inputs. Both exports ship the same shared audio/crypto runtime and matching bindings/source payloads.

MediaCrypto JNI calls the original Encryption C API, explicitly selecting legacy Win32 seed width for password compatibility. File resolution/materialization/cache leases remain in native Fusion application storage. The FileSource compatibility branch is restricted to Android 32-bit API 21–23; other platform/API paths remain unchanged. The standalone Encrypter Fusion extension is still deferred.

Validation for this cleanup and history rewrite

• Independently compiled and ran all ten portable C++ prefixes, from the portability commit through the generic hardware-lifecycle commit, with host FFmpeg 7.1.5/SDL2. Checks grow with each available API and cover first frame/stride, fractional seek, delayed-frame EOF, audio tail, pause and callback reentry.
• PCM fragmentation regression demonstrably fails immediately before the one-line mix-length fix and passes immediately afterward; an all-silence result is rejected.
• Final standalone C++ regressions passed ASan/UBSan: frame/drain counts, filter/resampler audio tail, fractional seek/rewind/preview, Unicode paths, delayed-audio silence/clock, frame and seek callbacks reentering controls, state-only pause, borrowed memory read/seek/size/EOF, full memory A/V playback to EOF, an independent memory-preview context, caller-owned data preservation after player destruction, and invalid-open cleanup. LeakSanitizer is disabled in the managed ptrace environment.
• The packaged core source/header closure compiled with CMake using the existing DeLib helper module. The pinned four-ABI iconv files and corresponding source hash were verified through FTW's existing validator.
• Shared Windows/Android FFmpeg version and stale source/version/ABI/missing-library rejection checks passed. Current Windows object helpers are unchanged from the reviewed revision; all 56 helper regressions passed.
• The minimized Send First revision reran all five affected core stages and independently ran the decoder commit's protocol regression. The final shared suite passed ASan/UBSan, including real FFmpeg-call wrappers checking Send First ordering, queue-wait early return without codec calls, source EAGAIN recovery, and IO EIO/codec EPERM propagation without EOF/waiting collisions. Existing video EOF, DSP tail, delayed-audio, seek and memory-source checks also passed.
• The audio-order review reran the final shared ASan/UBSan suite and independently compiled/ran the four affected C++ prefixes. Focused 0.25x, 0.5x, 2x and 4x runs reached audio EOF with expected sample totals and identical full PCM for large versus 4-byte callbacks. Only decode_apacket changes in the final tree; the correction is folded into the audio-pipeline commit, preserving the first seven commits and keeping Android last.
• Verified commit boundaries/authorship, exact cleaned final-tree equivalence and git diff --check. The later format-neutral buffer rename and removal of the redundant timeline-origin getter passed syntax-only C++ compilation. The initialization-time timeline-origin cache passed the shared ASan/UBSan regressions and a before/after seek comparison on a video starting at 7 seconds (relative 1500 ms seeks map to source 8.5 seconds in both versions).

The current core prefixes from the changed ownership commit onward, the expanded ASan/UBSan regressions and the packaged-core CMake build were rerun after the MemView review changes. A focused negative check also confirms that removing the destructor adaptor guard crashes on missing-input construction.

Earlier integration validation is recorded in the Android READMEs: four-ABI native/crypto builds, JNI reference/ACE/lifecycle fixtures, encryption interoperability and the independent Android playback probe. Those results were obtained before this cleanup. This rewrite has not rerun a complete Android export/NDK build, Windows MSBuild or physical-device playback; the new common-helper/iconv linkage still needs those integration builds.

Open review finding: single-image preview

A host comparison before the clocks commit, at the clocks commit, and at current HEAD found the same PNG preview limitation with automatic/2-thread decoding: forwardFrame receives EAGAIN on its single packet, then reaches demux EOF without draining delayed codec output, producing no first-frame callback. This predates the clocks commit and remains unresolved; it is not a deletion of the queue/timing safeguards. Single-thread PNG and JPEG at 1/2/automatic threads render one valid frame without a black replacement; current shared-policy repeated seek/playback calls preserve it. These are host software-decoding ASan/UBSan checks, not Windows/Fusion/device-renderer validation. Track codec EOF draining in forwardFrame separately without reopening already-reviewed core commits.

Build and user testing

See the combined build/file/encryption guide and each extension's Android/README.md. Rebuild/install both ZIPs from the same revision and shared SDK caches.

Opening, seeking, video decoding and materialization remain synchronous on the Fusion calling thread. Native Windows pointer/cSurface expressions retain their documented Android empty/0 behavior.

[defisym]

在这里重置-1强制刷新，但是前面为什么赋初始值啊？！

[defisym]

就这里

[defisym]

这是用于快速跳转视频用于抓帧用的，所以音频包都丢掉了啊

[defisym]

你这里加进去反而影响主解码的队列出问题了