WaveBlock is an open-source, purely integer-based hybrid video compression architecture designed explicitly for resource-constrained IoT devices (e.g., ESP32-S3) that lack a Floating Point Unit (FPU).
Standard video codecs like MJPEG or H.264 either consume too much bandwidth or require complex floating-point mathematics that overwhelm low-end microcontrollers. WaveBlock solves this by using 100% multiplication-free integer arithmetic (bit-shifts and additions). It allows a basic $5 microcontroller with a camera to stream real-time video over WiFi without burning its CPU.
By combining Integer Discrete Wavelet Transform (DWT) for homogeneous regions and Absolute Moment Block Truncation Coding (AM-BTC) for high-frequency textures, WaveBlock achieves real-time video encoding (10-15 FPS) on edge devices while using 50% less RAM and performing up to 3x faster than standard MJPEG.
WaveBlock employs a Content-Aware Hybrid Codec approach. Instead of compressing every part of an image the same way, it analyzes the image in 16x16 pixel blocks and makes smart decisions:
- Motion Detection (Conditional Refresh): It compares the current block with the previous frame. If the block hasn't changed (e.g., a static background wall), the encoder simply skips it. This is packaged into our custom IOTAV container, saving up to 60% bandwidth on static scenes like surveillance feeds.
- Smooth Areas (DWT): If the block has changed but is mostly smooth (like a clear sky), the encoder applies a highly compressed Integer S-Transform (DWT). It throws away microscopic noise and sends only the base color structure.
- Complex Textures (BTC): If the block has sharp edges or complex textures (like text or a person's face), it switches to a 4x4 AM-BTC algorithm. This keeps the edges sharp and readable without blurring them.
This dynamic switching ensures the highest possible perceptual quality while keeping the mathematical load so light that an ESP32 can encode it on the fly.
- Multi-Language Core: Production-ready encoder/decoder libraries in C/C++ and Go.
- 100% Multiplication-Free: Uses bit-shifts and additions exclusively (via S-Transform).
- Conditional Refresh (IOTAV): Saves up to 60% bandwidth on static scenes by aggressive sensor noise filtering.
- Fast-Forward Seeking: Playback instantly recovers background details using sub-millisecond in-memory packet burst reconstruction without relying on traditional GOP structures.
- VLC Integration: Drop-in VLC Media Player plugin to natively play WaveBlock streams with full seek support.
graph TD
subgraph "Encoder (IoT Device / Streamer)"
Video[Raw Video Frame] --> TileSplit[16x16 Tile Splitter]
TileSplit --> CondRefresh{Motion Detected?}
CondRefresh -- "No (Static)" --> Skip[Skip Frame - IOTAV]
CondRefresh -- "Yes (Motion)" --> Hybrid{Homogeneous?}
Hybrid -- "Yes" --> DWT[Integer DWT]
Hybrid -- "No" --> BTC[AM-BTC]
DWT --> Pack[Packetizer]
BTC --> Pack
Skip --> Pack
end
subgraph "Network"
Pack -- "UDP/TCP Packets" --> Net[Network Transport]
end
subgraph "Decoder (Players)"
Net --> Unpack[Depacketizer]
Unpack --> IOTAV[IOTAV Reassembly]
IOTAV --> DecDWT[DWT Decoder]
IOTAV --> DecBTC[BTC Decoder]
DecDWT --> Canvas[Frame Buffer / Display]
DecBTC --> Canvas
end
├── go/ # Go Library (Encoder/Decoder API & CLI Tools)
├── cpp/ # C++ Library (Decoder API)
├── firmware/ # ESP32-S3 Core Embedded Library (FreeRTOS)
├── tools/ # Ready-to-use Utilities & Plugins
│ └── vlc_plugin/ # VLC Media Player plugin for decoding WaveBlock streams
├── examples/ # Tutorials & Sample Code
│ ├── esp32_cam_basic/ # Barebones ESP32-S3-CAM capture & encode example
│ └── mini_player/ # Minimal player implementation example
├── docs/ # Documentation & Specs
│ ├── IOTAV_TOOLS.md # IOTAV File Container specification
│ └── README.md # SIU 2026 Paper abstract & citation info
├── data/ # Standard test video datasets
└── scripts/ # Reproducibility & Benchmark scripts
The Go implementation contains the reference encoder and a desktop player.
cd go
go build -o ../tools/go_streamer ./cmd/encoder
go build -o ../tools/go_player ./cmd/iotav-playercd cpp
mkdir build && cd build
cmake ..
makeTo run WaveBlock directly on your ESP32-S3 camera module:
- Open
examples/esp32_cam_basic/in your preferred IDE (ESP-IDF or PlatformIO). - Configure your WiFi credentials in the source file.
- Build and flash the firmware to your board.
- The board will connect to WiFi and start streaming UDP WaveBlock packets on port 14444.
- Use the C++
native_playeror VLC to view the live stream:./cpp/build/native_player -p 14444
WaveBlock is designed to be highly tunable for ultra-low bandwidth environments. Use the Go encoder flags to push compression to the limit:
-q <val>(Quantization): Increase to 8 or 16 for aggressive compression. (Default: 4)-key-interval <frames>: Extend the keyframe interval (e.g., 300 for 30s at 10fps). The decoder uses Fast-Forward Seeking to instantly recover the background without needing frequent keyframes.-waveblock-thresh <val>: Increase to 20000+ to force more blocks into the highly compressed DWT path instead of BTC.
Example: Ultra-Low Bandwidth Encoding (e.g. 400 KB for 10s video)
./bin/go_streamer -q 8 -key-interval 300 -waveblock-thresh 25000 -url input.mp4 -out output.iotavWant to play your ESP32's WaveBlock stream directly in VLC?
Navigate to tools/vlc_plugin/ and follow the build instructions for your platform (Linux/Windows). Once installed, VLC will automatically demux and decode iotav:// network streams.
This project originated from a paper presented at SIU 2026. For academic transparency, you can reproduce the exact PSNR, Bitrate, and Speed gains reported in our paper (Table II).
- Ensure FFmpeg and Go are installed.
- Run the benchmark script:
./scripts/benchmark.sh
- Verify your results match the published academic paper (with ±15% tolerance):
./scripts/verify_results.sh
(See docs/README.md for the SIU 2026 paper abstract and BibTeX citation).
This project is licensed under the MIT License - see the LICENSE file for details.