Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/rust-detection-only-feature.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'@smooai/file': patch
---

**Rust: a detection-only build.** The crate's I/O `File` type (local, URL, stream and S3 sources) is now behind a `file` feature, on by default, so existing users see no change. With `default-features = false`, `smooai-file` is just magic-byte detection (`detection`), `content_disposition` and the validation error types, and depends on only `infer`, `mime_guess` and `thiserror`, with no AWS SDK, reqwest or tokio. That makes it usable from a service's core library that only needs to sniff bytes it already holds, which is where dogfooding needs it: the Smoo AI media pipeline had two hand-written `infer` sniffers that drifted apart. CI now checks both builds.

**Rust: `infer` 0.16 → 0.19** for newer magic-number signatures.

**.NET: SourceLink 8.0.0 → 10.0.303.** `Microsoft.Build.Tasks.Git` 8.0.0 picked up advisory GHSA-23fw-v26w-5fgq (CVE-2026-62900), and with `TreatWarningsAsErrors` that failed every restore, which red-lit CI. This is build-time only (`PrivateAssets="All"`), so nothing ships to consumers.
2 changes: 1 addition & 1 deletion dotnet/Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,6 @@
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0" PrivateAssets="All" />
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="10.0.303" PrivateAssets="All" />
</ItemGroup>
</Project>
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@
"python:test": "(cd python && uv run poe test)",
"python:typecheck": "(cd python && uv run poe typecheck)",
"rust:build": "(cd rust/file && cargo build --release)",
"rust:check": "(cd rust/file && cargo check --all-targets)",
"rust:check": "(cd rust/file && cargo check --all-targets && cargo check --no-default-features --all-targets)",
"rust:fmt": "(cd rust/file && cargo fmt)",
"rust:fmt:check": "(cd rust/file && cargo fmt --all -- --check)",
"rust:lint": "(cd rust/file && cargo clippy --all-targets -- -D warnings)",
Expand Down
4 changes: 2 additions & 2 deletions rust/file/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

61 changes: 44 additions & 17 deletions rust/file/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,28 +10,55 @@ keywords = ["file", "s3", "streaming", "smooai"]
categories = ["filesystem"]
readme = "README.md"

[features]
# `file` = the I/O `File` type (local, URL, stream and S3 sources). On by default.
# `default-features = false` leaves `detection`, `content_disposition` and the
# validation error types, with no AWS SDK, reqwest or tokio — for services that
# only need to sniff bytes they already hold.
default = ["file"]
file = [
"dep:reqwest",
"dep:bytes",
"dep:tokio",
"dep:tokio-util",
"dep:serde",
"dep:serde_json",
"dep:sha2",
"dep:hex",
"dep:base64",
"dep:aws-sdk-s3",
"dep:aws-config",
"dep:chrono",
"dep:futures",
"dep:tracing",
"dep:url",
"dep:tempfile",
]

[dependencies]
reqwest = { version = "0.12", features = ["json", "stream", "multipart"] }
bytes = "1"
tokio = { version = "1", features = ["full"] }
tokio-util = { version = "0.7", features = ["io"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
infer = "0.16"
infer = "0.19"
mime_guess = "2"
sha2 = "0.10"
hex = "0.4"
base64 = "0.22"
aws-sdk-s3 = "1"
aws-config = "1"
chrono = { version = "0.4", features = ["serde"] }
thiserror = "2"
futures = "0.3"
tracing = "0.1"
url = "2"
tempfile = "3"
reqwest = { version = "0.12", features = ["json", "stream", "multipart"], optional = true }
bytes = { version = "1", optional = true }
tokio = { version = "1", features = ["full"], optional = true }
tokio-util = { version = "0.7", features = ["io"], optional = true }
serde = { version = "1", features = ["derive"], optional = true }
serde_json = { version = "1", optional = true }
sha2 = { version = "0.10", optional = true }
hex = { version = "0.4", optional = true }
base64 = { version = "0.22", optional = true }
aws-sdk-s3 = { version = "1", optional = true }
aws-config = { version = "1", optional = true }
chrono = { version = "0.4", features = ["serde"], optional = true }
futures = { version = "0.3", optional = true }
tracing = { version = "0.1", optional = true }
url = { version = "2", optional = true }
tempfile = { version = "3", optional = true }

[dev-dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["test-util", "macros", "rt-multi-thread"] }
wiremock = "0.6"
memory-stats = "1"
7 changes: 7 additions & 0 deletions rust/file/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,13 @@ Or via cargo:
cargo add smooai-file
```

Only need to sniff bytes you already hold? Turn off the default `file` feature. That leaves `detection`, `content_disposition` and the validation errors, with no AWS SDK, reqwest or tokio:

```toml
[dependencies]
smooai-file = { version = "2", default-features = false }
```

### Multi-Language Support

smooai-file is available as native implementations in **TypeScript**, **Python**, **Rust**, and **Go** — each built with idiomatic patterns for its ecosystem.
Expand Down
1 change: 1 addition & 0 deletions rust/file/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ pub enum FileError {
Io(#[from] std::io::Error),

/// An HTTP error occurred while fetching a file from a URL.
#[cfg(feature = "file")]
#[error("HTTP error: {0}")]
Http(#[from] reqwest::Error),

Expand Down
46 changes: 37 additions & 9 deletions rust/file/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,30 +14,58 @@
//! - **Streams**: Async byte streams
//! - **Amazon S3**: Objects in S3 buckets
//!
//! # Features
//!
//! - `file` (default): the I/O [`File`] type and its sources. Without it
//! (`default-features = false`) the crate is just [`detection`],
//! [`content_disposition`] and the [`error`] types — no AWS SDK, reqwest or
//! tokio.
//!
//! # Examples
//!
//! ```no_run
//! # use smooai_file::File;
//! # use bytes::Bytes;
//! # async fn example() -> smooai_file::error::Result<()> {
//! let file = File::from_bytes(Bytes::from("hello world"), None).await?;
//! let text = file.read_text().await?;
//! assert_eq!(text, "hello world");
//! # Ok(())
//! # }
//! Sniffing bytes you already hold (no features needed):
//!
//! ```
//! use smooai_file::detection::detect_from_bytes;
//!
//! let png = [0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A, 0, 0, 0, 0x0D];
//! assert_eq!(detect_from_bytes(&png, None).mime_type.as_deref(), Some("image/png"));
//! ```
#![cfg_attr(
feature = "file",
doc = r#"
With the `file` feature:

```no_run
# use smooai_file::File;
# use bytes::Bytes;
# async fn example() -> smooai_file::error::Result<()> {
let file = File::from_bytes(Bytes::from("hello world"), None).await?;
let text = file.read_text().await?;
assert_eq!(text, "hello world");
# Ok(())
# }
```
"#
)]

pub mod content_disposition;
pub mod detection;
pub mod error;
#[cfg(feature = "file")]
pub mod file;
#[cfg(feature = "file")]
pub mod metadata;
#[cfg(feature = "file")]
pub mod source;

// Re-export primary types at the crate root for convenience.
pub use crate::error::{FileError, FileValidationError};
#[cfg(feature = "file")]
pub use crate::file::{File, PresignedUploadOptions, LAZY_HEAD_BYTES};
#[cfg(feature = "file")]
pub use crate::metadata::{Metadata, MetadataHint};
#[cfg(feature = "file")]
pub use crate::source::FileSource;

/// The crate version.
Expand Down
2 changes: 2 additions & 0 deletions rust/file/tests/detection_tests.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#![cfg(feature = "file")]

//! Integration tests for file type detection.

use smooai_file::detection::{
Expand Down
2 changes: 2 additions & 0 deletions rust/file/tests/file_tests.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#![cfg(feature = "file")]

//! Integration tests for the File struct.

use bytes::Bytes;
Expand Down
2 changes: 2 additions & 0 deletions rust/file/tests/integration_tests.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#![cfg(feature = "file")]

//! Full pipeline integration tests.

use bytes::Bytes;
Expand Down
2 changes: 2 additions & 0 deletions rust/file/tests/lazy_contract_tests.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#![cfg(feature = "file")]

//! The Rust loader for the shared lazy-streaming contract.
//!
//! Every port has one of these and they all read the SAME file
Expand Down
2 changes: 2 additions & 0 deletions rust/file/tests/lazy_stream_tests.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#![cfg(feature = "file")]

//! Lazy-stream tests for SMOODEV-967.
//!
//! These exercise `File::from_stream_lazy`, `iter_bytes`, and the lazy path
Expand Down
Loading