Skip to content

feat: expand SupportedContentType enum with common media types#2233

Open
lxcxjxhx wants to merge 2 commits into
microsoft:mainfrom
lxcxjxhx:feat/expand-supported-content-types
Open

feat: expand SupportedContentType enum with common media types#2233
lxcxjxhx wants to merge 2 commits into
microsoft:mainfrom
lxcxjxhx:feat/expand-supported-content-types

Conversation

@lxcxjxhx

Copy link
Copy Markdown

Motivation

The \SupportedContentType\ enum in \pyrit/memory/storage/storage.py\ had a TODO comment indicating that more media types should be added. Currently only \PLAIN_TEXT\ was supported, which limits PyRIT's ability to handle multimodal data (images, audio, video, PDFs) when uploading to Azure Blob Storage.

This is particularly relevant since the existing doc example \doc/code/executor/5_workflow.py\ already references \SupportedContentType.HTML\ which doesn't exist in the enum yet.

Changes

Expanded \SupportedContentType\ enum with commonly used media types:

Text types: HTML, JSON, XML, CSV, Markdown
Image types: PNG, JPEG, GIF, WebP, SVG, BMP
Audio types: WAV, MP3, OGG, FLAC, M4A
Video types: MP4, WebM, OGG, AVI
Document types: PDF, ZIP, TAR, GZIP

Added unit test \ est_supported_content_type_has_expected_values\ to verify all enum values.

Testing

  • Added comprehensive unit test covering all new enum values
  • Code compiles successfully
  • Formatted with black

Checklist

  • Code follows the project's coding standards
  • Tests have been added
  • Commit messages follow conventional format
  • Changes are minimal and focused on the TODO item

Note: Due to local environment limitations (Windows without full PyRIT dependencies), relying on CI/CD automated testing for full test suite validation.

Add support for image, audio, video, and document content types
to address TODO comment in storage.py. This enables PyRIT to handle
multimodal data (images, audio, video, PDFs) when uploading to
Azure Blob Storage.

Added types:
- Text: HTML, JSON, XML, CSV, Markdown
- Image: PNG, JPEG, GIF, WebP, SVG, BMP
- Audio: WAV, MP3, OGG, FLAC, M4A
- Video: MP4, WebM, OGG, AVI
- Document: PDF, ZIP, TAR, GZIP

Includes unit tests to verify all enum values.
# TODO, add other media supported types
# Text types
PLAIN_TEXT = "text/plain"
HTML = "text/html"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think these values change normal upload behavior: AzureSQLMemory leaves AzureBlobStorageIO at its text/plain default, and serializers cannot pass a per-file content type. Please wire MIME selection into write_file_async (or infer it from the path/serializer) and consolidate the separate AzureBlobStorageTarget enum; otherwise this does not enable the advertised media types.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the detailed feedback, @romanlutz! I've addressed your concerns:

  1. Modified AzureBlobStorageIO.write_file_async: Added an optional content_type parameter that allows specifying the MIME type per file.

  2. Auto-detection of MIME types: Implemented logic using mimetypes.guess_type to automatically infer the content type from file extensions when not explicitly provided.

  3. Updated serializers: Modified save_data_async, save_b64_image_async, and save_formatted_audio_async to pass the inferred content_type when saving data.

  4. Consolidated SupportedContentType enum: Removed the duplicate AzureBlobStorageTarget enum and unified the content type definitions in pyrit.memory.storage.storage.

  5. Added unit tests: Created tests to verify the content type inference and propagation logic.

  6. Testing: Due to local environment constraints (incomplete dependencies on Windows), I'm relying on the CI/CD pipeline to run the full test suite automatically.

The changes should now properly enable the advertised media types by wiring MIME selection into the file upload workflow. Please let me know if you need any additional modifications!

- Modify AzureBlobStorageIO.write_file_async to accept content_type parameter
- Infer content_type from file extension using mimetypes.guess_type
- Pass content_type through serializers (save_data_async, save_b64_image_async, save_formatted_audio_async)
- Consolidate SupportedContentType enum imports
- Add unit tests for content_type inference and propagation

Addresses maintainer feedback on PR microsoft#2233
@lxcxjxhx

Copy link
Copy Markdown
Author

感谢 @romanlutz 的详细反馈!我已经按照您的建议进行了修改:

  1. 修改了 AzureBlobStorageIO.write_file_async 方法:新增了可选的 content_type 参数,允许为每个文件指定 MIME 类型。

  2. 实现了 MIME 类型自动推断逻辑:使用 mimetypes.guess_type 根据文件扩展名自动推断内容类型(当未明确提供时)。

  3. 更新了序列化器:修改了 save_data_asyncsave_b64_image_asyncsave_formatted_audio_async,在保存数据时传递推断的 content_type

  4. 整合了重复的枚举:移除了重复的 AzureBlobStorageTarget 枚举,统一从 pyrit.memory.storage.storage 导入 SupportedContentType

  5. 添加了单元测试:创建了测试用例来验证 content_type 的推断和传递逻辑。

  6. 测试说明:由于本地环境限制(Windows 缺少完整依赖),依赖 CI/CD 自动运行完整测试套件。

这些修改应该能够正确支持广告中声明的媒体类型,通过将 MIME 选择逻辑集成到文件上传工作流中。如果需要进一步修改,请告诉我!

@lxcxjxhx

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

@romanlutz

Copy link
Copy Markdown
Contributor

For context, I'm adding a translation here (I can't read Chinese):

Thanks to @romanlutz for the detailed feedback! I have made the following changes based on your suggestions:

Modified the AzureBlobStorageIO.write_file_async method: Added an optional content_type parameter to allow specifying the MIME type for each file.

Implemented automatic MIME type inference: Used mimetypes.guess_type to automatically infer the content type based on the file extension (when not explicitly provided).

Updated serializers: Modified save_data_async, save_b64_image_async, and save_formatted_audio_async to pass the inferred content_type when saving data.

Consolidated duplicate enums: Removed the redundant AzureBlobStorageTarget enum and standardized the import of SupportedContentType from pyrit.memory.storage.storage.

Added unit tests: Created test cases to verify the logic for inferring and passing the content_type.

Testing note: Due to local environment limitations (missing full dependencies on Windows), the full test suite relies on CI/CD for execution.

These changes should correctly support the media types advertised by integrating the MIME selection logic into the file upload workflow. Please let me know if further modifications are needed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants