Skip to content
Navigation Menu
Sign in
Appearance settings
Platform
AI CODE CREATION
GitHub Copilot
Write better code with AI
GitHub Copilot app
Direct agents from issue to merge
MCP Registry
Integrate external tools
DEVELOPER WORKFLOWS
Actions
Automate any workflow
Codespaces
Instant dev environments
Issues
Plan and track work
Code Review
Manage code changes
Code Quality
Enforce quality at merge
APPLICATION SECURITY
GitHub Advanced Security
Find and fix vulnerabilities
Code security
Secure your code as you build
Secret protection
Stop leaks before they start
EXPLORE
Why GitHub
Documentation
Blog
Changelog
Marketplace
View all features
Solutions
BY COMPANY SIZE
Enterprises
Small and medium teams
Startups
Nonprofits
BY USE CASE
App Modernization
DevSecOps
DevOps
CI/CD
View all use cases
BY INDUSTRY
Healthcare
Financial services
Manufacturing
Government
View all industries
View all solutions
Resources
EXPLORE BY TOPIC
AI
Software Development
DevOps
Security
View all topics
EXPLORE BY TYPE
Customer stories
Events & webinars
Ebooks & reports
Business insights
GitHub Skills
SUPPORT & SERVICES
Documentation
Customer support
Community forum
Trust center
Partners
View all resources
Open Source
COMMUNITY
GitHub Sponsors
Fund open source developers
PROGRAMS
Security Lab
Maintainer Community
Accelerator
GitHub Stars
Archive Program
REPOSITORIES
Topics
Trending
Collections
Enterprise
ENTERPRISE SOLUTIONS
Enterprise platform
AI-powered developer platform
AVAILABLE ADD-ONS
GitHub Advanced Security
Enterprise-grade security features
Copilot for Business
Enterprise-grade AI features
Premium Support
Enterprise-grade 24/7 support
Pricing
Type
/
to search
Sign in
Sign up
Appearance settings
You signed in with another tab or window.
Reload
to refresh your session.
You signed out in another tab or window.
Reload
to refresh your session.
You switched accounts on another tab or window.
Reload
to refresh your session.
Dismiss alert
{{ message }}
Uh oh!
There was an error while loading.
Please reload this page
.
Modalities
/
modalities
Public
Notifications
You must be signed in to change notification settings
Fork
16
Star
111
Code
Issues
67
Pull requests
17
Discussions
Actions
Projects
Security and quality
0
Insights
Additional navigation options
Code
Issues
Pull requests
Discussions
Actions
Projects
Security and quality
Insights
WIP: Mixture-of-Experts
- #454
#454
Open
rrutmann
wants to merge 17 commits into
main
Modalities/modalities:main
from
moe
Modalities/modalities:moe
Copy head branch name to clipboard
Conversation
Commits
17
(17)
Checks
Files changed
Open
WIP: Mixture-of-Experts
#454
rrutmann
wants to merge 17 commits into
main
Modalities/modalities:main
from
moe
Modalities/modalities:moe
Copy head branch name to clipboard
Commits
Commits on Jun 9, 2026
feat: Add moe model
rrutmann
committed
8c972a1
View commit details
Copy full SHA for 8c972a1
Browse repository at this point
Commits on Jun 10, 2026
refactor(moe): Merge MoE implementation into modalities core
rrutmann
committed
7dbd86d
View commit details
Copy full SHA for 7dbd86d
Browse repository at this point
fix(moe): Fix dtype and state_dict mismatch
rrutmann
committed
b70ec51
View commit details
Copy full SHA for b70ec51
Browse repository at this point
chore: Remove outdated files
rrutmann
committed
e5edeeb
View commit details
Copy full SHA for e5edeeb
Browse repository at this point
Commits on Jun 11, 2026
docs: Add removed comments
rrutmann
committed
e09aa06
View commit details
Copy full SHA for e09aa06
Browse repository at this point
test: Add tests for MoE components
rrutmann
committed
baf94e9
View commit details
Copy full SHA for baf94e9
Browse repository at this point
test: Add e2e moe test
rrutmann
committed
2a3b81a
View commit details
Copy full SHA for 2a3b81a
Browse repository at this point
refactor: Merge shared rotary embedding logic
rrutmann
committed
f7f87b1
View commit details
Copy full SHA for f7f87b1
Browse repository at this point
Commits on Jun 12, 2026
Merge pull request #451 from Modalities/moe_refactor
Show description for 293e8a0
rrutmann
authored
293e8a0
View commit details
Copy full SHA for 293e8a0
Browse repository at this point
Commits on Jun 17, 2026
implemented torch native expert parallelism, torchtitan dependency removed
gbesposito
committed
3eb373e
View commit details
Copy full SHA for 3eb373e
Browse repository at this point
feat: add dedicated EP dimension to device mesh
gbesposito
committed
e0217fb
View commit details
Copy full SHA for e0217fb
Browse repository at this point
fix: mixed precision bug in ep layers
gbesposito
committed
f638aec
View commit details
Copy full SHA for f638aec
Browse repository at this point
fix: apply black formatting to expert_parallel
gbesposito
committed
3730c09
View commit details
Copy full SHA for 3730c09
Browse repository at this point
Commits on Jun 18, 2026
Merge pull request #453 from Modalities/moe_fixes
Show description for ee2c5dc
rrutmann
authored
ee2c5dc
View commit details
Copy full SHA for ee2c5dc
Browse repository at this point
Commits on Jun 23, 2026
refactor: Merge moe model factory into general model factory
rrutmann
committed
a801e99
View commit details
Copy full SHA for a801e99
Browse repository at this point
refactor: Merge MoECrossEntropyLoss into modalities core
rrutmann
committed
daea062
View commit details
Copy full SHA for daea062
Browse repository at this point
Commits on Jul 17, 2026
Merge pull request #457 from Modalities/moe_rf
Show description for f8534f6
rrutmann
authored
f8534f6
View commit details
Copy full SHA for f8534f6
Browse repository at this point
You can’t perform that action at this time.
