torchrun

Here are 3 public repositories matching this topic...

MGhanayim / k8s-distributed-llm-finetuning

Multi-node PyTorch DDP fine-tuning of a causal LM on Nebius GPU Kubernetes, with SkyPilot workload orchestration — a 2-node torchrun job with verified NCCL collectives.

kubernetes gpu k8s distributed-training nccl mlops pytorch-ddp skypilot llm-ops llm-finetuning nebius gpu-orchestration workload-orchestration ml-orchestration torchrun

Updated Jun 26, 2026
Python

GoKu1402 / ddp-scaling-h100

Star

Why 4xH100 DDP ran 2.4x slower than a single GPU — a measured scaling study of GPT-2 Large on Nebius Managed Kubernetes, tracing the collapse to NCCL socket fallback. PyTorch DDP · SkyPilot · torchrun

kubernetes benchmarking gpu pytorch performance-analysis scaling ddp distributed-training nccl mlops gpt-2 llm h100 skypilot torchrun

Updated Aug 24, 2026
Python

arpitpatel1364 / SsH-Distrib-Train

Star

This is an end-to-end distributed deep learning orchestrator designed to scale model training across multi-node clusters. It simplifies Distributed Data Parallel (DDP), networking, SSH orchestration, and telemetry by unifying them into a seamless workflow with a centralized, real-time web dashboard for monitoring, control, and performance insights.

ssh clustering distributeddataparallel torchrun

Updated Aug 3, 2026
Python

Add this topic to your repo

To associate your repository with the torchrun topic, visit your repo's landing page and select "manage topics."

Learn more