Building frontier AI architectures from scratch in raw PyTorch.
LLMs · Latent Diffusion · Multimodal · Video Understanding · Agentic ML · State-Space Models · Long-Context Attention
🎯 Open To:
Deep Learning Research Engineer
LLM Engineer
GenAI / Diffusion Engineer
Agentic ML Engineer
🌍 Remote-friendly · Available worldwide
Building and shipping production-grade from-scratch AI — from LLM pre-training infrastructure to autonomous multi-agent research orchestration. Every architecture is implemented layer-by-layer in raw PyTorch, verified with rigorous tests, and tracked on W&B / Comet.
|
🧠 HyMo Flagship Hybrid LLM 750M active · 1.86B stored 3:1 GDN/MLA · Asymmetric MoE · MTP Custom Triton GDN · FSDP-2 |
📉 LLaMA-3-Lite Memory-Engineered LLM 78% peak memory reduction 92 GB → 20 GB on A100 batch 96 · 2× headroom |
🎨 Stable Diffusion From-Scratch UNet 860M UNet · 7 training phases 0.0947 loss @ epoch 16 2× RTX 5090 · 1.3M+ images |
|
🔭 GPT-OSS-Lite Long-Context MoE 2× KV-cache cut at 128K Sliding/Full alt + learned sinks 502M params · 130 tests |
⚡ Mamba-3-Lite Complex-Valued SSD 50% smaller state (N=64) Parity loss vs Mamba-2 N=128 Pure PyTorch · no custom CUDA |
🤖 AutoML Researcher Multi-Agent Platform 15 phases · 23 agents · 878 tests 61 tools · 186 models Paper → experiment → report |
| Project | Scale | Key Innovation | Hardware | Repo |
|---|---|---|---|---|
| HyMo ⭐ | 750M act / 1.86B stored | 3:1 GDN/MLA hybrid · Asymmetric MoE · MTP · custom Triton GDN kernel · FSDP-2 | 4× A100 80GB | → |
| GPT-OSS-Lite | 502M / 247M active | Sliding(128)/Full attn alt · learned sink bias · YaRN 128K · top-2-of-8 MoE · 2× KV-cache cut · 130 tests | A100 80GB | → |
| Mamba-3-Lite | ~380M | Complex64 SSD (N=64) · MIMO head mixing · zero causal conv · pure PyTorch · parity loss at half state size | A100 80GB | → |
| DeepSeek-v3-Lite | ~422M | MLA + AuxLossFree MoE + MTP · absorption-trick inference · 643-line MLA technical deep-dive | A100 80GB | → |
| LLaMA-3-Lite | ~515M | GQA · RoPE · SwiGLU · chunked CE · 78% memory cut (92 GB → 20 GB) · batch 96 on single A100 | A100 80GB | → |
| TranslationLM | ~44M | Encoder-decoder Transformer · EN→IT · loss 6.17→2.28 · BLEU/CER/WER · attention viz | P100 | → |
| GPT-2 | ~124M | nanoGPT-style · tiktoken BPE · HF weight loading · educational decoder-only scaffold | MPS/CPU | → |
Predecessors: FusionLLM (MLA+GDN+MoE+MTP, 415M active) · GPT-From-Scratch (character-level, 6M params, ~94 min train)
| Project | Scale | Key Innovation | Hardware | Repo |
|---|---|---|---|---|
| Stable Diffusion 1.x | 860M UNet | Custom UNet from random init · 7-phase curriculum · 1.3M+ images · best loss 0.0947 · epoch-42 checkpoint | 2× RTX 5090 | → |
| Detect-Objects | ~50M | RT-DETR/DINO deformable detector · no anchors/NMS · COCO 2017 · Gradio + ONNX | 2× RTX 5090 | → |
| Upscale-SR | ~75M | 4× real-world SR · latent-diffusion UNet + SSM refiner · Real-ESRGAN degradation | 2× RTX 5090 | → |
| ActionRecognition | 120 cls | HRNet pose + Two-Stream CTR-GCN · ~30 FPS inference · ONNX + TensorRT + FastAPI | RTX 3090 | → |
| FaceAgingCycleGAN | 256² | AdaIN per-layer conditioning · 3-scale PatchGAN · LSGAN + R1 GP · 31/50 epochs | RTX 6000 Ada | → |
| FaceGenerationVAE | β-VAE | 50 epochs · recon MSE 0.0152 · linear KL annealing · bilinear-upsample decoder (no checkerboard) | P100 | → |
| DCGAN-Face-Generation | 6.4M | N(0,0.02) init · 202K CelebA · D loss → ln 2 ≈ 0.693 GAN equilibrium | 2× T4 | → |
| VisionLanguageModel | PaliGemma-style | SigLIP ViT + Gemma decoder · linear projector · zero pretrained weights · COCO 2014 | P100 | → |
| Project | Scale | Key Innovation | Hardware | Repo |
|---|---|---|---|---|
| Autonomous ML Research Engineer | 15 phases · 23 agents · 61 tools | Full paper → conclusions loop · self-repair · provider-agnostic LLM routing · 878 tests · 186-model registry | Local + Ollama | → |
| newsagent | 237 tests | Autonomous AI research-intel agent · daily 15+ source sweep · LLM-reasoned reports with provenance | Local | → |
| Document | Length | Covers |
|---|---|---|
| Multi-Head Latent Attention — Technical Deep-Dive | 643 lines | KV-cache math · low-rank compression algebra · absorption-trick derivation · decoupled RoPE · SDPA vs manual attention |
| Attention Sinks — StreamingLLM for GPT-OSS | 600 lines | Per-head learned sink bias · BF16 stability (clamp [-10,15]) · sliding/full alt interaction |
| State-Space Duality — The Mamba-3 SSD Algorithm | Full derivation | Chunkwise SSD · complex64 packing · naive O(T) recurrence equivalence |
Languages & ML Core
Architectures
Transformers GQA MLA MoE GDN MTP
SSD (real & complex64) MIMO Diffusion UNet
VAE GAN CycleGAN AdaIN
ST-GCN HRNet SigLIP Deformable DETR
Optimization & Numerics
BF16 FP16 FP8 Flash Attention 2
SDPA torch.compile channels_last
Gradient checkpointing μP scaling WSD LR
NorMuon CautiousAdamW Chunked CE
Fused optimizers Chinchilla-optimal scaling
Hardware Validated
Tooling
- From-scratch PyTorch — no Trainer, no Lightning, no accelerate; every layer is written by hand
- Single-GPU feasibility — every large project fits one consumer GPU via BF16, gradient checkpointing, FA2,
channels_last, fused optimizers - Faithful reproductions — DeepSeek-V3, LLaMA-3, GPT-OSS, Mamba-3, PaliGemma, DCGAN — implemented to the paper
- Novel hybrids — HyMo (GDN + MLA + MoE + MTP), FaceAgingCycleGAN (AdaIN-conditioned), GPT-OSS-Lite (sink bias + sliding/full alt)
- Production hygiene — atomic checkpoints (
.tmp.pt→os.rename), full RNG-state reproducibility, W&B / Comet tracking, CI lint + tests - Hardware breadth — MPS/CPU → Kaggle T4/P100 → A100 80GB → 2× RTX 5090 → RTX 6000 Ada
B.Tech, 2024 · Heritage Institute of Technology, Kolkata. Self-taught in deep learning through two years of from-scratch implementation — engineering discipline from infrastructure and constraint work translates directly to memory budgets, distributed training, and reproducible ML systems.
