Public story · 2026-02-23 · source-backed
xaskasdf/ntransformer — Llama 70B on a Single RTX 3090 (Show HN, 380 pts)
Story
Streams model layers through GPU memory via PCIe with an optional NVMe-to-GPU bypass. Runs Llama 3.1 70B on a single RTX 3090 (24GB VRAM). Warning: NVMe bypass mode can brick your drive — never use on your boot drive. Non-bypass mode alone is a serious consumer-hardware inference option.
Clone it: github.com/xaskasdf/ntransformer | C++, CUDA
Related stories
Each link below shares sources, entities, or timing with this story.
Shared entities / Same source domain / Shared topic / What happened next / Downstream implication
llama.cpp Gets Backend-Agnostic Tensor Parallelism, and the NVIDIA Tax Just Got Optional
Both cover CUDA, GPU, Llama, VRAM; reported by the same outlet (github.com); overlapping topics (inference, llama, model).
Shared entities / Shared topic / What happened next / Tension
Intel Arc Pro B70: 32GB VRAM for $949.
Both cover CUDA, GPU, VRAM; overlapping topics (cuda, inference); picks up the CUDA thread on 2026-04-21.
Shared entities / Same source domain / What happened next
Local Models Just Crossed the "Good Enough" Line for Agentic Coding
Both cover GPU, RTX, VRAM; reported by the same outlet (github.com); picks up the GPU thread on 2026-05-22.
Shared entities / Same source domain / Shared topic / What happened next
Nvidia's Blackwell Ultra Just Made Self-Hosted Inference a Real Option: 2.49M Tokens/Sec, 30 Cents Per Million
Both cover GPU, RTX; reported by the same outlet (github.com); overlapping topics (inference, model).
Shared entities / Same source domain / Earlier coverage / Tension
cjpais/Handy — Free Offline Speech-to-Text (15.7K stars, +94/day)
Both cover CUDA, GPU; reported by the same outlet (github.com); earlier CUDA coverage from 2026-02-22.
Shared entities / Same source domain / Shared topic
tnm/zclaw — AI Agent in 888 KiB on ESP32 (Show HN, 269 pts)
Both cover Clone, Show HN; reported by the same outlet (github.com); overlapping topics (clone, memory).
superhq-ai/shuru — Disposable MicroVM Sandbox for AI Agents (Show HN, 187 pts)
Both cover Clone, Show HN; reported by the same outlet (github.com); overlapping topics (boot, clone).
Shared entity: Show HN / Same source domain / Shared topic / What happened next
ScreenMind runs a vision model on every screenshot, fully on-device.
Both cover Show HN; reported by the same outlet (github.com); overlapping topics (inference, memory, model).
Source trail
Entities
Provenance
- Canonical issue
- Ramsay Research Agent — 2026-02-23
- AI generated
- no
- Story unit
- 2026-02-23-xaskasdf-ntransformer-llama-70b-on-a-single-rtx-3090-show-hn-380-pts
- Labels
- source-backed, canonical briefing excerpt