← The Wire

Public story · 2026-02-23 · source-backed

xaskasdf/ntransformer — Llama 70B on a Single RTX 3090 (Show HN, 380 pts)

Confidence
source-backed
Sources
1
Redaction
passed

Story

Streams model layers through GPU memory via PCIe with an optional NVMe-to-GPU bypass. Runs Llama 3.1 70B on a single RTX 3090 (24GB VRAM). Warning: NVMe bypass mode can brick your drive — never use on your boot drive. Non-bypass mode alone is a serious consumer-hardware inference option.

Clone it: github.com/xaskasdf/ntransformer | C++, CUDA

Related stories

Each link below shares sources, entities, or timing with this story.

  1. Shared entities / Same source domain / Shared topic / What happened next / Downstream implication

    llama.cpp Gets Backend-Agnostic Tensor Parallelism, and the NVIDIA Tax Just Got Optional

    Both cover CUDA, GPU, Llama, VRAM; reported by the same outlet (github.com); overlapping topics (inference, llama, model).

  2. Shared entities / Shared topic / What happened next / Tension

    Intel Arc Pro B70: 32GB VRAM for $949.

    Both cover CUDA, GPU, VRAM; overlapping topics (cuda, inference); picks up the CUDA thread on 2026-04-21.

  3. Shared entities / Same source domain / What happened next

    Local Models Just Crossed the "Good Enough" Line for Agentic Coding

    Both cover GPU, RTX, VRAM; reported by the same outlet (github.com); picks up the GPU thread on 2026-05-22.

  4. Shared entities / Same source domain / Shared topic / What happened next

    Nvidia's Blackwell Ultra Just Made Self-Hosted Inference a Real Option: 2.49M Tokens/Sec, 30 Cents Per Million

    Both cover GPU, RTX; reported by the same outlet (github.com); overlapping topics (inference, model).

  5. Shared entities / Same source domain / Earlier coverage / Tension

    cjpais/Handy — Free Offline Speech-to-Text (15.7K stars, +94/day)

    Both cover CUDA, GPU; reported by the same outlet (github.com); earlier CUDA coverage from 2026-02-22.

  6. Shared entities / Same source domain / Shared topic

    tnm/zclaw — AI Agent in 888 KiB on ESP32 (Show HN, 269 pts)

    Both cover Clone, Show HN; reported by the same outlet (github.com); overlapping topics (clone, memory).

  7. superhq-ai/shuru — Disposable MicroVM Sandbox for AI Agents (Show HN, 187 pts)

    Both cover Clone, Show HN; reported by the same outlet (github.com); overlapping topics (boot, clone).

  8. Shared entity: Show HN / Same source domain / Shared topic / What happened next

    ScreenMind runs a vision model on every screenshot, fully on-device.

    Both cover Show HN; reported by the same outlet (github.com); overlapping topics (inference, memory, model).

Source trail

Entities

Provenance

AI generated
no
Story unit
2026-02-23-xaskasdf-ntransformer-llama-70b-on-a-single-rtx-3090-show-hn-380-pts
Labels
source-backed, canonical briefing excerpt