by ARahim3
Qwen3.8-27B on a free Kaggle TPU: OpenAI-compatible endpoint, 262k context, ~130 tok/s, works with Claude Code, Codex, Opencode and Pi.
Unlocks once the catalog security scan passes (runs nightly).
⚠️ Third-Party Software Notice
This skill is third-party open-source software developed and hosted independently on GitHub. SkillsLLM is an informational directory and does not control or maintain the underlying repository.
Any security checks, ratings, or warnings displayed by SkillsLLM are automated and limited in scope. They do not constitute a security certification or guarantee that the software is safe, error-free, or free from malicious code, vulnerabilities, compromised dependencies, or prompt-injection risks.
Review the source code, permissions, dependencies, and configuration before installing or running any third-party skill. Use is at your own risk. To the maximum extent permitted by applicable law, SkillsLLM is not liable for losses arising from third-party software.
The deep catalog scan for this skill is still queued. Run an instant dependency check now instead.
# Add to your Claude Code skills
git clone https://github.com/ARahim3/kaggle-tpu-labGuides for using ai agents skills like kaggle-tpu-lab.
Serve Qwen3.8-27B — a frontier-class 27B hybrid-attention model — on Kaggle's free TPU v5e-8, with a public OpenAI-compatible endpoint you can plug into Claude Code, Codex CLI, opencode, or anything else that speaks the OpenAI API.
No paid GPU, no cloud account, no quantization. Full bf16 weights, up to the model's native 262,144-token context, and real speed:
| What | Measured (TPU v5e-8, bf16, TP=8) |
|---|---|
| Decode, single stream | ~130 tok/s with MTP speculative decoding (78 without, measured before the token-bucket change) |
| Decode, 8 concurrent streams | ~540 tok/s aggregate (~107 tok/s each, MTP on) |
| Decode, 16 concurrent streams | ~900 tok/s aggregate (with --mtp 0 — see tuning note below) |
| Prefill | 10,300 tok/s — a 105k-token prompt in ~10 s |
| Native 262k context | works — 225k-token prompt prefilled in ~28 s |
| Time to live endpoint | ~22 min with the env dataset attached (~12 with --text-only, ~6 with --fast-start; ~40 without the dataset) |
| Output correctness with MTP | verified lossless — 12/12 greedy prompts exactly match non-speculative |
Tuning note: speculative decoding pays off up to ~8 concurrent streams and fades beyond
that (verification competes with batch compute). Serving many users? Launch with
--max-model-len 131072 --max-num-seqs 16 --mtp 0 for max aggregate throughput.
Qwen3.8-27B is a hybrid: 48 of its 64 layers are gated-DeltaNet linear attention, only 16 are classic full attention. That makes its KV cache tiny (~64 KB/token), which is why a 27B can serve 131k+ contexts on 8×16 GB TPU chips with room to spare. Until recently no TPU stack could run the DeltaNet layers — vllm-tpu 0.28.0 shipped native Pallas kernels for them (Aug 2026), and this repo is the recipe that puts it all together on Kaggle's free tier: a pre-built Python runtime with pinned versions, pre-mirrored weights, a pre-built XLA compile cache, MTP speculative decoding, and a tunnel to the outside world.
Copy & Edit the published Kaggle notebook and Run it —
kaggle.com/code/rahim3/qwen3-8-27b-bf16-on-kaggle-tpu-130-tok-s-api
— or upload notebook/qwen38-tpu-serve.ipynb yourself.
Set Accelerator = TPU VM v5e-8, Internet = ON, attach the two datasets named in
the first cell, and run top to bottom. The last cell is the server — the endpoint URL
and API key appear in its output.
You need Python 3.9+ and a Kaggle account with TPU access (Settings → phone-verify your account if you haven't; free tier includes ~20 TPU hours/week).
# 1. Kaggle CLI + API token (one-time)
pip install kaggle
# kaggle.com → Settings → API → Create New Token, then place the file:
# Linux/macOS: ~/.kaggle/kaggle.json Windows: %USERPROFILE%\.kaggle\kaggle.json
# 2. Get this repo and launch
git clone https://github.com/ARahim3/kaggle-tpu-lab
cd kaggle-tpu-lab
python launch.py serve
That's it. The launcher pushes a script kernel to your Kaggle account, and streams progress to your terminal so you always know what's happening:
[14:05] Pushing kernel you/qwen38-tpu-serve (TPU v5e-8)...
[14:06] Kaggle: queued — waiting for a TPU v5e-8 slot...
[14:08] Kaggle: provisioning the VM and attaching datasets (a few minutes)...
[14:14] Building the Python runtime with uv (~30 s)...
[14:15] Runtime ready.
[14:15] XLA compile cache restored for this exact config — fast start.
[14:15] Weights found mounted (no download needed).
[14:15] Starting vLLM — loading 55 GB of weights, then TPU graph compile...
[14:15] Endpoint URL reserved: https://xxxx-yyyy.trycloudflare.com/v1 (not live yet — wait for the banner)
[14:19] Loading / compiling... 4 min elapsed (typically ~20 min with the env dataset, ~35 min without)
[14:35] Server is HEALTHY after 20 min.
==================================================================
YOUR ENDPOINT IS LIVE
base URL : https://xxxx-yyyy.trycloudflare.com/v1
API key : sk-....
model : qwen3.8-27b (context: 262144)
==================================================================
Ctrl-C detaches without stopping the server. Re-attach with
python launch.py status -f; kill the TPU session with python launch.py stop.
Useful flags (the default launch serves the full native 262k context, 4 concurrent sequences):
python launch.py serve --max-model-len 131072 --max-num-seqs 16 # many parallel streams (~900 tok/s aggregate)
python launch.py serve --reasoning-effort medium # server-side default
python launch.py serve --keepalive-min 120 # auto-stop after 2 h
python launch.py serve --text-only # skip the vision tower: ~10 min faster, no image inputs
python launch.py serve --fast-start # live in ~6 min; common shapes warmed after, rare ones stall ~1 min once
The endpoint is standard OpenAI API, with tool calling enabled
(--enable-auto-tool-choice, qwen3_coder parser — matches Qwen3.8's XML tool format)
and the qwen3 reasoning parser, so thinking content is separated properly. The
launcher/notebook print the endpoint URL and API key in a banner when the server is up.
Codex CLI / opencode / aider / anything OpenAI-compatible:
export OPENAI_BASE_URL="https://<your-tunnel>.trycloudflare.com/v1"
export OPENAI_API_KEY="sk-<your-key>"
# model name: qwen3.8-27b
Claude Code — the bundled vLLM also exposes an Anthropic-compatible /v1/messages
(reasoning arrives as proper thinking blocks; we verified Claude Code end-to-end
against it). One gotcha: the server authenticates with a Bearer header only, so use
ANTHROPIC_AUTH_TOKEN, not ANTHROPIC_API_KEY:
export ANTHROPIC_BASE_URL="https://<your-tunnel>.trycloudflare.com"
export ANTHROPIC_AUTH_TOKEN="sk-<your-key>"
export ANTHROPIC_MODEL="qwen3.8-27b"
export ANTHROPIC_SMALL_FAST_MODEL="qwen3.8-27b"
claude
Qwen3.8 has three thinking levels: xhigh (default), medium, low — plus off. Set a per-request level (works from any client that lets you add request fields):
{"chat_template_kwargs": {"reasoning_effort": "low"}}
or turn thinking off entirely with {"enable_thinking": false}. To change the
server-side default (for clients that can't pass extra fields), launch with
--reasoning-effort medium.
launch.py the CLI: serve / status / stop, with live progress
kernel/serve_qwen38.py the Kaggle kernel: runtime → cache → weights → vLLM → tunnel → READY
notebook/qwen38-tpu-serve.ipynb the same flow as a run-it-yourself notebook
patches/mtp-rollback-v0280.diff GDN state-rollback fix (port of tpu-inference PR #3178)
tools/embed_patch.py re-embeds the patch into the kernel script after edits
Plus two public Kaggle datasets the kernel attaches:
rahim3/qwen3-8-27b-bf16 — mirror of Qwen/Qwen3.8-27B (55.6 GB safetensors).
Attaching it skips the HF download entirely.rahim3/qwen38-tpu-env-v5e8 — the JAX/XLA compile cache for the documented
configs (262k/4 and 131k/16 with images on, plus 262k/4 text-only; all MTP k=3), a
cloudflared binary, and a manifest.json recording the build date and versions.
The kernel pins its uv dependency resolution to that build date so the cache keeps
matching. If the Python or vllm-tpu version ever drifts the cache is ignored and the
graphs compile cold — slower, never broken.The first version of this recipe took ~50 min from "Run" to a live URL. Measured now: ~22
(~12 with --text-only, ~6 with --fast-start):
| Step | Before | Now | How |
|---|---|---|---|
| pip install | 11 min | 30 s | uv into a fresh venv with CPU torch (the PyPI default is the CUDA build + 3 GB of NVIDIA libs), resolution pinned to the cache's build date |
| Weights → TPU | 3 min | 3 min | reading 55 GB; unchanged |
| Vision-tower graphs | 13–23 min | ~8 min (0 with --text-only) |
the compile part is now cached; the rest is tpu-inference tracing the vision encoder at warm-up, which no cache can skip |
Text graphs at all in --fast-start |
— | 0 at startup | precompile skipped; the script warms the common shapes right after READY, rare shapes stall ~1 min once |
| Text graphs | 8 buckets, ~19 min cold | 6 buckets, ~4 min | cache built for the shipped config (the old cache never matched, so every run compiled cold), MIN_TOKEN_BUCKET=64, 4 parallel compile threads |
| Tunnel | after the self-test | in parallel with the server start | URL is printed early, banner marks readiness |
Maintainers rebuild the bundle with python launch.py build-env (serves each config
once on a TPU kernel, ~2 h) and create/version the dataset from the kernel's output
folder in the Kaggle UI (Output tab → New Dataset).
--keepalive-min so a forgotten
session doesn't eat your quota.kaggle-tpu-lab is an open-source ai agents skill for AI coding assistants such as Claude Code, Codex CLI, and ChatGPT, built by ARahim3. Qwen3.8-27B on a free Kaggle TPU: OpenAI-compatible endpoint, 262k context, ~130 tok/s, works with Claude Code, Codex, Opencode and Pi. It has 50 GitHub stars.
kaggle-tpu-lab's catalog security scan is still queued. You can run an instant dependency and prompt-injection check now with the "Scan for vulnerabilities" button above.
Clone the repository with "git clone https://github.com/ARahim3/kaggle-tpu-lab" and add it to your Claude Code skills directory (see the Installation section above).
kaggle-tpu-lab is primarily written in Python. It is open-source under ARahim3 on GitHub, so you can review or fork the full source.
Yes. SkillsLLM lists many other AI Agents skills you can browse and compare side by side. Open the AI Agents category from the badge at the top of this page, or use the Related Skills and comparison links further down to weigh kaggle-tpu-lab against similar tools.
No comments yet. Be the first to share your thoughts!