⚡ Qwen 3.8 27B — Multi-Kaggle Cluster

Authenticated as admin · 64K Context Ultra-Low Latency Engine

Active GPU Backends

- / -

Checking nodes...

Total Tokens Processed

0 Prompt · 0 Output

Avg Generation Speed

0.0 tok/s

Native C++ Dual-T4 Engine

Total Requests

0 ms avg latency

🔗 Quick Start Summary

Open a Kaggle Notebook (with GPU T4 x2 & Internet ON) and paste 02_serve_qwen_llamacpp.py.

Set Kaggle Secrets: ACCOUNT_ID = "gaurav", ADMIN_TOKEN = "qwen-secret-a87f2e19b4c036d7e98a2f5c1d4b6e8a".

Hit Run All. The node automatically starts native CUDA llama-server and registers below!

🟢 Live Kaggle Backends & Remote Control ROUND_ROBIN

Account Node	GPU Config	Session Runtime	Last Heartbeat	Status	Actions
Loading backends...

⚙️ Load Balancer & Rotation Policy

Strategy

⚡ Real-Time Generation Speed (tok/s)

⏱️ End-to-End Latency (ms)

📊 Token Volume per Request (Recent)

🔑 Client API Keys (for IDE / Cursor / Apps)

Key Name	Created	Requests	Total Tokens	Last Used	Status	Actions

👥 Kaggle Account Roster

Account ID	Display Name	Kaggle User	Weekly Limit	Status	Actions
Loading accounts...

📊 Paginated Traffic Logs

Timestamp	Served By	Method	Endpoint	Tokens (In / Out)	Speed	Status	Latency

Showing 0-0 of 0 requests

Page 1 of 1

⚡ Cluster Credentials & Endpoints

OpenAI API Base URL

https://qwen-proxy.gauravsuthar980.workers.dev/v1

Gateway Worker URL

https://qwen-proxy.gauravsuthar980.workers.dev

Shared Admin Secret

qwen-secret-a87f2e19b4c036d7e98a2f5c1d4b6e8a

Model Identifiers

qwen3.8-27b 64K Context

Reasoning aliases: qwen3.8-27b-fast, qwen3.8-27b-high

🔐 Step 1: Configure Kaggle Notebook Settings & Secrets

In your Kaggle notebook, open the right sidebar settings:

⚡ Accelerator: GPU T4 x2 (Dual T4) 🌐 Internet: On (Required) 📁 Persistence: No persistence / Files only

Go to Settings → Add-ons → Secrets in your notebook and add the following keys:

Secret Key	Recommended Value	Description
`WORKER_URL`	https://qwen-proxy.gauravsuthar980.workers.dev	Your Cloudflare Worker Gateway URL
`ADMIN_TOKEN`	qwen-secret-a87f2e19b4c036d7e98a2f5c1d4b6e8a	Admin token for tunnel registration & watchdog
`ACCOUNT_ID`	gaurav (or `rahul`, `friend-2`)	Unique node ID used for load balancing
`ACCOUNT_NAME`	Gaurav Dual-T4	Display name shown on the live cluster table

🚀 Step 2: Serving Script (02_serve_qwen_llamacpp.py)

Paste this script into your Kaggle Notebook cell. It automatically compiles native CUDA llama-server (or reuses pre-built binary), applies Layer Pipelining across Dual T4s with Flash Attention & 64K context, creates a Cloudflare tunnel, and connects to this control plane.

02_serve_qwen_llamacpp.py — Native C++ Dual-T4 Server

📦 Step 3: One-Time Model Setup (01_download_gguf_and_create_kaggle_dataset.py)

Run this once in any notebook to download the pre-quantized Unsloth UD-Q6_K GGUF (~22.5 GB) directly to Kaggle:

Run the download script below in a Kaggle notebook.

Click Save Version → Save & Run All (Commit).

After completion, open the notebook output and click + New Dataset → Name it qwen38-27b-gguf. Attach it to your serving notebooks to boot in seconds!

01_download_gguf_and_create_kaggle_dataset.py

💻 Step 4: Connect Applications (IDE, Cursor, Python, cURL)

Python (OpenAI SDK 1.0+) with Streaming Reasoning

cURL Streaming Request

Cursor / Continue.dev / Cline Custom Model Configuration

LangChain ChatOpenAI Example

🛡️ Cluster Admin Sign In