Sajjad Pakdamansavoji

savoji

https://sajjadpsavoji.github.io

AI & ML interests

VLM, VLA, LLM, Robotics, Computer Vision

Recent Activity

posted an update about 5 hours ago

Cutting an agent's context window from 256K to 16K decodes 3.6x to 7x more tokens per second on identical H100s (GLM 5.2 3.6x, Kimi K2.6 5.6x, Qwen 3 Coder Next 7x), and in the paper's tests accuracy barely moved. "You don't need long context" (arXiv 2611.05306) argues that long-horizon agents mostly need memory management, not bigger windows. Frequent compaction into a compact state, plus selective reading of inputs with bash tools, kept accuracy close to full-history baselines while contexts stayed short. 📊 The numbers • Compaction over 6 benchmarks: Opus 4.5 -3.7 to +5.6 ppts, GPT 5.6 Sol -3.3 to +4.4 versus full history • One action per context: SWE-bench Verified 70.2 for Opus 4.5 (baseline 70.0) and 82.4 for GPT 5.6 Sol (baseline 84.4), median peak context 7K to 9K tokens • Long inputs stored as files and read selectively: Opus 4.5 -0.4 to +40.0 ppts, estimated cost savings 4.4x to 233.3x across both models 🛠️ Try it • Set a compaction threshold near 12K to 36K tokens • Add a tool that reads files or history on demand • Compare accuracy, total tokens and cost against full history ⚠️ Caveats • Speedups come from open models, accuracy results from closed ones (Opus 4.5, GPT 5.6 Sol) • Cost savings are estimates, the harness is bash-only, and the preprint has no independent replication yet https://huggingface.co/papers/2611.05306 https://arxiv.org/abs/2611.05306 What would your agent need to carry in its handoff state, and what could it safely fetch later? 📚 Read it elsewhere • The deep dive on Medium: https://medium.com/p/41f69cdbe3b3 • The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/qwen-3-coder-next-decodes-7x-faster • The thread on X: https://x.com/saavoji/status/2118402118927438127 • Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#claude-opus-4

repliedto their post 1 day ago

LFM3-1.7B from @LiquidAI is the #1 trending model on the Hub (855 likes). It takes about 1 GB at 4-bit, and in the vendor's own table it beats Qwen3.5 9B on several agent benchmarks. The edge decision moves from "are small models good enough?" to "which architecture and runtime do I pick?" One 1.7B backbone now ships as text, VL-2B and Omni-2B checkpoints with open weights. 🔍 Architecture and speed • 24 layers: 13 gated convolutions (kernel 7), 5 sliding-window attention (256 tokens), 6 global attention, chosen by a search over 14 device and backend targets • llama.cpp Q4_0 at 32K context: 1.8 to 2.8x faster decode than the next model on 3 devices • Full 128K context needs 1.6 GB of KV cache vs 4.3 GB (Qwen3.5-4B) and 5.6 GB (MiniCPM5-2B) 📊 Liquid AI's table • Average 49.37 vs 47.24 for Qwen3.5 9B; BFCL V4 71.39 vs 62.83; WebArena-Verified 38.42 vs 28.94 • Qwen3.5 9B still wins SWE-bench Verified (51.20 vs 43.40) and GPQA Diamond (80.40 vs 63.37) 🛠️ Try it • GGUF (174,177 downloads last month) runs in llama.cpp b11960+, Ollama and LM Studio; MLX 4-bit for Apple Silicon; SGLang for GPUs • llama-cli -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M --temp 0.1 --top-k 50 --repeat-penalty 1.1 • Omni-2B runs in the browser in the WebGPU Space ⚠️ Caveats • The numbers are vendor-reported, with no independent reproduction in the sources. The card says it is not for agentic coding or complex multi-step reasoning. https://huggingface.co/LiquidAI/LFM3-1.7B https://huggingface.co/LiquidAI/LFM3-1.7B-GGUF https://huggingface.co/LiquidAI/LFM3-Omni-2B https://huggingface.co/spaces/LiquidAI/LFM3-Omni-2B-WebGPU Has a sub-2B model held up on your own tool-calling harness? • Medium: https://medium.com/p/05507e4e81c8 • Substack: https://sajjadpakdamansavoji.substack.com/p/liquid-ais-lfm3-turns-the-small-model • X: https://x.com/saavoji/status/2117307514177941672 • Website: https://sajjadpsavoji.github.io/blog#liquid-ais-lfm3

repliedto their post 1 day ago

Open-weight Qwen3.8-Flash-Next hits a RoboDojo Score of 31.39 with no depth or task-verifier feedback, above GPT-6 Astra's published 28.97 and every listed VLA/WAM baseline. The lever is the harness. The same Qwen scores 1.90 with RoboProbe, 7.75 with RPent, 21.01 with GAP (Hybrid) and 31.39 with uni-robot-agent. Under that harness GPT-6 Astra reaches 51.90, so a gap to closed models remains. 📊 The numbers • Qwen costs $1.09 per episode; Astra costs $21.29 • Best VLA/WAM baseline, Xiaomi-Robotics-1: 20.07 Score • Real robot, 19 tabletop tasks: Qwen 49.50, fine-tuned pi0.5 47.75, zero-shot Dream-1 35.50 • No depth keeps 87.5% of Qwen's score; RGB only without camera parameters drops it to 9.13 🛠️ Try it • pip install uni-robot-agent, then uni-robot-agent setup --integration robodojo (MIT, Python 3.11, Linux GPU host) • Reusable ideas: immutable observation snapshots, duplicate-action rejection, pipelined execution (84.5% to 93.8% of motion overlaps model calls) ⚠️ Caveats • Scores are low in absolute terms; Qwen needs 95.5 min to reach a score of 25, Astra 42.6 • The package has no physical-robot support yet; the real-world benchmark is in-house on one dual-arm platform https://huggingface.co/papers/2611.01560 https://github.com/Open-RUA/uni-robot-agent https://arxiv.org/abs/2611.01560 Before buying depth sensors or a frontier API, how much of your robot agent's gap is really the harness? 📚 Read it elsewhere • The deep dive on Medium: https://medium.com/p/51efcc243344 • The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/open-qwen38-flash-next-beats-gpt • The thread on X: https://x.com/saavoji/status/2117670449111007255 • Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#qwen3-8-flash

View all activity

Organizations

None yet

Posts 11

Post

Cutting an agent's context window from 256K to 16K decodes 3.6x to 7x more tokens per second on identical H100s (GLM 5.2 3.6x, Kimi K2.6 5.6x, Qwen 3 Coder Next 7x), and in the paper's tests accuracy barely moved.

"You don't need long context" (arXiv 2611.05306) argues that long-horizon agents mostly need memory management, not bigger windows. Frequent compaction into a compact state, plus selective reading of inputs with bash tools, kept accuracy close to full-history baselines while contexts stayed short.

📊 The numbers
• Compaction over 6 benchmarks: Opus 4.5 -3.7 to +5.6 ppts, GPT 5.6 Sol -3.3 to +4.4 versus full history
• One action per context: SWE-bench Verified 70.2 for Opus 4.5 (baseline 70.0) and 82.4 for GPT 5.6 Sol (baseline 84.4), median peak context 7K to 9K tokens
• Long inputs stored as files and read selectively: Opus 4.5 -0.4 to +40.0 ppts, estimated cost savings 4.4x to 233.3x across both models

🛠️ Try it
• Set a compaction threshold near 12K to 36K tokens
• Add a tool that reads files or history on demand
• Compare accuracy, total tokens and cost against full history

⚠️ Caveats
• Speedups come from open models, accuracy results from closed ones (Opus 4.5, GPT 5.6 Sol)
• Cost savings are estimates, the harness is bash-only, and the preprint has no independent replication yet

You don't need long context (2611.05306)
https://arxiv.org/abs/2611.05306

What would your agent need to carry in its handoff state, and what could it safely fetch later?

📚 Read it elsewhere
• The deep dive on Medium: https://medium.com/p/41f69cdbe3b3
• The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/qwen-3-coder-next-decodes-7x-faster
• The thread on X: https://x.com/saavoji/status/2118402118927438127
• Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#claude-opus-4

Post

All 1,125 turns in eight weight-space experiments with OpenAI's gpt-6-astra are logged and audited as tool-free, and the repo already has 877 stars and 109 forks.

The logs support "weights are an interface worth probing". They do not support "LLMs natively program neural networks". Every prompt, reply, code run and evaluation is published, so you can check where the line falls. The paper has 12,564 alphaXiv visits and 511 votes this week.

🔍 What was tested
• Reading: which game a network was trained on (Exp 1), what a diffusion model generates (Exp 2), which language an LM learned (Exp 3)
• Editing: fix a failing network (Exp 4), improve Qwen3.5-0.8B at math with 10, 25 and 50 passes (Exp 8)
• Writing: MinAtar game networks (Exp 5), a 1.5M-weight Go network (Exp 6)

⚠️ Caveats
• Exps 1, 2, 4, 5 had no tools. Exps 3, 6, 7, 8 used a no-network bubblewrap sandbox, one Python block per turn
• Effort differs: high (1, 2, 4), xhigh (3, 5, 6, 7), max (8), so results are not like-for-like
• One closed model, the authors' own runs, and trial runs of earlier prompts are left out

🛠️ Audit it yourself
• Check audit.tool_free in each turn's run.json and the raw events.jsonl
• Open exp8_math: prompts, tool_calls.jsonl, eval files. Edits ship as exact diffs against the public base weights, with an apply script

https://github.com/RikiyaT/lm_understand-logs

https://huggingface.co/papers/2611.weight-space-programming

Which experiment would you rerun first with an open-weight model?

📚 Read it elsewhere
• The deep dive on Medium: https://medium.com/p/3ed2af66a0c9
• The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/openais-gpt-6-astra-reads-network
• The thread on X: https://x.com/saavoji/status/2118036030452650268
• Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#openais-gpt-6-10