Tools

Free, no-signup tools for people shipping LLM systems. The rule for what earns a spot: it has to do something you would not paste into a chatbot, either because the data is sensitive (so it runs entirely in your browser and never leaves the tab) or because it leans on data I maintain and measure by hand. Nothing is logged.

alexander@aros:~$ ls tools/ --sort value

LLM Bill Analyzer llm-bill-analyzer.html

Drop your real Anthropic or OpenAI usage export. Get spend by model and day, your actual cache hit rate, what caching would have saved, and what routing to a cheaper model is worth. Your CSV never leaves the tab.

Prompt Cache Linter prompt-cache-linter.html

Paste two consecutive request payloads and find the exact character where your prompt cache breaks, what broke it, and what the miss costs per month. The tool your bill wishes you had.

LLM Price Tracker llm-price-tracker.html

Hand-verified prices across three providers with a dated event log, blended cost per workload shape, CSV and JSON export, and an RSS feed that fires only when a price actually moves.

Chunking Benchmark Lab chunking-benchmark-lab.html

Real measured data: 800 questions, three splitters, recall@k and duplication, from a published seeded script. Then the same chunkers and metrics on your own pasted corpus, in your browser.

Eval Sample Size Calculator eval-sample-size.html

How many eval cases to detect a real difference, with paired designs and judge-noise attenuation. And the inverse: what your current 50-case suite can actually see (brace yourself).

RAG Chunking Visualizer chunk-visualizer.html

Paste a document, pick a strategy and overlap, and see every boundary before your embedding model does. The qualitative companion to the Benchmark Lab's numbers.

These pair with the essays: the bill analyzer and cache linter come from Prompt Caching Is Free Money, the sample size calculator from Your LLM Evals Are Lying to You, and the benchmark lab from Chunking Strategies, Measured. Requests and bug reports: LinkedIn. The five lead tools are open source with tests and CLIs under github.com/AlexRyan92: audit the engine, run it in CI, or file a correction.