reyhan bayraktar

GenAI Engineer · LLM inference, evaluation and agents

I serve large language models on hardware I can touch, and I measure them until the numbers hold up. Many of my favourite results are speedups that turned out not to exist.

At NewMind AI in Istanbul I built the evaluation system for a production agent platform and brought up around twenty open-weight model families on H200, GB300 and DGX Spark.

selected work

2026 · eval

mecellem-legal-bench

An open benchmark that asks whether models apply the law or just cite it. 42 Turkish legal practice questions, 367 rubric criteria, an LLM judge, and a public leaderboard of 16 frontier models. dataset · leaderboard

judge agreement with reference verdicts → ~90% · scored pass^3, infra failures kept out
2026 · inference

Benchmarks that lie

A reported 4–5× speculative-decoding speedup turned out to be a comparison against a baseline with CUDA graphs switched off. Same story with a model I had written off as slow. I now check the flags before I believe the number.

--enforce-eager: ~15 tok/s → graphs on: 147 tok/s
2026 · agents

Grading agents, not answers

Trajectory-level evaluation for a production agent platform on MCP tools. Each question runs k times, each run is traced end to end, and each step is graded: which tool, in what order, whether the citations are real, and what it cost. It became the release gate for new tool servers.

every run traced · every tool call checked · pass^k

also

publications

experience