LLM Engineer and Manager

I am on a fellowship at Handshake AI, running human-in-the-loop evaluation of a leading large language model as it learns how professional design software actually works.

The work has two halves. One is producing expert-level reference deliverables in industry-standard creative tools, 2D graphics, CAD, and game development, so there is something correct for a model to learn from. The other is writing and evaluating the prompts that test whether it learned anything, across real workflows rather than toy examples.

I also author the rubrics that define what quality means for a professional design task, which turns out to be the hard part. Saying an illustration is good is easy. Saying precisely what would make it wrong, in terms a model and a reviewer can both apply, is not.

Alongside my own output I oversee other designers' work, reviewing deliverables and giving the feedback that keeps the reference set consistent.