How do you evaluate a new language model when no benchmark fits your task?

I'm working on a task where the standard benchmarks don't really capture what I care about, and building a new one feels like a project in itself.

I'm curious how others decide when a custom evaluation is rigorous enough to publish.

What has worked for you when the existing benchmarks fall short?