I’m trying to compare OpenAI, Claude, Grok, Perplexity, and DeepSeek for a small set of research and writing tasks. The comparison gets messy when one assistant can search the web, another has different tools enabled, and their answers are formatted differently.

For a useful cross-provider evaluation set, would you test each service with its native features enabled, or keep tools off and compare only the underlying responses? I’d also like to know how people account for model updates without rerunning every test constantly.