ZZZBenchmarks

Harness benchmark / 2026-11-04

GPT-5.6 Terra Custom skills supplied: None

Time taken

Execution time, seconds

  1. Codex49.9s
  2. OpenCode59.5s
  3. Hermes80.8s
  4. Prime Agent84.9s
  5. Pi94.9s
  6. JetBrains Junie100.4s
  7. Cursor Cloud153.4s
  8. Factory Droid168.4s

Cost

Priced token subtotal, USD · * incomplete · — unknown

  1. Codex$0.1094
  2. Hermes$0.1647*
  3. OpenCode$0.1734
  4. Prime Agent$0.2102
  5. JetBrains Junie$0.2838
  6. Pi$0.3069*
  7. Factory Droid$0.4742*
  8. Cursor Cloud—
Grocery app benchmark results
HarnessRunTotalInput tokensOutput tokensCache readReasoningOriginal previewPublication check
CodexComplete49.9s59.4s16,8205,41553,737$0.1094MediumPublishedAdd + persistence passed
OpenCodeComplete59.5s188.6s29,5785,949224,185$0.1734Not exposedPublishing failedAdd + persistence passed
HermesComplete80.8s234.1s25,0535,735228,921$0.1647*Not exposedPublishedStartup error
Prime AgentComplete84.9s280.7s35,6566,788287,166$0.2102MediumPublishedAdd + persistence passed
PiComplete94.9s105.2s57,5948,294460,836$0.3069*MediumPublishedAdd + persistence passed
JetBrains JunieComplete100.4s124.8s49,02610,532296,871$0.2838Not exposedPublishing failedAdd + persistence passed
Cursor CloudComplete153.4s216.0sUnknownUnknownUnknownUnknownNot exposedNo public URLAdd + persistence passed
Factory DroidComplete168.4s184.9s37,09726,000439,953$0.4742*Not exposedPublishing failedAdd + persistence passed
Download results JSON ↗

Sorted by time taken, fastest first.

Codex grocery app

Codex

49.9s · $0.1094

Add + persistence passed

Agent reported 2 passing tests.

OpenCode grocery app

OpenCode

59.5s · $0.1734

Add + persistence passed

Agent reported 2 passing tests. Usage includes GPT-5.4 Nano: 1,037 input and 13 output tokens. Public preview needed separate recovery.

Hermes grocery app

Hermes

80.8s · $0.1647*

Startup error

Agent reported static core-flow and HTTP checks. Publication check: fresh storage triggers “Cannot access save before initialization”. Original app preserved.

Prime Agent grocery app

Prime Agent

84.9s · $0.2102

Add + persistence passed

Agent reported syntax and local checks. Usage was provisional in the first summary and has since settled.

Pi grocery app

Pi

94.9s · $0.3069*

Add + persistence passed

Agent reported 3 passing tests. Editing uses native prompt dialogs.

JetBrains Junie grocery app

JetBrains Junie

100.4s · $0.2838

Add + persistence passed

Agent reported 4 passing tests. Public preview needed separate recovery.

Cursor Cloud grocery app

Cursor Cloud

153.4s · Unknown

Add + persistence passed

Agent reported syntax and local HTTP checks. Used Cursor’s native cloud environment. Usage returned zeros and is treated as unknown. Public preview needed separate recovery.

Factory Droid grocery app

Factory Droid

168.4s · $0.4742*

Add + persistence passed

Agent reported syntax and local HTTP checks. Public preview needed separate recovery.