Skip to content

Lesson 4.5 · 4:38

A fair model test

Goal: Compare two models on *your* task fairly: same task, same files, same checks.

Try it

  1. Pick one tiny task with three acceptance checks (reuse 4.2's).
  2. Make sure your app is at a save point, then copy the folder twice: app-A, app-B.
  3. Open OpenCode in app-A, pick model A with /models, paste the prompt. Stop at 5 minutes.
  4. Open a fresh OpenCode session in app-B, pick model B, paste the same prompt. Stop at 5 minutes.
  5. Run the same three checks on both and fill in the table.

You're done when your table is filled in with only things you saw, and you've written one sentence starting "For this task, I'll use…".

Recap

  • "Best model" only means something for a specific task.
  • Fair test: same task, same starting files, same checks, same time, fresh sessions. Only the model changes.
  • One small test supports a choice for this task, not a universal ranking. Write "unknown" instead of guessing.

Next lesson (5.1): Module 5 begins. Meet a practice app you're allowed to break.