Synthetic evaluation examples
Six fictional roles. Different asset-planning needs.
See how a broad request could become individual asset instructions and review criteria. Explore different game constraints and follow-up requests.
What a real evaluation would check
The constraints and checks below are evaluation targets. They do not establish that a planned file was generated or passed inspection.
- Check that individual instructions preserve the brief’s requirements.
- Inspect actual generated files, meshes, materials and engine behavior separately.
- Keep consented real-user feedback and provider, model, timing and cost records in separate evidence.
SIM-01Pixel-game size and palette consistency
Solo pixel-game developer — fictional role
Pixel-game size and palette consistency
Solo pixel-game developer — fictional role
A fictional developer is organizing the first asset batch for a small forest exploration game. This does not describe a real game or participant.
Illustrative input brief
Plan an idle character, a forest berry icon, and a grass floor tile for a forest exploration scene. Leave animation for a later task; only still images are needed now.
Simulated request
Simulated request: Design the first image batch so character scale stays consistent.
Authored plan example
Authored plan example: Plan one idle character, one berry icon, and one seamless floor tile as separate PNGs, with dimension and palette checks for each file.
Simulated follow-up
Simulated follow-up: Keep large decorations out of the floor tile and inspect nearest-neighbor enlargement after real generation.
Illustrative production plan
Requested limit: 3 assets. Filenames below are proposed outputs, not actual files.
Top-down pixel art with a defined 12-color palette, upper-right lighting, and a one-pixel outline. Preserve pixel edges when enlarged.
forest_runner_idle.pngPlayer idle pose for the exploration scene
Plan a 32×32 transparent idle character. Align the feet to bottom center and use the chosen 12-color palette with a one-pixel outline.
- The PNG must be 32×32 with a transparent background.
- Compare color samples with the agreed palette and inspect for blur or semi-transparent edges.
- Inspect the silhouette and foot anchor at 4× nearest-neighbor enlargement.
forest_berry_icon.pngRecognizable berry icon for the inventory
Place a single berry cluster in a 16×16 transparent PNG. Use no text and keep the character's palette and lighting direction.
- The file must be 16×16 without clipping the icon.
- A reviewer checks whether the berry remains recognizable at native size on a dark UI background.
forest_grass_tile.pngRepeatable ground for the exploration map
Plan an opaque 32×32 grass floor tile. Match edge brightness and patterns for tiling; omit large objects and text.
- Inspect seams and obvious repeating patches in a 3×3 tiled preview.
- Check that tile contrast stays below the character outline in a real composite scene.
Post-production review checklist
- Review dimensions, alpha, and palette across all files.
- Check that the still-image plan does not promise animation frames or unsupported capabilities.
Evaluation criteria
- Evaluate whether size, palette, and lighting requirements remain consistent across the three files.
- The plan must allow seams and alpha to be rechecked on actual files after generation.
Questions to investigate
- Do not assume an image generator will obey pixel-level constraints exactly.
- Keep acceptance of the plan separate from actual display in a game engine.
Proposed response to the follow-up
Proposed revision: Reduce ground contrast and add a nearest-neighbor review. This revision has not been generated or validated.
Observed results, satisfaction, processing time and token cost: not measured. This record cannot establish real user testing or Claude execution.
SIM-02Low-poly prop budgets for mobile
Mobile low-poly developer — fictional role
Low-poly prop budgets for mobile
Mobile low-poly developer — fictional role
A fictional developer is planning readable props for a small mobile delivery game.
Illustrative input brief
Plan a delivery crate, a stop sign, and a transport cart. Collision shapes will be made separately in the engine; each GLB should contain only the visual model.
Simulated request
Simulated request: I want polygon and material limits defined at the planning stage for mobile props.
Authored plan example
Authored plan example: Specify a triangle cap and dimensions for each of three GLB props. Reserve collider and frame-rate checks for later engine validation.
Simulated follow-up
Simulated follow-up: Keep the cart handle readable, but leave wheel animation outside this scope.
Illustrative production plan
Requested limit: 3 assets. Filenames below are proposed outputs, not actual files.
Use low polygon counts, softened corners, opaque solid-color PBR materials, meters, Y-up, and a bottom-center origin. Prioritize silhouettes on small screens.
delivery_crate.glbSilhouette for the delivery target
Plan a 0.6×0.5×0.5m crate with no more than two opaque solid-color materials, at most 600 triangles, and a bottom-center origin.
- Check that the exported GLB contains at most 600 triangles and two materials.
- Check bounds, units, origin, and silhouette from the default camera.
delivery_stop_sign.glbProp marking a delivery stop
Plan a 1.2m sign using a large symbol instead of text, with at most two opaque materials and 450 triangles.
- Review whether the sign symbol is legible in an actual mobile preview.
- Check the final GLB's triangle and material caps and the pole's ground contact.
delivery_cart.glbStatic cart illustrating the delivery route
Plan a static 1m-long cart with distinct handle and wheel shapes, at most three materials and 1,200 triangles. Do not include rigging or animation.
- Verify the target caps of 1,200 triangles and three materials on the actual file.
- The handle, cargo area, and wheels should have distinct silhouettes in a small camera view.
Post-production review checklist
- Use consistent scale, axes, and origin rules across all props.
- Perform later material compatibility and rendering-cost checks on a real device. This document contains no performance measurements.
Evaluation criteria
- Evaluate whether budgets, materials, and camera readability can be checked per prop.
- Evaluate whether manual engine review is clearly separated from production-tool capabilities.
Questions to investigate
- Triangle counts alone cannot guarantee mobile performance.
- The cart is a static visual model, not a physics or driving implementation.
Proposed response to the follow-up
Proposed revision: Emphasize the cart handle's silhouette check and exclude wheel animation. No performance improvement has been observed.
Observed results, satisfaction, processing time and token cost: not measured. This record cannot establish real user testing or Claude execution.
SIM-03GLB export review and source preservation
Technical artist reviewing 3D exports — fictional role
GLB export review and source preservation
Technical artist reviewing 3D exports — fictional role
A fictional technical artist is defining engine-handoff rules while preserving prop sources. This is not a record of defects discovered in actual files.
Illustrative input brief
Plan newly named review GLB versions of a shelf and workbench. Check negative scale, flipped faces, missing external textures, and origin position without overwriting sources.
Simulated request
Simulated request: Outline a review sequence for possible engine differences in existing GLBs. Do not alter sources.
Authored plan example
Authored plan example: Use new _review_v01 filenames and plan transform, face orientation, material, and bounds checks. Record defects only after inspecting actual files.
Simulated follow-up
Simulated follow-up: Separate automatically verified checks from checks a person must perform in the engine.
Illustrative production plan
Requested limit: 2 assets. Filenames below are proposed outputs, not actual files.
Low-poly indoor props using meters, Y-up, and opaque PBR. Height and ground contact must match the engine grid.
indoor_shelf_review_v01.glbEngine-handoff review version of the shelf
Plan a new export version of a 1.6m shelf, targeting at most 1,000 triangles. Review transforms and face orientation and embed required resources in the GLB.
- Verify a distinct path and filename and check that the source remains unchanged.
- Check GLB validity, embedded resource references, triangle count, and finite bounds.
- Also inspect face orientation and ground contact in a real viewer and target engine.
indoor_workbench_review_v01.glbWorkbench version for unit and material review
Plan a separate GLB version of a 1.4×0.85×0.7m workbench, targeting at most 1,200 triangles. Review opaque PBR materials, meter units, and a bottom-center origin.
- Check bounds against target dimensions and inspect for negative scale.
- Inspect materials, alpha modes, and resource references, then compare visual results in the actual engine.
- Leave undiagnosable checks unverified instead of marking them passed.
Post-production review checklist
- Record source paths, new output paths, file hashes, and review status during a real run.
- If Blender is used, disable script auto-execution and do not execute generated code as an asset input.
- Record automated structural checks separately from visual engine checks.
Evaluation criteria
- Evaluate whether the plan supports source preservation, versioned outputs, and unverified states.
- Evaluate whether structural facts are separated from judgments requiring visual review.
Questions to investigate
- A valid GLB structure does not guarantee visual quality in a game engine.
- No actual files were inspected, so this example contains no defect diagnosis or passed review record.
Proposed response to the follow-up
Proposed revision: Include an unverified review state and separate engine checks. This does not mean an actual defect was fixed.
Observed results, satisfaction, processing time and token cost: not measured. This record cannot establish real user testing or Claude execution.
SIM-04Separating multilingual briefs from image instructions
Korean/English content designer — fictional role
Separating multilingual briefs from image instructions
Korean/English content designer — fictional role
A fictional designer is translating Korean design intent into English asset instructions while separating UI copy from image production.
Illustrative input brief
Plan a quest-board background, map icon, and crafting-material icon in Korean and English. Keep text out of images and manage UI copy in separate game string resources.
Simulated request
Simulated request: I want map and material meanings preserved when translating the Korean brief into English.
Authored plan example
Authored plan example: Plan three text-free PNGs with paired Korean/English instructions. Leave copy translation and cultural suitability for separate human review.
Simulated follow-up
Simulated follow-up: Leave the board center clear for longer translations and avoid a country-specific outline in the map icon.
Illustrative production plan
Requested limit: 3 assets. Filenames below are proposed outputs, not actual files.
Warm hand-painted fantasy UI with teal and sand colors and consistent outline thickness. Do not rely on flags, language-specific text, or color alone to convey meaning.
quest_board_background.pngQuest-board background for multilingual UI copy
Plan a 1,024×768 PNG with the central 60% left free of decoration. Include only fantasy board material and borders, with no text or text-like marks.
- Check the requested dimensions and clear central region on the actual output.
- Overlay Korean and English UI strings separately to check contrast and overlap.
- Inspect for unreadable pseudo-text or watermarks.
quest_map_icon.pngIcon representing quest navigation
Plan a folded map and location pin in a 128×128 transparent PNG. Avoid country outlines and text, and keep the shared outline style.
- Compare Korean and English instructions for the same visual intent.
- Check whether the map and pin remain recognizable in a grayscale preview.
quest_material_bundle.pngIcon representing a crafting-material list
Plan a wood-and-metal material bundle in a 128×128 transparent PNG. Match the map icon's line weight and lighting and communicate without text.
- Both instruction languages must include the same materials and constraints.
- Review size, line weight, and lighting consistency beside the map icon.
Post-production review checklist
- Compare materials, dimensions, and excluded elements across both languages.
- Record copy translation, visual meaning, and cultural suitability as separate reviews.
Evaluation criteria
- Evaluate whether both languages preserve constraints and keep text separate from images.
- Evaluate whether longer translations and grayscale readability can be reviewed later.
Questions to investigate
- Writing English instructions does not validate translation quality or cultural suitability.
- Text artifacts and UI contrast must be checked on real images and game screens.
Proposed response to the follow-up
Proposed revision: Specify the clear board center and a country-neutral map. No translation or image-output success has been measured.
Observed results, satisfaction, processing time and token cost: not measured. This record cannot establish real user testing or Claude execution.
SIM-05Scope control and handoff for a game-jam team
Small game-jam team — fictional role
Scope control and handoff for a game-jam team
Small game-jam team — fictional role
A fictional small team is preparing essential props in a shared style under a short production schedule. This is not a record of a real team's schedule or results.
Illustrative input brief
Plan only a cargo crate, a goal beacon, and a floor pattern for a small robot-delivery scene. Exclude the robot, rigging, and music; all three files must support independent handoff.
Simulated request
Simulated request: We need a minimal batch that keeps style consistent across independently assigned work.
Authored plan example
Authored plan example: Fix the scope at three files with shared color, unit, and naming rules. Independent work is a design intention, not a record of a parallel run.
Simulated follow-up
Simulated follow-up: A static beacon is enough. Leave flashing and victory effects for the engine.
Illustrative production plan
Requested limit: 3 assets. Filenames below are proposed outputs, not actual files.
Blocky sci-fi props with teal accents, gray bases, and simple silhouettes. Models use meters, Y-up, and bottom-center origins; the floor pattern contains no text.
jam_cargo_crate.glbMovable target in the delivery game
Plan a blocky 0.7m crate using the shared teal accent and opaque materials, targeting at most 900 triangles.
- Check units, origin, and triangle cap on the final file.
- Compare accent color and silhouette with the beacon in one scene.
jam_goal_beacon.glbStatic beacon marking the delivery goal
Plan a static 1.1m beacon with at most 800 triangles. Use basic PBR color without emission for the goal silhouette; exclude animation.
- Check that the goal silhouette remains recognizable without animation or emission.
- Verify bottom-center origin and ground contact in the actual scene.
jam_floor_route.pngFloor pattern suggesting a delivery route
Plan a simple direction pattern as an opaque 512×512 PNG. Use the shared teal-and-gray palette and omit text and product logos.
- Check file dimensions and opaque alpha.
- Review whether the route remains readable from a perspective camera without competing with props.
Post-production review checklist
- Include filenames, formats, shared colors, and size constraints in each handoff.
- Review style and placement in a scene containing all three actual files.
Evaluation criteria
- Evaluate whether per-prop instructions remain independent while preserving shared style constraints.
- Evaluate whether the plan avoids adding features or assets beyond the three essentials.
Questions to investigate
- Splitting a plan is not evidence of successful parallel execution or measured time savings.
- Runtime effects, collision, and interaction remain outside visual-asset scope.
Proposed response to the follow-up
Proposed revision: Specify a static beacon and move victory effects to later engine work. Parallel execution and time savings have not been measured.
Observed results, satisfaction, processing time and token cost: not measured. This record cannot establish real user testing or Claude execution.
SIM-06Offline educational production and privacy boundaries
Privacy-conscious educator — fictional role
Offline educational production and privacy boundaries
Privacy-conscious educator — fictional role
A fictional educator is using de-identified briefs and local production because classroom material may contain personal data. No actual students or classroom materials are used.
Illustrative input brief
Plan a shareable exercise kit without names, photos, or original student work. Only a simple block-sample GLB and unlabeled palette PNG are needed. Do not transmit inputs to an external AI service.
Simulated request
Simulated request: I need a production plan that does not send classroom sources or student material outside the device.
Authored plan example
Authored plan example: Plan only two local/manual files from a non-personal text brief. Do not run remote Claude calls in this scenario; mark remote features as incompatible with its constraint.
Simulated follow-up
Simulated follow-up: After a real export, I also want to check for hidden author names or local paths in shared files.
Illustrative production plan
Requested limit: 2 assets. Filenames below are proposed outputs, not actual files.
Basic geometry and clear color contrast using local or manual production. Exclude student data, school logos, real photos, and people; use shape as well as color for distinction.
classroom_shape_blocks.glbExercise sample for comparing shapes without personal data
Plan local/manual creation of a static GLB with basic 0.3m block shapes, targeting at most 600 triangles. Include no people, names, or school identifiers.
- Inspect for personal names, school identifiers, external URIs, and unnecessary author metadata.
- Review GLB structure, triangle count, units, and shape distinction on the actual file.
- Do not mark offline validation complete before checking the network behavior of every tool used.
classroom_palette_tiles.pngExercise palette for discussing color contrast alongside shapes
Plan unlabeled color-and-pattern tiles in a 256×256 PNG using a local editor. Include no names, photos, QR codes, or school logos; do not encode answers through color alone.
- Check dimensions, pattern distinction, and readability in grayscale.
- Inspect PNG text metadata for personal information or local paths before distribution.
Post-production review checklist
- De-identify inputs before a real run, preserve originals, and use new output paths.
- Identify network-using operations such as downloads, updates, and remote generation separately before classroom production.
- Inspect contents, metadata, and external references of actual shared files before distribution.
Evaluation criteria
- Evaluate whether the no-external-transfer constraint takes precedence and incompatible features are identified.
- Evaluate whether file contents and metadata can be reviewed before actual sharing.
Questions to investigate
- Remote Claude API calls are incompatible with a no-external-transfer requirement. This example neither runs them nor claims local Claude execution.
- This fictional design does not certify privacy protection or legal compliance.
Proposed response to the follow-up
Proposed revision: Add metadata review and per-tool network checks before sharing. No actual personal data, transfer logs, or classroom outcomes were collected.
Observed results, satisfaction, processing time and token cost: not measured. This record cannot establish real user testing or Claude execution.