Agent SkillsAgent Skills
CharlesWiltgen

axiom-realitykit-diag

@CharlesWiltgen/axiom-realitykit-diag
CharlesWiltgen
759
61 forks
Updated 4/7/2026
View on GitHub

Use when RealityKit entities not visible, anchors not tracking, gestures not responding, performance drops, materials wrong, or multiplayer sync fails

Installation

$npx agent-skills-cli install @CharlesWiltgen/axiom-realitykit-diag
Claude Code
Cursor
Copilot
Codex
Antigravity

Details

Path.claude-plugin/plugins/axiom/skills/axiom-realitykit-diag/SKILL.md
Branchmain
Scoped Name@CharlesWiltgen/axiom-realitykit-diag

Usage

After installing, this skill will be available to your AI coding assistant.

Verify installation:

npx agent-skills-cli list

Skill Instructions


name: axiom-realitykit-diag description: Use when RealityKit entities not visible, anchors not tracking, gestures not responding, performance drops, materials wrong, or multiplayer sync fails license: MIT metadata: version: "1.0.0"

RealityKit Diagnostics

Systematic diagnosis for common RealityKit issues with time-cost annotations.

When to Use This Diagnostic Skill

Use this skill when:

  • Entity added but not visible in the scene
  • AR anchor not tracking or content floating
  • Tap/drag gestures not responding on 3D entities
  • Frame rate dropping or stuttering
  • Material looks wrong (too dark, too bright, incorrect colors)
  • Multiplayer entities not syncing across devices
  • Physics bodies not colliding or passing through each other

For RealityKit architecture patterns and best practices, see axiom-realitykit. For API reference, see axiom-realitykit-ref.


Mandatory First Step: Enable Debug Visualization

Time cost: 10 seconds vs hours of blind debugging

// In your RealityView or ARView setup
#if DEBUG
// Xcode: Debug β†’ Attach to Process β†’ Show RealityKit Statistics
// Or enable in code:
arView.debugOptions = [
    .showStatistics,       // Entity count, draw calls, FPS
    .showPhysics,          // Collision shapes
    .showAnchorOrigins,    // Anchor positions
    .showAnchorGeometry    // Detected plane geometry
]
#endif

If you can't see collision shapes with .showPhysics, your CollisionComponent is missing or misconfigured. Fix collision before debugging gestures or physics.


Symptom 1: Entity Not Visible

Time saved: 30-60 min β†’ 2-5 min

Entity added but nothing appears
β”‚
β”œβ”€ Is the entity added to the scene?
β”‚   └─ NO β†’ Add to RealityView content:
β”‚        content.add(entity)
β”‚        βœ“ Entities must be in the scene graph to render
β”‚
β”œβ”€ Does the entity have a ModelComponent?
β”‚   └─ NO β†’ Add mesh and material:
β”‚        entity.components[ModelComponent.self] = ModelComponent(
β”‚            mesh: .generateBox(size: 0.1),
β”‚            materials: [SimpleMaterial(color: .red, isMetallic: false)]
β”‚        )
β”‚        βœ“ Bare Entity is invisible β€” it's just a container
β”‚
β”œβ”€ Is the entity's scale zero or nearly zero?
β”‚   └─ CHECK β†’ Print: entity.scale
β”‚        USD models may import with unexpected scale.
β”‚        Try: entity.scale = SIMD3(repeating: 0.01) for meter-scale models
β”‚
β”œβ”€ Is the entity behind the camera?
β”‚   └─ CHECK β†’ Print: entity.position(relativeTo: nil)
β”‚        In RealityKit, -Z is forward (toward screen).
β”‚        Try: entity.position = SIMD3(0, 0, -0.5) (half meter in front)
β”‚
β”œβ”€ Is the entity inside another object?
β”‚   └─ CHECK β†’ Move to a known visible position:
β”‚        entity.position = SIMD3(0, 0, -1)
β”‚
β”œβ”€ Is the entity's isEnabled set to false?
β”‚   └─ CHECK β†’ entity.isEnabled = true
β”‚        Also check parent: entity.isEnabledInHierarchy
β”‚
β”œβ”€ Is the entity on an untracked anchor?
β”‚   └─ CHECK β†’ Verify anchor is tracking:
β”‚        entity.isAnchored (should be true)
β”‚        If using plane anchor, ensure surface is detected first
β”‚
└─ Is the material transparent or OcclusionMaterial?
    └─ CHECK β†’ Inspect material:
         If using PhysicallyBasedMaterial, check baseColor is not black
         If using blending = .transparent, check opacity > 0

Quick Diagnostic

func diagnoseVisibility(_ entity: Entity) {
    print("Name: \(entity.name)")
    print("Is enabled: \(entity.isEnabled)")
    print("In hierarchy: \(entity.isEnabledInHierarchy)")
    print("Is anchored: \(entity.isAnchored)")
    print("Position (world): \(entity.position(relativeTo: nil))")
    print("Scale: \(entity.scale)")
    print("Has model: \(entity.components[ModelComponent.self] != nil)")
    print("Children: \(entity.children.count)")
}

Symptom 2: Anchor Not Tracking

Time saved: 20-45 min β†’ 3-5 min

AR content not appearing or floating
β”‚
β”œβ”€ Is the AR session running?
β”‚   └─ For RealityView on iOS 18+, AR runs automatically
β”‚       For ARView, check: arView.session.isRunning
β”‚
β”œβ”€ Is SpatialTrackingSession configured? (iOS 18+)
β”‚   └─ CHECK β†’ Ensure tracking modes requested:
β”‚        let config = SpatialTrackingSession.Configuration(
β”‚            tracking: [.plane, .object])
β”‚        let result = await session.run(config)
β”‚        if let notSupported = result {
β”‚            // Handle unsupported modes
β”‚        }
β”‚
β”œβ”€ Is the anchor type appropriate for the environment?
β”‚   β”œβ”€ .plane(.horizontal) β†’ Need a flat surface visible to camera
β”‚   β”œβ”€ .plane(.vertical) β†’ Need a wall visible to camera
β”‚   β”œβ”€ .image β†’ Image must be in "AR Resources" asset catalog
β”‚   β”œβ”€ .face β†’ Front camera required (not rear)
β”‚   └─ .body β†’ Full body must be visible
β”‚
β”œβ”€ Is minimumBounds too large?
β”‚   └─ CHECK β†’ Reduce minimum bounds:
β”‚        AnchorEntity(.plane(.horizontal, classification: .any,
β”‚            minimumBounds: SIMD2(0.1, 0.1)))  // Smaller = detects sooner
β”‚
β”œβ”€ Is the device supported?
β”‚   └─ CHECK β†’ Plane detection requires A12+ chip
β”‚       Face tracking requires TrueDepth camera
β”‚       Body tracking requires A12+ chip
β”‚
└─ Is the environment adequate?
    └─ CHECK β†’ AR needs:
         - Adequate lighting (not too dark)
         - Textured surfaces (not blank walls)
         - Stable device position during initial detection

Symptom 3: Gesture Not Responding

Time saved: 15-30 min β†’ 2-3 min

Tap/drag on entity does nothing
β”‚
β”œβ”€ Does the entity have a CollisionComponent?
β”‚   └─ NO β†’ Add collision shapes:
β”‚        entity.generateCollisionShapes(recursive: true)
β”‚        // or manual:
β”‚        entity.components[CollisionComponent.self] = CollisionComponent(
β”‚            shapes: [.generateBox(size: SIMD3(0.1, 0.1, 0.1))])
β”‚        βœ“ Collision shapes are REQUIRED for gesture hit testing
β”‚
β”œβ”€ [visionOS] Does the entity have InputTargetComponent?
β”‚   └─ NO β†’ Add it:
β”‚        entity.components[InputTargetComponent.self] = InputTargetComponent()
β”‚        βœ“ Required on visionOS for gesture input
β”‚
β”œβ”€ Is the gesture attached to the RealityView?
β”‚   └─ CHECK β†’ Gesture must be on the view, not the entity:
β”‚        RealityView { content in ... }
β”‚            .gesture(TapGesture().targetedToAnyEntity().onEnded { ... })
β”‚
β”œβ”€ Is the collision shape large enough to hit?
β”‚   └─ CHECK β†’ Enable .showPhysics to see shapes
β”‚        Shapes too small = hard to tap.
β”‚        Try: .generateBox(size: SIMD3(repeating: 0.1)) minimum
β”‚
β”œβ”€ Is the entity behind another entity?
β”‚   └─ CHECK β†’ Front entities may block gestures on back entities
β”‚        Ensure collision is on the intended target
β”‚
└─ Is the entity enabled?
    └─ CHECK β†’ entity.isEnabled must be true
         Disabled entities don't receive input

Quick Diagnostic

func diagnoseGesture(_ entity: Entity) {
    print("Has collision: \(entity.components[CollisionComponent.self] != nil)")
    print("Has input target: \(entity.components[InputTargetComponent.self] != nil)")
    print("Is enabled: \(entity.isEnabled)")
    print("Is anchored: \(entity.isAnchored)")

    if let collision = entity.components[CollisionComponent.self] {
        print("Collision shapes: \(collision.shapes.count)")
    }
}

Symptom 4: Performance Problems

Time saved: 1-3 hours β†’ 10-20 min

Frame rate dropping or stuttering
β”‚
β”œβ”€ How many entities are in the scene?
β”‚   └─ CHECK β†’ Print entity count:
β”‚        var count = 0
β”‚        func countEntities(_ entity: Entity) {
β”‚            count += 1
β”‚            for child in entity.children { countEntities(child) }
β”‚        }
β”‚        Under 100: unlikely to be entity count
β”‚        100-500: review for optimization
β”‚        500+: definitely needs optimization
β”‚
β”œβ”€ Are mesh/material resources shared?
β”‚   └─ NO β†’ Share resources across identical entities:
β”‚        let sharedMesh = MeshResource.generateBox(size: 0.05)
β”‚        let sharedMaterial = SimpleMaterial(color: .white, isMetallic: false)
β”‚        // Reuse for all instances
β”‚        βœ“ RealityKit batches entities with identical resources
β”‚
β”œβ”€ Is a System creating components every frame?
β”‚   └─ CHECK β†’ Look for allocations in update():
β”‚        Creating ModelComponent, CollisionComponent, or materials
β”‚        every frame causes GC pressure.
β”‚        Cache resources, only update when values change.
β”‚
β”œβ”€ Are collision shapes mesh-based?
β”‚   └─ CHECK β†’ Replace generateCollisionShapes(recursive: true)
β”‚        with simple shapes (box, sphere, capsule) for dynamic entities
β”‚
β”œβ”€ Is generateCollisionShapes called repeatedly?
β”‚   └─ CHECK β†’ Call once during setup, not every frame
β”‚
β”œβ”€ Are there too many physics bodies?
β”‚   └─ CHECK β†’ Dynamic bodies are most expensive.
β”‚        Convert distant/static objects to .static mode.
β”‚        Remove physics from non-interactive entities.
β”‚
└─ Is the model polygon count too high?
    └─ CHECK β†’ Decimate models for real-time use.
         Target: <100K triangles total for mobile AR.
         Use LOD (Level of Detail) for distant objects.

Symptom 5: Material Looks Wrong

Time saved: 15-45 min β†’ 5-10 min

Colors, lighting, or textures look incorrect
β”‚
β”œβ”€ Is the scene too dark?
β”‚   └─ CHECK β†’ Missing environment lighting:
β”‚        Add DirectionalLightComponent or EnvironmentResource
β”‚        In AR, RealityKit uses real-world lighting automatically
β”‚        In non-AR, you must provide lighting explicitly
β”‚
β”œβ”€ Is the baseColor set?
β”‚   └─ CHECK β†’ PhysicallyBasedMaterial defaults to white
β”‚        material.baseColor = .init(tint: .red)
β”‚        If using a texture, verify it loaded:
β”‚        try TextureResource(named: "albedo")
β”‚
β”œβ”€ Is metallic set incorrectly?
β”‚   └─ CHECK β†’ metallic = 1.0 makes surfaces mirror-like
β”‚        Most real objects: metallic = 0.0
β”‚        Only metals (gold, silver, chrome): metallic = 1.0
β”‚
β”œβ”€ Is the texture semantic wrong?
β”‚   └─ CHECK β†’ Use correct semantic:
β”‚        .color for albedo/baseColor textures
β”‚        .raw for data textures (metallic, roughness)
β”‚        .normal for normal maps
β”‚        .hdrColor for HDR textures
β”‚
β”œβ”€ Is the model upside down or inside out?
β”‚   └─ CHECK β†’ Try:
β”‚        material.faceCulling = .none (shows both sides)
β”‚        If that fixes it, the model normals are flipped
β”‚
└─ Is blending/transparency unexpected?
    └─ CHECK β†’ material.blending
         Default is .opaque
         For transparency: .transparent(opacity: ...)

Symptom 6: Physics Not Working

Time saved: 20-40 min β†’ 5-10 min

Objects pass through each other or don't collide
β”‚
β”œβ”€ Do both entities have CollisionComponent?
β”‚   └─ NO β†’ Both sides of a collision need CollisionComponent
β”‚
β”œβ”€ Does the moving entity have PhysicsBodyComponent?
β”‚   └─ NO β†’ Add physics body:
β”‚        entity.components[PhysicsBodyComponent.self] = PhysicsBodyComponent(
β”‚            mode: .dynamic)
β”‚
β”œβ”€ Are collision groups/filters configured correctly?
β”‚   └─ CHECK β†’ Entities must be in compatible groups:
β”‚        Default: group = .default, mask = .all
β”‚        If using custom groups, verify mask includes the other group
β”‚
β”œβ”€ Is the physics mode correct?
β”‚   β”œβ”€ Two .static bodies β†’ Never collide (both immovable)
β”‚   β”œβ”€ .dynamic + .static β†’ Correct (common setup)
β”‚   β”œβ”€ .dynamic + .dynamic β†’ Both move on collision
β”‚   └─ .kinematic + .dynamic β†’ Kinematic pushes dynamic
β”‚
β”œβ”€ Is the collision shape appropriate?
β”‚   └─ CHECK β†’ .showPhysics debug option
β”‚        Shape may be too small, offset, or wrong type
β”‚
└─ Are entities on different anchors?
    └─ CHECK β†’ "Physics bodies and colliders affect only
         entities that share the same anchor" (Apple docs)
         Move entities under the same anchor for physics interaction

Symptom 7: Multiplayer Sync Issues

Time saved: 30-60 min β†’ 10-15 min

Entities not appearing on other devices
β”‚
β”œβ”€ Does the entity have SynchronizationComponent?
β”‚   └─ NO β†’ Add it:
β”‚        entity.components[SynchronizationComponent.self] =
β”‚            SynchronizationComponent()
β”‚
β”œβ”€ Is the MultipeerConnectivityService set up?
β”‚   └─ CHECK β†’ Verify MCSession is connected before syncing
β”‚
β”œβ”€ Are custom components Codable?
β”‚   └─ NO β†’ Non-Codable components don't sync
β”‚        struct MyComponent: Component, Codable { ... }
β”‚
β”œβ”€ Does the entity have an owner?
β”‚   └─ CHECK β†’ Only the owner can modify synced properties
β”‚        Request ownership before modifying:
β”‚        entity.requestOwnership { result in ... }
β”‚
└─ Is the entity anchored?
    └─ CHECK β†’ Unanchored entities may not sync position correctly
         Use a shared world anchor for reliable positioning

Common Mistakes

MistakeTime CostFix
No CollisionComponent on interactive entity15-30 minentity.generateCollisionShapes(recursive: true)
Missing InputTargetComponent on visionOS10-20 minAdd InputTargetComponent()
Gesture on wrong view (not RealityView)10-15 minAttach .gesture() to RealityView
Entity scale wrong for USD model15-30 minCheck units: meters vs centimeters
No lighting in non-AR scene10-20 minAdd DirectionalLightComponent
Storing entity refs in System30-60 min crash debuggingQuery with EntityQuery each frame
Components not registered10-15 minCall registerComponent() in app init
Systems not registered10-15 minCall registerSystem() before scene load
Physics across different anchors20-40 minPut interacting entities under same anchor
Calling generateCollisionShapes every framePerformance degradationCall once during setup

Diagnostic Quick Reference

SymptomFirst CheckTime Saved
Not visibleHas ModelComponent? Scale > 0?30-60 min
No gesture responseHas CollisionComponent?15-30 min
Not trackingAnchor type matches environment?20-45 min
Frame dropsEntity count? Resource sharing?1-3 hours
Wrong colorsHas lighting? Metallic value?15-45 min
No collisionBoth have CollisionComponent? Same anchor?20-40 min
No syncSynchronizationComponent? Codable?30-60 min
Sim OK, device crashMetal features? Texture format?15-30 min

Symptom 8: Works in Simulator, Crashes on Device

Time cost: 15-30 min (often misdiagnosed as model issue)

Q1: Is the crash a Metal error (MTLCommandBuffer, shader compilation)?
β”œβ”€ YES β†’ Simulator uses software rendering, device uses real GPU
β”‚   Common causes:
β”‚   - Custom Metal shaders with unsupported features
β”‚   - Texture formats not supported on device GPU
β”‚   - Exceeding device texture size limits (max 8192x8192 on older)
β”‚   Fix: Check device GPU family, use supported formats
β”‚
└─ NO β†’ Check next

Q2: Is it an out-of-memory crash?
β”œβ”€ YES β†’ Simulator has more RAM available
β”‚   Common: Large USDZ files with uncompressed textures
β”‚   Fix: Compress textures, reduce polygon count, use LOD
β”‚   Check: USDZ file size (keep < 50MB for reliable loading)
β”‚
└─ NO β†’ Check next

Q3: Is it an AR-related crash (camera, tracking)?
β”œβ”€ YES β†’ Simulator has no real camera/sensors
β”‚   Fix: Test AR features on device only, use simulator for UI/layout
β”‚
└─ NO β†’ Check device capabilities
    - A12+ required for RealityKit
    - LiDAR for scene reconstruction
    - TrueDepth for face tracking

Resources

WWDC: 2019-603, 2019-605, 2023-10080, 2024-10103

Docs: /realitykit, /realitykit/entity, /realitykit/collisioncomponent, /realitykit/physicsbodycomponent

Skills: axiom-realitykit, axiom-realitykit-ref