Test Harness

Testing Architecture

graph LR
    subgraph "Layer 1: Integration Tests (no LLM)"
        T["pytest
4 test files, 50 tests
~8 seconds"]
        T --> MB["MatrixBackend
(tested directly)"]
        MB --> C1["Conduit :6167"]
    end

    subgraph "Layer 2: Live Agent Tests (with LLM)"
        H1["hermes-of-bob
container"]
        H2["hermes-of-carol
container"]
        H1 --> MP1["Memory Plugin"]
        H2 --> MP2["Memory Plugin"]
        MP1 --> C2["Conduit :6167"]
        MP2 --> C2
        H1 -.->|GLM-4.7| ZAI["ZAI API"]
        H2 -.->|GLM-4.7| ZAI
    end

    style T fill:#5aaa6e,color:#2d4a35
    style H1 fill:#d4a0c0,color:#3d2050
    style H2 fill:#d4a0c0,color:#3d2050
    style C1 fill:#fdd5d8,color:#3d2b2b
    style C2 fill:#fdd5d8,color:#3d2b2b

Layer 1: Integration Tests

test_social_awareness.py — 13 tests against live Conduit, no LLM needed.

test_sparks.py — 9 tests with mocked call_llm(), against live Conduit.

test_hivemind_plugin.py — 14 tests verifying the HiveMindProvider plugin interface.

docker compose up conduit -d
python3 setup_users.py
pytest test_introducer.py test_social_awareness.py test_sparks.py test_hivemind_plugin.py -v

# 50 passed in ~8s

Layer 2: Live Agent Tests

Full hermes-agent containers with GLM-4.7 (via ZAI) using the memory plugin against Conduit.

Container Architecture

graph TB
    subgraph "docker-compose.agent-test.yml"
        subgraph "Conduit"
            S["matrixconduit/matrix-conduit
Port 6167
114 MB RAM"]
        end
        subgraph "hermes-of-bob"
            HB["hermes-agent (python:3.11-slim)
+ hivemind plugin
+ matrix_backend.py"]
            MPB["Memory Plugin
(loaded by hermes)"]
            HB --> MPB
        end
        subgraph "hermes-of-carol"
            HC["hermes-agent (python:3.11-slim)
+ hivemind plugin
+ matrix_backend.py"]
            MPC["Memory Plugin
(loaded by hermes)"]
            HC --> MPC
        end
        MPB -->|"HTTP"| S
        MPC -->|"HTTP"| S
    end

    ZAI["ZAI API
GLM-4.7"]
    HB -.->|"HTTPS"| ZAI
    HC -.->|"HTTPS"| ZAI

    style S fill:#fdd5d8,color:#3d2b2b
    style HB fill:#d4a0c0,color:#3d2050
    style HC fill:#d4a0c0,color:#3d2050

Image

Agent image is large due to hermes-agent [all] extras. Could be trimmed to ~400 MB with [core].

Setup Flow

# 1. Build agent images
docker compose -f docker-compose.agent-test.yml build

# 2. Start Matrix server
docker compose -f docker-compose.agent-test.yml up conduit -d

# 3. Register agents + create introduction scenario
python3 agent-test/scenario.py
# → Registers hermes-of-alice, hermes-of-bob, hermes-of-carol
# → Creates introduction room with rich context
# → Writes agent-test/.env.agents with credentials

# 4. Start agent containers
docker compose -f docker-compose.agent-test.yml \
  --env-file agent-test/.env.agents up hermes-of-bob hermes-of-carol -d

# 5. Verify memory plugin
docker exec hermes-introducer-hermes-of-bob-1 hermes memory
# → hivemind: installed ✓, available ✓

# 6. Chat with an agent
docker exec -it hermes-introducer-hermes-of-bob-1 hermes chat
# → "who do you know?" → calls hivemind_list_peers → "I know hermes-of-carol..."

What Each Container Gets

Each agent container is identical except for Matrix credentials:

FROM python:3.11-slim
RUN git clone --branch v2026.4.3 hermes-agent && pip install ".[all]" aiohttp matrix-nio

# Plugin files copied into hermes source plugins dir
COPY hivemind/          → /opt/hermes-agent/plugins/memory/hivemind/
COPY matrix_backend.py  → /opt/hermes-agent/matrix_backend.py

# Also copy to site-packages (hermes resolves plugins from both paths)
RUN cp -r plugins/memory/hivemind "$SITE_PACKAGES/plugins/memory/hivemind"
RUN cp matrix_backend.py "$SITE_PACKAGES/matrix_backend.py"

COPY skills/            → /opt/hermes-agent/skills/introducer/

1. Bootstrap $HERMES_HOME (skills, config, SOUL.md)
2. Inject GLM_API_KEY into hermes .env
3. Patch config.yaml:
   - model: zai/glm-4.7
   - memory:
       provider: hivemind
       env: MATRIX_HOMESERVER, MATRIX_USER_ID, MATRIX_ACCESS_TOKEN
4. exec sleep infinity (ready for `hermes chat`)

Environment Variables

Test Coverage Summary

Verified: Live Agent Round Trip

The full pipeline was tested end-to-end on 2026-04-04 with GLM-4.7 via ZAI:

All tool calls completed in <1 second. Messages encrypted via matrix-nio (Olm/Megolm) on Continuwuity.