RealReplicaBench
Nested detail page

OpenClaw 容器 Runner 深度解析

OpenClaw harness 如何注入自主指令、provider config、视觉能力判断和状态归档。

§ 00 · IMPORT CHAIN

导入链与边界

▸为什么这个模块是入口

OpenClaw 容器 Runner 的关键不是单个函数,而是几组契约一起生效:src/real_replica_bench/harnesses/openclaw/runner.py:64、src/real_replica_bench/harnesses/openclaw/runner.py:78、src/real_replica_bench/harnesses/openclaw/runner.py:120、src/real_replica_bench/harnesses/openclaw/runner.py:128。这些行分别描述任务定位、能力声明、评分或 harness 行为,因此改动其中任何一处都会改变 benchmark 的可比较性。

src/real_replica_bench/harnesses/openclaw/runner.py · L64-160Python
64# Autonomous-agent directive, prepended to every OpenClaw prompt in the most
65# prominent position. OpenClaw's default framing is a conversational personal
66# assistant, and persona-forward models (notably GPT-5.6 Luna/Terra/Sol) latch
67# onto it — they open with a "我刚上线,你希望怎么称呼我?" self-introduction plus a
68# question, end the turn with zero tool calls, and since `openclaw agent
69# --message` is a single non-interactive invocation there is no user to answer →
70# the loop terminates and the task scores ~0. Reproduced on this runtime with
71# gpt-5.6-luna: one upstream call, zero tool calls, reward 0.
72#
73# It carries NO task-specific or verifier-facing hints, and is applied uniformly
74# to every OpenClaw run so the cross-model comparison stays fair. Because it sits
75# on the scoring boundary — changing it moves every model's starting line — it is
76# a module-level constant rather than an inline string: greppable, diffable in
77# review, and overridable in an A/B harness without editing this file.
78OPENCLAW_AGENT_DIRECTIVE = (
79    "# 角色:自主任务执行 Agent\n\n"
80    "你是一个自主运行的任务执行 agent,**不是**聊天助手。请**立即开始执行下面的任务**。\n"
81    "- **不要**自我介绍、不要给自己取名字、不要询问「如何称呼你 / 我该怎么称呼你」、不要任何寒暄开场白。\n"
82    "- 当前是**无人值守的单次执行**:没有人会回答你的任何反问,任何"
83    "「等待用户确认 / 请用户提供」都会导致任务直接失败。\n"
84    "- 直接调用工具(read / exec / browser 等)完成任务,把最终产物写到 `/task/outputs/`。"
85    "遇到不确定的地方,基于任务描述做出最合理的假设并继续推进,不要停下来等待确认。\n\n"
86    "---\n\n"
87)
88
89# Vision-capable model substrings used by the OpenClaw set-image precheck.
90# Match is case-insensitive substring against the resolved primary model name
91# (which is what OpenClaw's `models set-image` would otherwise be left set to
92# when --openclaw-image-model is not provided). Anything not in this list is
93# treated as non-vision and forces the user to pass --openclaw-image-model
94# explicitly. Adding a new model: add the lowercase substring.
95_OPENCLAW_VISION_MODEL_SUBSTRINGS: tuple[str, ...] = (
96    "gemini-3-flash-preview",
97    "gemini-3.1-pro-preview",
98    "gemini-3.1-flash",
99    "gemini-3.5-flash",
100    "gemini-3.5-pro",
101    "claude",
102    "qwen-vl",
103    "qwen3.7-preview",
104    "qwen3.7-plus",  # multimodal GA (2026-06-01, DashScope + OpenRouter)
105    # Substring covers the whole qwen3.8-max family. The -preview variant was
106    # probed on DashScope (2026-07-26: image_url decoded, image_tokens billed);
107    # the GA id is covered by the same entry rather than a second line.
108    "qwen3.8-max",
109    "qwen-latest-series",
110    "kimi-vl",
111    "glm-vl",
112    "gpt-5",
113    "gpt-5.5",
114    "gpt-5.2",
115    "gpt-4o",
116    "gpt-4-turbo",
117)
118
119
120def _openclaw_model_is_vision_capable(model: str) -> bool:
121    """Heuristic substring match against a curated vision-capable list."""
122    if not model:
123        return False
124    lower = model.lower()
125    return any(substr in lower for substr in _OPENCLAW_VISION_MODEL_SUBSTRINGS)
126
127
128def _openclaw_is_relay_mode(args: argparse.Namespace) -> bool:
129    """OpenClaw browser-driver mode dispatch by image tag.
130
131    The explicitly named legacy images (`commercecraftbench/openclaw:base-v1.3`
132    and `commercecraftbench/openclaw:ccb-v1.3`) are built on an OpenClaw
133    2026.3.x runtime base. That release ships a Chrome extension `Browser
134    Relay` that we patch + attach via CDP.
135    Profile name = `chrome-relay`, driver = `extension`, cdpUrl points
136    at an in-process relay server.
137
138    v2026.5.22 dropped the extension entirely (`assets/chrome-extension/`
139    removed, no `chrome-relay` profile in dist) and only supports two
140    drivers — `openclaw` (managed) and `existing-session` (limited). The
141    correct config for the upstream-entrypoint Chrome at 9222 is:
142        driver=openclaw, attachOnly=true, cdpUrl=http://127.0.0.1:9222
143    which yields `local-managed` capability (full snapshot+act+tabs) while
144    re-using the noVNC-visible Chrome from the entrypoint. We previously
145    used driver=existing-session, but that caps capabilities to
146    `local-existing-session` (no selector targeting, no managed tabs) and
147    triggers the "user-style profile cannot use sandbox browser" error.
148
149    Detection is deliberately opt-in for those two legacy tags. The public
150    release pins the May image by digest, so looking for ``2026.5.22`` in the
151    image string would misclassify the canonical digest as the March runtime.
152    Current and custom images therefore use managed-attach mode by default.
153    """
154    image = (getattr(args, "image", "") or "").lower()
155    legacy_tags = (
156        "commercecraftbench/openclaw:base-v1.3",
157        "commercecraftbench/openclaw:ccb-v1.3",
158    )
159    return any(tag in image for tag in legacy_tags)
160
§ 01 · CORE IMPL

核心实现,逐段拆

▸状态与输入如何被收束

第二段源码展示了实现层的防漏设计:src/real_replica_bench/harnesses/openclaw/runner.py:162、src/real_replica_bench/harnesses/openclaw/runner.py:174、src/real_replica_bench/harnesses/openclaw/runner.py:240、src/real_replica_bench/harnesses/openclaw/runner.py:263、src/real_replica_bench/harnesses/openclaw/runner.py:280。它不是为了让页面好看,而是把任务输入、运行时状态、模型协议和输出证据都保持在可审计范围内,方便后续报告复原。

src/real_replica_bench/harnesses/openclaw/runner.py · L162-237Python
162def collect_openclaw_state_artifacts(container: str, agent_dir: Path) -> dict[str, Any]:
163    """Tar /root/.openclaw/agents/main/ + copy /tmp/openclaw/openclaw-*.log
164    out of the container into ``agent_dir`` for post-mortem inspection.
165
166    Mirrors the spirit of ``collect_accio_state_artifacts`` but the OpenClaw
167    container is destroyed per-task, so there is no host-side cleanup pass.
168    Returns a dict for the manifest / debug file.
169    """
170    out: dict[str, Any] = {"collected": False, "files": [], "openclaw_agents_present": False}
171
172    # 1. agents/main tarball — quick check first to avoid a noisy tar of a
173    # missing path.
174    probe = docker(
175        "exec", container, "bash", "-lc",
176        "test -d /root/.openclaw/agents/main && echo yes || echo no",
177        check=False, capture=True,
178    )
179    has_agents = (probe.stdout or "").strip() == "yes"
180    out["openclaw_agents_present"] = has_agents
181    if has_agents:
182        tar_path = agent_dir / "openclaw_state.tar.gz"
183        try:
184            with tar_path.open("wb") as tf:
185                proc = subprocess.run(
186                    [
187                        "docker", "exec", container, "bash", "-lc",
188                        "tar --exclude=main/agent/auth-profiles.json "
189                        "--exclude=main/agent/credentials.json "
190                        "-czf - -C /root/.openclaw/agents main",
191                    ],
192                    stdout=tf,
193                    stderr=subprocess.PIPE,
194                    check=False,
195                )
196            if proc.returncode == 0 and tar_path.is_file() and tar_path.stat().st_size > 0:
197                out["files"].append(str(tar_path.relative_to(agent_dir)))
198                # Untar locally for grep-ability.
199                state_dir = agent_dir / "openclaw_state"
200                state_dir.mkdir(parents=True, exist_ok=True)
201                untar = subprocess.run(
202                    ["tar", "-xzf", str(tar_path), "-C", str(state_dir)],
203                    capture_output=True, text=True, check=False,
204                )
205                if untar.returncode == 0:
206                    out["files"].append(str(state_dir.relative_to(agent_dir)) + "/")
207                else:
208                    out["untar_error"] = (untar.stderr or "").strip()[:500]
209            else:
210                out["tar_error"] = (proc.stderr.decode("utf-8", errors="replace") if isinstance(proc.stderr, bytes) else (proc.stderr or "")).strip()[:500]
211                if tar_path.is_file() and tar_path.stat().st_size == 0:
212                    tar_path.unlink(missing_ok=True)
213        except OSError as exc:
214            out["tar_error"] = str(exc)
215
216    # 2. OpenClaw runtime logs at /tmp/openclaw/openclaw-*.log. List first,
217    # then docker cp each. Tolerate misses.
218    list_proc = docker(
219        "exec", container, "bash", "-lc",
220        "ls -1 /tmp/openclaw/openclaw-*.log 2>/dev/null || true",
221        check=False, capture=True,
222    )
223    log_paths = [line.strip() for line in (list_proc.stdout or "").splitlines() if line.strip()]
224    if log_paths:
225        runtime_dir = agent_dir / "openclaw_runtime_logs"
226        runtime_dir.mkdir(parents=True, exist_ok=True)
227        for remote_log in log_paths:
228            local_dst = runtime_dir / Path(remote_log).name
229            cp = docker("cp", f"{container}:{remote_log}", str(local_dst), check=False, capture=True)
230            if cp.returncode == 0 and local_dst.is_file():
231                out["files"].append(str(local_dst.relative_to(agent_dir)))
232
233    if out["files"]:
234        out["collected"] = True
235    elif not has_agents:
236        out["reason"] = "openclaw_agents_dir_missing"
237    return out

▸运行时细节如何暴露给调试

第三段代码补上执行末端:src/real_replica_bench/harnesses/openclaw/runner.py:64 与 src/real_replica_bench/harnesses/openclaw/runner.py:280 附近的实现让失败不只表现为 pass/fail,还能回到日志、配置、工具调用和标准输出定位原因。

src/real_replica_bench/harnesses/openclaw/runner.py · L240-280Python
240def _parse_openclaw_provider_headers(raw_headers: list[str]) -> dict[str, str]:
241    """Parse repeated `--openclaw-provider-header KEY:VALUE` flags into a dict.
242
243    Only the FIRST colon is treated as the separator, so values that
244    themselves contain ``:`` (URLs, times) round-trip verbatim.
245    """
246    parsed: dict[str, str] = {}
247    for raw in raw_headers or []:
248        if ":" not in raw:
249            raise SystemExit(
250                f"--openclaw-provider-header must be formatted 'Key:Value' (got: {raw!r})"
251            )
252        key, _, value = raw.partition(":")
253        key = key.strip()
254        value = value.strip()
255        if not key:
256            raise SystemExit(
257                f"--openclaw-provider-header has empty key (got: {raw!r})"
258            )
259        parsed[key] = value
260    return parsed
261
262
263def _synthesize_openclaw_models_config(args: argparse.Namespace) -> dict[str, Any]:
264    """Build a one-provider models_config JSON from --openclaw-* CLI flags.
265
266    Fires from `inject_openclaw_models_config` when the user passes
267    ``--openclaw-api`` without also passing ``--openclaw-models-config``.
268    The result has the same shape as a committed preset JSON — one entry in
269    ``providers`` keyed by the model-id prefix — so the downstream
270    injection path (docker cp + in-container merge into
271    ``~/.openclaw/openclaw.json['models']``) is unchanged.
272    """
273    api = args.openclaw_api
274    base_url = getattr(args, "openclaw_provider_base_url", None)
275    if not base_url:
276        raise SystemExit(
277            "--openclaw-provider-base-url is required when --openclaw-api is set."
278        )
279    model = args.openclaw_model or ""
280    if "/" not in model:
§ 02 · KEY FLOW

关键流程

task config → contract object → runtime materialization → agent execution → verifier/report

从这些源码行可以看到,RealReplicaBench 的流程不是把 Agent 输出丢给一个脚本评分,而是先构造可复现任务世界,再让 harness 运行,最后把 verifier、trajectory 和 report 合并成证据链。

OpenClaw 适配层的风险集中在容器边界:src/real_replica_bench/harnesses/openclaw/runner.py:72 注入自主执行指令,src/real_replica_bench/harnesses/openclaw/runner.py:96 组织 agent workspace,src/real_replica_bench/harnesses/openclaw/runner.py:187 抽取状态证据,src/real_replica_bench/harnesses/openclaw/runner.py:256 解析 provider header,src/real_replica_bench/harnesses/openclaw/runner.py:274 过滤空值配置。

§ 03 · EDGE CASES

边界 / 失败模式

▸失败在哪里会被放大

  • 如果任务声明不完整,src/real_replica_bench/harnesses/openclaw/runner.py:78 附近的解析会提前中断,而不是让 Agent 在缺环境时跑出伪结果。
  • 如果 mock 或 provider 配置漂移,src/real_replica_bench/harnesses/openclaw/runner.py:120 附近的检查会让问题在发布前暴露。
  • 如果 Agent 只完成部分业务动作,最终分数会被 verifier 端硬门禁压成失败,需要回看 trajectory。
§ 04 · RELATED

关联模块

相关子页
Generated · 2026-08-07Source · Accio-org/RealReplicaBench