open-krittKritt
§ · Nested detail page

后处理与人工复核流水线 深度解析

从 semantic dedupe、bounty ranker 到 post-script enrichment,open-kritt 把模型输出转成可排序、可评论、可回查的漏洞清单。

§ 00 · IMPORT CHAIN

导入链与契约

这页专看漏洞结果如何从模型输出进入可复核列表。API 的 backend/src/routes/scans.js:254 拉取 scan vulnerabilities,单条复核由 backend/src/routes/vulnerabilities.js:34 处理,前端摘要由 frontend/src/lib/scanPresentation.js:29 生成,快速点击则通过 frontend/src/pages/ScanDetail.jsx:145 的 mutation queue 串行化。

src/engine/open_kritt_engine/post_processing.py · L486-579Python / JavaScript
486    def _run_harness_with_retries(
487        self,
488        *,
489        metadata_id: int,
490        scan: dict[str, Any],
491        harness,
492        prompt: str,
493        schema: dict[str, Any],
494        validator,
495        prompt_template: str | None = None,
496        prompt_context: dict[str, Any] | None = None,
497        multi_output: bool = False,
498        kind: str = "post_process",
499    ) -> tuple[dict[str, Any], dict[str, Any] | None, str | None, str]:
500        prepared = None
501        try:
502            agent_skills = self._agent_skills(scan)
503            prepared = self._prepare_workspace(metadata_id, scan, agent_skills=agent_skills)
504            checked_out_commit = prepared.checked_out_commit
505            if prompt_template is not None:
506                context = {**(prompt_context or {}), **workspace_context(prepared)}
507                if "{{patched_since_history}}" in prompt_template:
508                    context["patched_since_history"] = patched_since_workspace_history_context(
509                        prepared.repo_dir,
510                        prepared.manifest,
511                        scan,
512                        context.get("file_path"),
513                    )
514                rendered = render_prompt(prompt_template, context)
515                prompt_body = harness_prompt(rendered, multi_output=multi_output, schema=schema)
516            else:
517                prompt_body = append_schema_prompt(prompt, schema)
518            prompt_parts = [
519                native_agent_skills_prompt(agent_skills, normalize_harness_name(scan["harness"])),
520                workspace_prompt_context(prepared.layout, prepared.manifest_json),
521                prompt_body,
522            ]
523            final_prompt = "\n\n".join(part for part in prompt_parts if part)
524            thinking_effort = scan.get("thinking_effort") or "medium"
525            with self.db.connect() as conn:
526                self.db.update_post_process_metadata(
527                    conn,
528                    metadata_id,
529                    status="running",
530                    error=None,
531                    run_time_ms=0,
532                    raw_token_usage=None,
533                    checked_out_commit=checked_out_commit,
534                    prompt_filled=final_prompt,
535                    phase="running_harness",
536                    codex_source_home=getattr(prepared.workspace, "codex_source_home", None),
537                    codex_account_id=getattr(prepared.workspace, "codex_account_id", None),
538                    codex_account_email=getattr(prepared.workspace, "codex_account_email", None),
539                )
540                conn.commit()
541
542            last_error = None
543            last_exception: Exception | None = None
544            attempt_errors: list[str] = []
545            for attempt in range(1, self._retry_count() + 2):
546                started = now_utc()
547                usage = None
548                codex_session_id = None
549                result = None
550                try:
551                    result = harness.run(
552                        prompt=final_prompt,
553                        schema=schema,
554                        repo_dir=prepared.repo_dir,
555                        model=scan["model"],
556                        thinking_effort=thinking_effort,
557                        env=prepared.workspace.env,
558                    )
559                    mark_provider_account_available(
560                        getattr(prepared.workspace, "provider_account_provider", None),
561                        getattr(prepared.workspace, "provider_account_home", None),
562                    )
563                    usage = result.usage
564                    codex_session_id = result.codex_session_id
565                    validator(result.payload)
566                    run_time_ms = int((now_utc() - started).total_seconds() * 1000)
567                    with self.db.connect() as conn:
568                        self.db.update_post_process_metadata(
569                            conn,
570                            metadata_id,
571                            status="running",
572                            error=None,
573                            run_time_ms=run_time_ms,
574                            raw_token_usage=usage,
575                            codex_session_id=codex_session_id,
576                            phase="writing_db",
577                        )
578                        conn.commit()
579                    return result.payload, usage, codex_session_id, checked_out_commit
§ 01 · CORE IMPL

核心实现,逐段拆

▸post-processing 三段式

post-processing 先 dedupe,再 rank,再执行 post-script 或完成 scan。它依赖 backend/src/routes/scans.js:254 输出的 ranked findings,也依赖 frontend/src/lib/scanPresentation.js:18 合并 enrichment 来源,最终让用户看到的是去重后的 canonical finding。

src/engine/open_kritt_engine/post_processing.py · L433-452Python / JavaScript
433    def process_once(self, scan: dict[str, Any], harness) -> bool:
434        scan_id = _int(scan["id"])
435        with self.db.connect() as conn:
436            current = self.db.load_scan(conn, scan_id)
437            if not current or current["status"] in {"paused", "stopped", "failed", "completed"}:
438                return False
439            vulnerabilities = self.db.load_vulnerabilities(conn, scan_id)
440            if not vulnerabilities:
441                self.db.set_scan_status_if_current(conn, scan_id, "post_processing", "completed")
442                conn.commit()
443                return True
444            conn.commit()
445
446        if any(row.get("dedupe_is_canonical") is None for row in vulnerabilities):
447            return self._run_next_dedupe_batch(scan, harness)
448
449        if any(row.get("dedupe_is_canonical") is True and row.get("bounty_rank") is None for row in vulnerabilities):
450            return self._run_next_ranker_batch(scan, harness)
451
452        return self._run_next_post_script_or_complete(scan, harness)

▸复核写入的窄接口

人工复核只允许 interesting 和 comments 两类字段。frontend/src/pages/ScanDetail.jsx:139 把 patch 入队,backend/src/routes/vulnerabilities.js:42 验证 interesting,backend/src/routes/vulnerabilities.js:58 更新后只返回最小字段集。

src/backend/src/routes/vulnerabilities.js · L32-68Python / JavaScript
32// PATCH /api/vulnerabilities/:id — user review: interesting flag and/or comments.
33// interesting: 1 (interesting), 0 (not interesting), or null (unmarked).
34router.patch('/:id', async (req, res, next) => {
35  try {
36    const id = BigInt(req.params.id);
37    const existing = await prisma.vulnerability.findUnique({ where: { id }, select: { id: true } });
38    if (!existing) return res.status(404).json({ error: 'Vulnerability not found.' });
39
40    const body = req.body || {};
41    const data = {};
42    if ('interesting' in body) {
43      const val = body.interesting;
44      if (val === null) data.interesting = null;
45      else if (val === 0 || val === 1 || val === '0' || val === '1') data.interesting = BigInt(Number(val));
46      else
47        return res
48          .status(422)
49          .json({ errors: [{ field: 'interesting', message: 'interesting must be 0, 1, or null.' }] });
50    }
51    if ('comments' in body) {
52      data.comments = body.comments === null || body.comments === '' ? null : String(body.comments);
53    }
54    if (Object.keys(data).length === 0) {
55      return res.status(422).json({ errors: [{ field: 'body', message: 'Provide interesting and/or comments.' }] });
56    }
57
58    const updated = await prisma.vulnerability.update({
59      where: { id },
60      data,
61      select: { id: true, interesting: true, comments: true },
62    });
63    res.json({
64      id: updated.id.toString(),
65      interesting:
66        updated.interesting === null || updated.interesting === undefined ? null : Number(updated.interesting),
67      comments: updated.comments ?? null,
68    });
§ 02 · KEY FLOW

关键流程

raw findings -> dedupe_batch -> ranker_batch -> post-script enrichments -> serialize -> ScanDetail optimistic review

这条链路的关键不是多一次模型调用,而是把模型输出变成可复核对象。backend/src/routes/vulnerabilities.js:8 读取单条 finding 与 duplicates,backend/src/routes/vulnerabilities.js:21 serialize 输出,frontend/src/lib/scanPresentation.js:39 提供摘要回退,frontend/src/pages/ScanDetail.jsx:163 让 interesting 在 null/1/0 间循环。

§ 03 · EDGE CASES

边界 / 失败模式

  • frontend/src/lib/scanPresentation.js:31 优先显示 severity/ease/patched 等安全语义字段,避免 UI 随机取值。
  • backend/src/routes/vulnerabilities.js:54 拒绝空 PATCH,减少无意义审计日志。
  • frontend/src/pages/ScanDetail.jsx:156 失败时 reload findings,避免乐观更新长时间污染界面。

▸复核证据补充

复核页的端到端证据是:backend/src/routes/scans.js:254 返回 scan findings,backend/src/routes/vulnerabilities.js:8 返回单条 finding 与 duplicates,frontend/src/lib/scanPresentation.js:31 选择摘要字段,frontend/src/pages/ScanDetail.jsx:139 发起乐观更新,frontend/src/pages/ScanDetail.jsx:156 在失败时回滚重载。这个路径让模型后处理结果保持可人工校正。

Sub-page · 2026-07-22 Source · Kritt-ai/open-kritt ↑ open-kritt 主页面