openscienceScience
§ · Nested detail page

科学数据库连接器 Registry 深度解析

这一页拆 OpenScience 如何把 UniProt、PDB、ChEMBL、arXiv 等外部数据库收敛成一个统一 search/fetch 契约。

§ 00 · IMPORT CHAIN

导入链与契约

科学连接器的核心不是某个 API,而是统一契约。backend/cli/src/science/connectors/types.ts:76 定义 Connector,backend/cli/src/science/connectors/types.ts:107 定义 ConnectorRegistry,backend/cli/src/science/connectors/index.ts:16 创建共享 registry,backend/cli/src/science/connectors/index.ts:30 把 proteins/genomics/chemistry/pathways/literature/omics 批量注册。

src/backend/cli/src/science/connectors/index.ts · L14-32TypeScript
14import { ConnectorRegistry } from "./types"
15
16export const registry = new ConnectorRegistry()
17
18// ── connector imports ──────────────────────────────────────────────────────
19// AUTO: connectors registered here by integration
20// (feature agents: create ./impl/<id>.ts, then add an import + registry.register
21//  call inside the block below during the integration stage — do NOT edit this
22//  file yourself.)
23import proteins from "./proteins"
24import genomics from "./genomics"
25import chemistry from "./chemistry"
26import pathways from "./pathways"
27import literature from "./literature"
28import omics from "./omics"
29
30for (const c of [...proteins, ...genomics, ...chemistry, ...pathways, ...literature, ...omics]) {
31  registry.register(c)
32}
§ 01 · CORE IMPL

核心实现,逐段拆

▸Connector 契约把数据库差异压成 search/fetch

接口在 backend/cli/src/science/connectors/types.ts:77 要求稳定 id,在 backend/cli/src/science/connectors/types.ts:83 标注 domain,在 backend/cli/src/science/connectors/types.ts:89 暴露 search,在 backend/cli/src/science/connectors/types.ts:91 暴露 fetch。SearchOptions 的 backend/cli/src/science/connectors/types.ts:43 到 backend/cli/src/science/connectors/types.ts:51 允许 limit、organism、params、AbortSignal,这些正是科研检索最常见的控制面。

src/backend/cli/src/science/connectors/types.ts · L42-92TypeScript
42/** Options accepted by `search`. Connectors ignore fields they don't support. */
43export interface SearchOptions {
44  /** Max hits to return. Connectors clamp to their own ceiling. */
45  limit?: number
46  /** Organism / taxon filter where meaningful (e.g. NCBI taxon id). */
47  organism?: string
48  /** Free-form per-source knobs. */
49  params?: Record<string, unknown>
50  /** Abort signal wired through from the tool context. */
51  signal?: AbortSignal
52}
53
54/** Options accepted by `fetch`. */
55export interface FetchOptions {
56  /** Desired representation, e.g. "json" | "pdb" | "fasta" | "sdf". */
57  format?: string
58  params?: Record<string, unknown>
59  signal?: AbortSignal
60}
61
62/**
63 * Per-host politeness limits a connector can request for its HTTP calls.
64 *
65 * Applied per URL host by the shared http layer, so a multi-DB fan-out is never
66 * over-serialized. arXiv, for example, asks for ~1 request every 3s
67 * (`minIntervalMs: 3000`); keyless Semantic Scholar wants ~1000ms.
68 */
69export interface RateLimit {
70  /** Minimum spacing between the START of requests to the same host, in ms. */
71  minIntervalMs?: number
72  /** Max concurrent in-flight requests to the same host. */
73  maxConcurrent?: number
74}
75
76/** The uniform contract every scientific data source implements. */
77export interface Connector {
78  /** Unique, stable, lowercase id used to route (e.g. "uniprot", "rcsb-pdb"). */
79  id: string
80  /** Display name (e.g. "UniProt", "RCSB PDB"). */
81  name: string
82  /** Primary domain for grouping in the catalog. */
83  domain: ConnectorDomain
84  /** One-line description shown in `science_list_dbs`. */
85  description: string
86  /** Homepage / docs URL. */
87  homepage?: string
88  /** Search the source; returns normalized hits. */
89  search(query: string, opts?: SearchOptions): Promise<ConnectorHit[]>
90  /** Fetch a single record by id. Return shape is source-specific. */
91  fetch(id: string, opts?: FetchOptions): Promise<unknown>
92}

▸Registry 提供 UI 和工具都能消费的 catalog

Registry 在 backend/cli/src/science/connectors/types.ts:108 用 Map 存储连接器,backend/cli/src/science/connectors/types.ts:110 防止重复 id,backend/cli/src/science/connectors/types.ts:125 返回全部连接器,backend/cli/src/science/connectors/types.ts:129 支持按 domain 过滤,backend/cli/src/science/connectors/types.ts:134 把函数剥离成可序列化 catalog。

src/backend/cli/src/science/connectors/types.ts · L103-142TypeScript
103/**
104 * In-memory registry of connectors. A single shared instance lives in
105 * `./index.ts`; feature agents call `.register()` there.
106 */
107export class ConnectorRegistry {
108  private readonly connectors = new Map<string, Connector>()
109
110  register(connector: Connector): void {
111    if (this.connectors.has(connector.id)) {
112      throw new Error(`Connector "${connector.id}" is already registered`)
113    }
114    this.connectors.set(connector.id, connector)
115  }
116
117  get(id: string): Connector | undefined {
118    return this.connectors.get(id)
119  }
120
121  has(id: string): boolean {
122    return this.connectors.has(id)
123  }
124
125  all(): Connector[] {
126    return [...this.connectors.values()]
127  }
128
129  byDomain(domain: ConnectorDomain): Connector[] {
130    return this.all().filter((c) => c.domain === domain)
131  }
132
133  /** Serializable catalog for tools / UI (drops the search/fetch functions). */
134  catalog(): CatalogEntry[] {
135    return this.all().map(({ id, name, domain, description, homepage }) => ({
136      id,
137      name,
138      domain,
139      description,
140      homepage,
141    }))
142  }
§ 02 · KEY FLOW

关键流程

connector impl → Connector contract → registry.register → science_search / science_list_dbs → workspace artifact

这条路径把外部数据库差异藏在连接器内部。backend/cli/src/science/connectors/types.ts:26 的 ConnectorHit 统一 id/title/summary/url/score,backend/cli/src/science/connectors/types.ts:62 的 RateLimit 给 HTTP 层限流提示,backend/cli/src/science/connectors/index.ts:35 再导出 types 与 http,让实现层共享同一工具箱。

▸为什么这样拆

这个子模块的拆法不是按文件大小,而是按运行时责任边界来切。主页面已经把 backend/cli/src/index.ts:55、backend/cli/src/config/config.ts:75、backend/cli/src/provider/provider.ts:57、backend/cli/src/science/connectors/types.ts:76 和 tooling/plugin/src/index.ts:148 串成整体;子页面只展开其中一个闭环,避免把所有源码证据挤成行号清单。

§ 03 · EDGE CASES

边界 / 失败模式

  • Registry 只防重复 id,见 backend/cli/src/science/connectors/types.ts:111;连接器结果质量仍依赖具体实现。
  • RateLimit 是连接器可请求的策略,backend/cli/src/science/connectors/types.ts:69 到 backend/cli/src/science/connectors/types.ts:73 没有强制每个实现都声明。
  • Catalog 在 backend/cli/src/science/connectors/types.ts:133 会丢弃函数,适合 UI,但调试时要回到具体 connector 源码。