科学数据库连接器 Registry 深度解析
这一页拆 OpenScience 如何把 UniProt、PDB、ChEMBL、arXiv 等外部数据库收敛成一个统一 search/fetch 契约。
导入链与契约
科学连接器的核心不是某个 API,而是统一契约。backend/cli/src/science/connectors/types.ts:76 定义 Connector,backend/cli/src/science/connectors/types.ts:107 定义 ConnectorRegistry,backend/cli/src/science/connectors/index.ts:16 创建共享 registry,backend/cli/src/science/connectors/index.ts:30 把 proteins/genomics/chemistry/pathways/literature/omics 批量注册。
14import { ConnectorRegistry } from "./types" 15 16export const registry = new ConnectorRegistry() 17 18// ── connector imports ────────────────────────────────────────────────────── 19// AUTO: connectors registered here by integration 20// (feature agents: create ./impl/<id>.ts, then add an import + registry.register 21// call inside the block below during the integration stage — do NOT edit this 22// file yourself.) 23import proteins from "./proteins" 24import genomics from "./genomics" 25import chemistry from "./chemistry" 26import pathways from "./pathways" 27import literature from "./literature" 28import omics from "./omics" 29 30for (const c of [...proteins, ...genomics, ...chemistry, ...pathways, ...literature, ...omics]) { 31 registry.register(c) 32}
核心实现,逐段拆
▸Connector 契约把数据库差异压成 search/fetch
接口在 backend/cli/src/science/connectors/types.ts:77 要求稳定 id,在 backend/cli/src/science/connectors/types.ts:83 标注 domain,在 backend/cli/src/science/connectors/types.ts:89 暴露 search,在 backend/cli/src/science/connectors/types.ts:91 暴露 fetch。SearchOptions 的 backend/cli/src/science/connectors/types.ts:43 到 backend/cli/src/science/connectors/types.ts:51 允许 limit、organism、params、AbortSignal,这些正是科研检索最常见的控制面。
42/** Options accepted by `search`. Connectors ignore fields they don't support. */ 43export interface SearchOptions { 44 /** Max hits to return. Connectors clamp to their own ceiling. */ 45 limit?: number 46 /** Organism / taxon filter where meaningful (e.g. NCBI taxon id). */ 47 organism?: string 48 /** Free-form per-source knobs. */ 49 params?: Record<string, unknown> 50 /** Abort signal wired through from the tool context. */ 51 signal?: AbortSignal 52} 53 54/** Options accepted by `fetch`. */ 55export interface FetchOptions { 56 /** Desired representation, e.g. "json" | "pdb" | "fasta" | "sdf". */ 57 format?: string 58 params?: Record<string, unknown> 59 signal?: AbortSignal 60} 61 62/** 63 * Per-host politeness limits a connector can request for its HTTP calls. 64 * 65 * Applied per URL host by the shared http layer, so a multi-DB fan-out is never 66 * over-serialized. arXiv, for example, asks for ~1 request every 3s 67 * (`minIntervalMs: 3000`); keyless Semantic Scholar wants ~1000ms. 68 */ 69export interface RateLimit { 70 /** Minimum spacing between the START of requests to the same host, in ms. */ 71 minIntervalMs?: number 72 /** Max concurrent in-flight requests to the same host. */ 73 maxConcurrent?: number 74} 75 76/** The uniform contract every scientific data source implements. */ 77export interface Connector { 78 /** Unique, stable, lowercase id used to route (e.g. "uniprot", "rcsb-pdb"). */ 79 id: string 80 /** Display name (e.g. "UniProt", "RCSB PDB"). */ 81 name: string 82 /** Primary domain for grouping in the catalog. */ 83 domain: ConnectorDomain 84 /** One-line description shown in `science_list_dbs`. */ 85 description: string 86 /** Homepage / docs URL. */ 87 homepage?: string 88 /** Search the source; returns normalized hits. */ 89 search(query: string, opts?: SearchOptions): Promise<ConnectorHit[]> 90 /** Fetch a single record by id. Return shape is source-specific. */ 91 fetch(id: string, opts?: FetchOptions): Promise<unknown> 92}
▸Registry 提供 UI 和工具都能消费的 catalog
Registry 在 backend/cli/src/science/connectors/types.ts:108 用 Map 存储连接器,backend/cli/src/science/connectors/types.ts:110 防止重复 id,backend/cli/src/science/connectors/types.ts:125 返回全部连接器,backend/cli/src/science/connectors/types.ts:129 支持按 domain 过滤,backend/cli/src/science/connectors/types.ts:134 把函数剥离成可序列化 catalog。
103/** 104 * In-memory registry of connectors. A single shared instance lives in 105 * `./index.ts`; feature agents call `.register()` there. 106 */ 107export class ConnectorRegistry { 108 private readonly connectors = new Map<string, Connector>() 109 110 register(connector: Connector): void { 111 if (this.connectors.has(connector.id)) { 112 throw new Error(`Connector "${connector.id}" is already registered`) 113 } 114 this.connectors.set(connector.id, connector) 115 } 116 117 get(id: string): Connector | undefined { 118 return this.connectors.get(id) 119 } 120 121 has(id: string): boolean { 122 return this.connectors.has(id) 123 } 124 125 all(): Connector[] { 126 return [...this.connectors.values()] 127 } 128 129 byDomain(domain: ConnectorDomain): Connector[] { 130 return this.all().filter((c) => c.domain === domain) 131 } 132 133 /** Serializable catalog for tools / UI (drops the search/fetch functions). */ 134 catalog(): CatalogEntry[] { 135 return this.all().map(({ id, name, domain, description, homepage }) => ({ 136 id, 137 name, 138 domain, 139 description, 140 homepage, 141 })) 142 }
关键流程
这条路径把外部数据库差异藏在连接器内部。backend/cli/src/science/connectors/types.ts:26 的 ConnectorHit 统一 id/title/summary/url/score,backend/cli/src/science/connectors/types.ts:62 的 RateLimit 给 HTTP 层限流提示,backend/cli/src/science/connectors/index.ts:35 再导出 types 与 http,让实现层共享同一工具箱。
▸为什么这样拆
这个子模块的拆法不是按文件大小,而是按运行时责任边界来切。主页面已经把 backend/cli/src/index.ts:55、backend/cli/src/config/config.ts:75、backend/cli/src/provider/provider.ts:57、backend/cli/src/science/connectors/types.ts:76 和 tooling/plugin/src/index.ts:148 串成整体;子页面只展开其中一个闭环,避免把所有源码证据挤成行号清单。
边界 / 失败模式
- Registry 只防重复 id,见
backend/cli/src/science/connectors/types.ts:111;连接器结果质量仍依赖具体实现。 - RateLimit 是连接器可请求的策略,
backend/cli/src/science/connectors/types.ts:69到backend/cli/src/science/connectors/types.ts:73没有强制每个实现都声明。 - Catalog 在
backend/cli/src/science/connectors/types.ts:133会丢弃函数,适合 UI,但调试时要回到具体 connector 源码。