refactor: migrate API keys to config, extend Kline intervals, add DB extensions
Security: - Move hardcoded Binance API key/secret from rest.ts to env.yaml (exchange config segment) - Add ExchangeConfig validation in config/validators.ts - Export typed exchange config from config/index.ts - Update AGENTS/07-caveats.md to reflect the new policy Kline intervals (add 3m / 2h / 6h / 8h / 1mon): - TypeScript: update KlineInterval type, KLINE_INTERVAL_MS mapping, build_aggregates_sql refresh chain - Python: sync KlineInterval Literal type, INTERVAL_TO_TABLE and INTERVAL_MS mappings, db_test table list - SQL: add 5 continuous aggregate materialized views (klines_3m/2h/6h/8h/1mon) with indexes - SQL: extend default kline_intervals in trading_pairs table - SQL: add cross-sectional query indexes for klines_1d and klines_1w DB: - Enable pg_prewarm extension (backtest warmup) - Enable pg_stat_statements extension (slow query monitoring) Other: - data/run/exchange.ts: graceful pgsql shutdown after backfill completes - Config path: load from data/env.yaml (symlink) instead of project root
This commit is contained in:
@@ -3,8 +3,8 @@
|
||||
// ============================================================
|
||||
// 用途:
|
||||
// 按月份粒度逐月刷新 klines 分层连续聚合物化视图链:
|
||||
// 5m → 15m → 30m → 1h → 4h → 1d → 1w
|
||||
// 每层依赖下一层的数据,因此严格按从低到高顺序刷新。
|
||||
// 基表 1m → 3m / 5m → 15m → 30m → 1h → 2h / 4h / 6h → 8h / 1d → 1w / 1mon
|
||||
// 严格按依赖顺序从低到高刷新。
|
||||
//
|
||||
// 使用方式:
|
||||
// # 仅生成 SQL 不执行(dry-run,默认)
|
||||
@@ -36,18 +36,26 @@ import { logger } from "../utils/logger";
|
||||
/**
|
||||
* 分层聚合视图链(按依赖顺序:低层级 → 高层级)
|
||||
*
|
||||
* 刷新顺序至关重要:
|
||||
* klines_5m 源数据来自 klines(1m 基表)
|
||||
* klines_15m 源数据来自 klines_5m
|
||||
* klines_30m 源数据来自 klines_15m
|
||||
* klines_1h 源数据来自 klines_30m
|
||||
* klines_4h 源数据来自 klines_1h
|
||||
* klines_1d 源数据来自 klines_4h
|
||||
* klines_1w 源数据来自 klines_1d
|
||||
* 实际依赖关系(来自于 03-continuous-aggregates.sql):
|
||||
* 基表 1m
|
||||
* ├── 3m (直接聚合 1m)
|
||||
* └── 5m (直接聚合 1m)→ 15m → 30m → 1h ──→ 2h
|
||||
* ├── 4h ──→ 8h
|
||||
* │ └── 1d ──→ 1w
|
||||
* │ └── 1mon
|
||||
* └── 6h
|
||||
*
|
||||
* 必须严格按此顺序刷新,否则高层级聚合会缺少数据。
|
||||
* 刷新时必须严格按此顺序,因为:
|
||||
* - klines_3m / klines_5m 无聚合依赖,可最先刷新
|
||||
* - klines_15m 依赖 klines_5m
|
||||
* - klines_30m 依赖 klines_15m
|
||||
* - klines_1h 依赖 klines_30m
|
||||
* - klines_2h / klines_4h / klines_6h 依赖 klines_1h
|
||||
* - klines_8h / klines_1d 依赖 klines_4h
|
||||
* - klines_1w / klines_1mon 依赖 klines_1d
|
||||
*/
|
||||
const AGGREGATE_VIEWS = [
|
||||
"klines_3m",
|
||||
"klines_5m",
|
||||
"klines_15m",
|
||||
"klines_30m",
|
||||
@@ -55,8 +63,10 @@ const AGGREGATE_VIEWS = [
|
||||
"klines_2h",
|
||||
"klines_4h",
|
||||
"klines_6h",
|
||||
"klines_8h",
|
||||
"klines_1d",
|
||||
"klines_1w",
|
||||
"klines_1mon",
|
||||
] as const;
|
||||
|
||||
/** 默认起始年月 */
|
||||
|
||||
+10
-1
@@ -41,4 +41,13 @@ for (const pair of allPairs) {
|
||||
} catch (err) {
|
||||
console.error("拉取失败:", err);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 所有币种回补完成以后等待1秒关闭pgsql连接退出此进程
|
||||
await new Promise((resolve) => setTimeout(resolve, 1000));
|
||||
const { AppDataSource } = await import("../db/data-source");
|
||||
if (AppDataSource.isInitialized) {
|
||||
await AppDataSource.destroy();
|
||||
console.log("pgsql 连接已关闭");
|
||||
}
|
||||
process.exit(0);
|
||||
Reference in New Issue
Block a user