Using Fate Partner's deterministic perpetual calendar engine, we built an open-source benchmark of 100 universal-UTC astronomical boundary birth cases (v1.1) covering Start of Spring, the 12 month-boundary solar terms, True Solar Time shifts in Tokyo and Taipei, late Zi hour day transitions, and historical Daylight Saving Time. Auditing 5 zero-tool LLMs on November 22, 2026 revealed a stark tier divide: non-reasoning DeepSeek-Chat failed 78.0% of charts, fast-reasoning Gemini 3.8 Flash failed 44.0%, Claude Sonnet 5.5 failed 16.0%, flagship Claude Opus 5.5 still failed 4.0%, and Gemini 4 Carbon reached 100/100 only by burning 105,418 thinking tokens across roughly 18 minutes.
1. Five-Model 100-Case Leaderboard (v1.1 Universal-UTC Ground Truth)
Even under a generous upper-bound prompt that explicitly supplied each city's decimal longitude, IANA timezone, True Solar Time flag, and historical DST rules, zero-tool results diverged dramatically: (1) DeepSeek-Chat (non-reasoning REST API, temp 0): 22.0% exact match, 78.0% error rate (22% wrong Day Masters, 52% wrong Hour Pillars); (2) Gemini 3.8 Flash (Medium thinking, ~53k output tokens): 56.0% exact match, 44.0% error rate (19% wrong Day Masters, 95% error on True Solar Time); (3) Claude Sonnet 5.5 (Medium thinking, ~49k tokens): 84.0% exact match, 16.0% error rate (32% error on 12-Jie month boundaries); (4) Claude Opus 5.5 (Medium thinking, ~38k tokens): 96.0% exact match, 4.0% error rate (12% error on 12-Jie month boundaries); (5) Gemini 4 Carbon (Medium thinking, 105,418 tokens, ~18 minutes): 100.0% exact match; (6) Fate Partner deterministic engine and MIT MCP server: 100.0% exact match in under 2 milliseconds and 0 LLM tokens. Transparency note: open-weight GPT-OSS 120B (Medium) refused zero-tool computation on Attempt 1 and collapsed into repeated placeholder output under 50-case batching on Attempt 2, so it was excluded from the leaderboard.
2. Failure Breakdown Across All 5 Astronomical Trap Categories
Examining each category shows where everyday chatbots break down: (1) Start of Spring transitions (25 cases): DeepSeek-Chat erred on 72.0%, Claude Sonnet 5.5 on 16.0%, and Gemini 3.8 Flash on 12.0%; (2) 12 month-boundary solar terms (25 cases): the hardest frontier barrier, where DeepSeek and Gemini 3.8 Flash both erred on 48.0%, Claude Sonnet 5.5 erred on 32.0%, and even flagship Claude Opus 5.5 erred on 12.0% (for example, 1992-07-07 05:20 in Taipei occurred 40 minutes after Minor Heat, yet Opus stayed in the prior month); (3) True Solar Time across Japan, Taiwan, and global cities (20 cases): DeepSeek erred on 100.0% (0/20) and Gemini 3.8 Flash erred on 95.0% (1/20), routinely missing Tokyo's +35 minute longitude and equation-of-time shift on 1992-11-03 00:35; (4) Late Zi 23:00 day boundary (15 cases): DeepSeek erred on 86.7% and Gemini 3.8 Flash erred on 33.3%; (5) Historical DST in Japan (1948-1951), Taiwan (1946-1979), and Mainland China (1986-1991) (15 cases): DeepSeek erred on 100.0% and Gemini 3.8 Flash erred on 33.3%.
3. 105,000 Thinking Tokens vs 2 Milliseconds: Why Tool Separation Wins
The real lesson is architectural economics rather than dismissing AI progress. Everyday users asking free or fast-tier chat apps for BaZi readings never paste decimal city longitudes or historical DST rules, putting real-world casual error rates in the 44% to 78%+ band. And even when a frontier model like Gemini 4 Carbon solves all 100 boundary charts mentally, it spends over 1,050 thinking tokens and roughly 11 seconds per single chart doing manual Julian Day arithmetic — more than 5,000 times slower than a deterministic 2-millisecond calendar program that uses zero tokens and never drifts.
4. Open-Source Audit Dividend (v1.1 Timezone Fix) & Hybrid Architecture
Running cross-model blind verification also paid an immediate engineering dividend: inspecting disagreements in v1.0 caught a 1-hour cross-timezone solar-term bug affecting 10 Japan (Asia/Tokyo, UTC+9 vs UTC+8 ephemeris) boundary cases, which we fixed and documented transparently in v1.1. The complete universal-UTC dataset, all 5 audited model transcripts, and our standalone MIT MCP server (bazi-solar-term-mcp) are open-sourced on GitHub (github.com/liu17513567-blip/bazi-solar-term-mcp). Fate Partner separates deterministic astronomy from plain-language relationship guidance so language models focus solely on thoughtful interpretation.