The Journey: 11,666 to 17,500 to 24,000+
Three months ago, we published our first testing transparency post. JieGou had 11,666 automated tests at 99.18% code coverage. It was already more than any other AI automation platform had published — because no other platform publishes test metrics at all.
Since then, the product has grown significantly. New features shipped: chat agents with 12 messaging channel integrations, graduated autonomy with 4 trust levels, a coding agent workflow step, conversation compaction, session branching, website knowledge base imports, custom tool lifecycle hooks, and an SDK for headless execution. Each feature brought new test surface area.
The numbers tell the story:
- February 2026: 11,666 tests
- Late February 2026: 17,500 tests
- March 2026: 24,000+ tests
That’s a 2x increase in test coverage in under three months — while shipping major features every week.
What We Test
Unit Tests (Vitest)
The bulk of the suite. Server-side logic, data transformations, validation rules, business logic, and utility functions. Every function in src/lib/server/ has corresponding test coverage. Key areas:
- LLM provider abstraction: Mock-based testing for Anthropic, OpenAI, Google, and OpenAI-compatible endpoints. Tool calling, structured output, streaming, error conditions, circuit breakers, and rate limiting.
- Workflow engine: Step execution, DAG resolution, parallel wave scheduling, convergence loops, approval gate state machines, crash-recovery checkpointing.
- Auth and RBAC: 5-role permission model (Owner > Admin > Manager > Editor > Viewer) with 20 granular permissions. Every permission boundary has positive and negative tests.
- Chat agents: Message routing across 12 channels (LINE, Instagram, Facebook Messenger, WhatsApp, Telegram, Slack, Discord, WeChat, Viber, SMS, email, web chat). FAQ matching, confidence scoring, auto-reply logic, human escalation rules.
- Encryption: AES-256-GCM envelope encryption for API keys with per-account HKDF key derivation. Key rotation without downtime.
Integration Tests
API route testing with realistic request/response cycles. Every +server.ts endpoint has tests covering:
- Authentication and authorization
- Input validation and error responses
- Happy path with expected outputs
- Edge cases: empty inputs, oversized payloads, concurrent requests
- Rate limiting and circuit breaker behavior
E2E Tests (Playwright)
Full browser automation exercising real user journeys:
- Admin onboarding flows
- Department lead review processes
- Developer workflow creation
- RBAC enforcement verification (unauthorized access blocked)
- Data consistency between API responses and UI rendering
- Accessibility audits using @axe-core for WCAG 2.1 AA compliance
LLM Mock Testing
Our LLM mock system provides deterministic test doubles for all 4 provider families. This is critical because AI outputs are non-deterministic — you can’t write expect(response).toBe("exact string") for LLM calls. Instead, we test:
- Response structure and schema compliance
- Tool calling sequences and parameter validation
- Streaming chunk assembly
- Error handling: timeouts, rate limits, malformed responses
- Provider-specific quirks (each has different JSON formatting, tool call schemas, etc.)
Why It Matters for Enterprise
SOC 2 Evidence
Our test suite is part of the SOC 2 evidence collection. Test coverage maps directly to Trust Services Criteria:
- CC5.2 (Control Activities): Test suite as quality control evidence
- CC6.2 (Access Controls): RBAC enforcement tests as access control proof
- CC7.1 (System Operations): Nightly CI as continuous monitoring
- CC8.1 (Change Management): PR test gate as change management control
When auditors ask “how do you ensure changes don’t introduce regressions?”, we have a concrete answer: 24,000+ tests, every commit, with a coverage gate that fails builds below 99%.
Competitive Signal
No other AI automation platform publishes test metrics. Not Zapier (enterprise-scale but closed-source quality practices), not n8n (8 CVEs in early 2026), not Make, not any of the new AI agent platforms. Publishing our test count isn’t marketing — it’s accountability.
When we say JieGou is enterprise-ready, the test suite is the evidence. When we say a feature works, there are hundreds of tests proving it.
How Quality Scales
The key insight is that test count should grow faster than feature count. Every new feature adds tests, but it also adds tests for interactions with existing features. A new messaging channel doesn’t just need channel-specific tests — it needs tests for how that channel interacts with FAQ matching, confidence scoring, approval gates, audit logging, and RBAC.
This multiplicative effect is why the test count doubled while the feature count grew linearly. It’s also why platforms that skip testing early find it progressively harder to add features reliably — technical debt compounds.
Our approach:
- Test-first for server logic. Every new function in
src/lib/server/gets tests before or alongside implementation. - Mock-heavy for LLM interactions. Deterministic mocks for all providers, so tests are fast and reproducible.
- E2E for critical paths. Browser automation for the journeys that matter most: onboarding, workflow creation, execution, and approval flows.
- Nightly regression suite. The full suite runs every night across all configurations, catching drift that incremental CI might miss.
What’s Next
We’re not slowing down. The roadmap includes more messaging channels, deeper MCP integrations, and expanded governance features. Each will bring more tests. Our target is to maintain coverage above 99% while continuing to ship weekly.
The test count is a trailing indicator of product quality. The leading indicator is that enterprises can deploy JieGou automations to production with confidence — because every template, every workflow step, and every governance control has been tested before it reaches their team.
24,000+ tests and counting.