EVOLVE AI Assistant
Production AI system for the #1 e-commerce community on Skool
Impact & Results
The Challenge
EVOLVE is the #1 Skool community teaching advanced Shopify sellers to scale to $100k/day. With 1000+ active members, they needed:
- 24/7 expert support that couldn't be delivered manually at scale
- 110K+ Discord messages + Skool course materials searchable for accurate, cited answers
- Zero tolerance for errors - wrong advice at $100k/day scale means major financial risk
The Solution
Production AI system with three components:
1. Discord Bot ("Chad")
- RAG with dual-retrieval (FTS + semantic)
- Multi-stage verification + hallucination detection
- Question decomposition for complex queries
- Forced citations for every claim
- Smart retry with gap-based refinement
2. Analytics Dashboard
- Real-time metrics and engagement analytics
- Semantic heatmap of conversation topics
- Knowledge base management with AI search
- Channel configuration and settings
3. Content Agent
- Discord search for community insights
- X/Twitter trend analysis
- Semantic content mapping
- Memory for iterative ideation
Technical Architecture
Multi-stage RAG pipeline prioritizing accuracy:
Question Analysis
Breaks complex questions into 1-3 subquestions. Extracts entities and intent.
Dual Retrieval
FTS for precision + semantic search for coverage. Top 10 from each, deduplicated.
LLM Selection
Filters for truly relevant spans. Generates gap queries if coverage is low.
Document Integration
Adds full docs (top 3) + highlights (up to 10 excerpts) from knowledge base.
Generate & Cite
Creates answers with forced inline citations [C1], [D2] for every claim.
Verify & Retry
Confidence scoring + hallucination check. Auto-retry if confidence < 0.8.
Key Innovations
Question Decomposition
Breaks complex questions into subquestions for separate retrieval. Dramatically improves completeness.
Gap-Based Retry
Auto-generates targeted queries when coverage is low. Self-correcting without manual intervention.
FTS + Semantic Hybrid
FTS for precision on keywords, semantic for broad coverage. Best of both worlds.
LLM Selection Filter
Extracts only truly relevant spans instead of relying on similarity scores alone.
Semantic Heatmap
Visualize conversation topics and trends
Analyzes all Discord messages to create interactive heatmaps showing topic clusters and how conversations evolve over time.
Topic Clusters
Auto-groups similar conversations. See ad strategies vs product selection vs scaling tactics at a glance.
Trend Analysis
Weekly/monthly views show shifting interests. Spot emerging topics early.
Content Strategy
Shaun & Spencer use this to guide social media and course content creation.
Interactive
Click clusters to see actual messages with full context.
Technology Stack
Backend
Frontend
Database
Infrastructure
Ready to build your AI solution?
Get in touch to discuss how we can help your business.
Get in touch