Competitive Intelligence App Overview

Turn competitor websites into strategic advantage — automatically crawl, analyze, and act on competitive insights with AI-powered intelligence that never sleeps.

---

What is the Competitive Intelligence App?

The Competitive Intelligence App is a comprehensive, AI-native platform that continuously monitors competitor websites, extracts actionable insights, and helps your team stay ahead of the competition. It automatically discovers and crawls competitor pages, uses AI to analyze content, identifies gaps in your coverage, scores competitor threats, and even generates draft content to close those gaps — all from a single dashboard.

Core Value Proposition:

• 🎯 Automated Monitoring — Continuously crawl and track competitor websites with intelligent scheduling
• 🧠 AI-Powered Analysis — Extract topics, claims, statistics, and competitive references using GPT-4o
• 📊 Strategic Insights — Identify content gaps, market themes, and competitor threat levels
• ✍️ Content Generation — Generate AI-powered draft content grounded in competitive data and your brand voice

---

At a Glance

🔍 Crawl Engine	🧠 AI Analysis	📄 Content Types	🤖 AI Chat
Sitemap-based discovery	GPT-4o / GPT-4o-mini	13 page types classified	Conversational Q&A

Perfect For:

• 🏢 System Administrators — Configure competitors, run pipelines, and monitor intelligence across the competitive landscape
• 📈 Product Marketing Teams — Identify content gaps, generate battle cards, and create competitive response content
• 🎯 Strategy & Leadership — Assess competitor threats, track market themes, and receive AI-generated competitive digests

---

How It Works

Intelligence Pipeline

┌─────────────────────────────────────────────────────────────────────────────┐
│                    COMPETITIVE INTELLIGENCE PIPELINE                        │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   ┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────┐             │
│   │  SEED    │───▶│ DISCOVER │───▶│  FETCH   │───▶│ ANALYZE  │             │
│   │Competitors│   │  URLs    │    │ Content  │    │ with AI  │             │
│   └──────────┘    └──────────┘    └──────────┘    └──────────┘             │
│                                                         │                   │
│                                                         ▼                   │
│   ┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────┐             │
│   │  ACT     │◀───│ GENERATE │◀───│ IDENTIFY │◀───│  EMBED   │             │
│   │  on it   │    │  Drafts  │    │   Gaps   │    │ Vectors  │             │
│   └──────────┘    └──────────┘    └──────────┘    └──────────┘             │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Pipeline Steps:

1. Seed Competitors — Import competitor websites from a configuration file with categories and URL filters
2. Discover URLs — Parse competitor sitemaps to find all available pages (fast, no content download)
3. Fetch Content — Download page content with incremental updates (conditional GET with ETag/Last-Modified)
4. AI Analysis — Classify content type, extract topics/claims/statistics, assess competitive intensity
5. Generate Embeddings — Create pgvector embeddings for semantic similarity search (1536 dimensions)
6. Identify Gaps — Compare competitor content coverage against your own to find content gaps
7. Generate Drafts — Create AI-powered content drafts grounded in competitive data
8. Act — Publish drafts to blog, create action items, respond to competitive moves

Content Ingestion Sources

                          ┌─────────────────────┐
                          │  CONTENT INGESTION   │
                          └──────────┬──────────┘
                                     │
         ┌───────────────┬───────────┼───────────┬───────────────┐
         ▼               ▼           ▼           ▼               ▼
   ┌───────────┐  ┌───────────┐ ┌─────────┐ ┌─────────┐  ┌───────────┐
   │  Sitemap  │  │  Manual   │ │Document │ │  Video  │  │  Config   │
   │  Crawling │  │   URLs    │ │ Upload  │ │ Upload  │  │   File    │
   └───────────┘  └───────────┘ └─────────┘ └─────────┘  └───────────┘

---

Key Features

🔍 Automated Web Crawling

The crawler engine automatically discovers and fetches competitor content using intelligent sitemap parsing and incremental crawl strategies.

Feature	Description
Sitemap Discovery	Automatically finds and parses XML sitemaps, sitemap indexes, and compressed .xml.gz files
Incremental Fetching	Conditional GET with ETag/Last-Modified headers — only re-downloads changed content
Rate Limiting	Adaptive rate limiting (0.05s own sites, 0.2s competitors) with backoff up to 2.0s
Multi-threaded Fetching	5 concurrent threads for fast batch processing (up to 10,000 pages per run)
Content Extraction	Strips boilerplate (nav, footer, ads, chat widgets) and converts to clean Markdown
Page Type Classification	Rule-based classifier identifies 9 page types from URL patterns (blog, case study, product, pricing, etc.)
Failure Recovery	Exponential backoff on failures (1h → 4h → 16h → 64h, capped at 72h)
URL Filtering	Path filters and exclude patterns to focus crawling on relevant content

Use Case: Add a competitor’s domain once, and the system continuously tracks all their new blog posts, case studies, product pages, and more — automatically detecting when content changes.

---

🧠 AI-Powered Content Analysis

Every crawled page is analyzed by GPT-4o-mini to extract structured intelligence that feeds into competitive insights.

Feature	Description
Topic Extraction	Identifies key topics and messaging themes from each page
Claim Detection	Extracts specific claims and value propositions (with boilerplate filtering)
Statistics Extraction	Pulls out data points, percentages, and metrics cited by competitors
Competitive References	Detects when competitors mention other companies (including you)
Content Quality Scoring	Rates content quality on a 0.0–1.0 scale
Competitive Intensity	Classifies pages as direct comparison, indirect positioning, neutral, or thought leadership
Industry Vertical Tagging	Tags content with industry verticals from 12 predefined categories
Batch Processing	Analyzes 15 pages per LLM call with 5 parallel threads for throughput

Use Case: Instead of manually reading hundreds of competitor blog posts, get structured summaries showing exactly what topics they’re pushing, what claims they’re making, and which industries they’re targeting.

---

📊 Content Gap Identification

Automatically identifies topics and themes that competitors cover but your company doesn’t — prioritized by competitive urgency.

Feature	Description
Gap Detection	SQL-based aggregation comparing competitor topics against your own content
Gap Types	Classifies gaps as content gaps, opportunities, or threats
Priority Scoring	Ranks gaps by priority (critical, high, medium, low) based on competitor coverage count
AI Recommendations	GPT-4o-mini generates article type, title, outline, and target audience for each gap
Gap Deduplication	pgvector cosine similarity (0.85 threshold) clusters and merges similar gaps
Status Workflow	Track gaps through actionable → idea created → dismissed lifecycle
Competitor Evidence	Shows which competitors cover each gap topic with evidence links

Use Case: Discover that 4 competitors have published case studies about “AI-powered employee onboarding” but you haven’t — and get an AI-generated article outline to close the gap.

💡 Pro Tip: Review content gaps weekly to stay ahead of competitor content strategies. Gaps flagged by multiple competitors often represent market-wide trends you shouldn’t ignore.

---

🏆 Competitor Profiles & Battle Cards

AI-generated competitor profiles synthesize analysis from hundreds of pages into actionable battle cards for sales and marketing teams.

Feature	Description
Profile Generation	GPT-4o synthesizes page analyses into comprehensive competitor profiles
Battle Cards	Elevator pitch, ideal customer profile, talking points, objection handling
Competitive Landmines	Questions designed to expose competitor weaknesses
Killer Questions	Questions that highlight your advantages over the competitor
Win/Loss Themes	Tracks why deals are won or lost against each competitor
Strengths & Weaknesses	SWOT-style analysis based on actual content evidence
Coverage Comparison	“They cover / We cover / Overlap” analysis versus your own content
Field Pinning	Pin important profile fields so they persist across regeneration cycles

Use Case: Before a sales call against a specific competitor, pull up their battle card for talking points, objection handling scripts, and killer questions to ask the prospect.

---

🎯 Threat Assessment & Scoring

AI-powered threat scoring evaluates each competitor’s strategic risk on a 0–100 scale with component-level detail.

Feature	Description
Threat Score	0–100 score with levels: critical (80+), high (60-79), moderate (40-59), low (<40)
Component Scores	Market overlap, competitive intensity, content momentum, feature parity, strategic alignment
Next Moves Prediction	AI predicts likely competitor actions, recommended responses, and key battlegrounds
Staleness Detection	Flags assessments older than 7 days for re-evaluation
Strategic Stances	Classify competitors as Displace, Co-exist, Monitor, or Ignore
Heuristic Fallback	If LLM assessment fails, falls back to rule-based threat scoring

Use Case: Get alerted when a competitor’s threat score jumps from “moderate” to “high” due to a surge in product content, and receive AI-recommended responses.

---

🔔 Change Detection & Events

Continuously monitors competitor content for significant changes and creates structured change events for review and response.

Feature	Description
New Page Detection	Tracks newly discovered competitor pages with relevance assessment
Content Change Tracking	Detects when existing pages are modified
Gap Escalation Alerts	Alerts when more competitors start covering a gap topic
Watch Topic Monitoring	Flags pages matching your strategic watch topics
Significance Levels	High, medium, low classification based on competitor stance and page type
Acknowledge Workflow	Mark events as acknowledged to track review status
Playbook Generation	AI generates response playbooks for significant competitive moves

Use Case: A competitor publishes a new pricing page — the system detects it, classifies it as high-significance, and generates a response playbook with immediate actions and messaging guidance.

---

📰 Competitive Digests

AI-generated periodic briefings that summarize the competitive landscape — delivered weekly or monthly.

Feature	Description
Weekly Digests	Covers last 7 days: new pages, content changes, action items, high-priority gaps
Monthly Deep Digests	Extended analysis with competitor activity rankings and trend review
Weather Forecast	Overall competitive weather (sunny → stormy) with per-priority outlook
Highlights & Recommendations	3–5 key findings and 3–5 recommended actions per digest
Markdown Output	Full markdown digest stored for reference and distribution
Fallback Generation	If LLM fails, generates a rule-based summary from available data

---

📈 Market Themes & Signals

Track emerging market themes and detect competitive signals before they become trends.

Feature	Description
Theme Tracking	Identifies common themes across competitor content with trend direction
Trend Direction	Growing, stable, or declining based on historical data
Saturation Scoring	Measures how many competitors are investing in each theme
Emerging Topic Detection	Finds brand-new topics or topics with >2x acceleration
Investment Signals	Detects page-type shifts (product, pricing, case study) indicating strategic investment
Positioning Shift Detection	Compares recent content themes against established profile to detect pivots

---

✍️ Action Items & Draft Generation

Convert competitive insights into actionable content with AI-powered draft generation in your brand voice.

Feature	Description
Auto-Generated Action Items	Created from content gaps, market themes, competitor announcements, and feature comparisons
Action Types	Write content, press release, competitive response, social share, feature request, research
Priority Assignment	Automatic priority based on competitor count, gap type, and competitive intensity
AI Draft Generation	GPT-4o generates full article drafts grounded in competitive data from 11 context sources
Brand Voice Integration	Drafts match your writing style via auto-detected brand voice profiles
Auto-Grading	Drafts are automatically graded; those scoring below 70 are auto-rewritten
Draft Workflow	Suggested → Accepted → Draft Generating → Draft Available → Ready → Published
Blog Publishing	One-click publish completed drafts directly to your blog

Use Case: A content gap is identified, an action item is auto-generated, you accept it, AI writes a draft article in your brand voice with competitive data woven in, and you publish it directly — all from the CI dashboard.

💡 Pro Tip: Bootstrap your brand voice early by ensuring your own website content is crawled first. The system analyzes your existing blog posts and case studies to learn your writing style automatically.

---

🔗 Feature Comparison Matrix

Understand exactly where your product stands versus competitors across feature categories.

Feature	Description
Automated Feature Extraction	GPT-4o-mini extracts product features from competitor marketing pages
Feature Categories	13 categories: communication, collaboration, HR, analytics, AI, security, integrations, and more
Evidence Tracking	Links features to specific competitor pages as evidence
MangoApps Matching	Automatically identifies which features MangoApps already has from own-content analysis
Gap & Advantage Analysis	Shows which features you have that competitors don’t (advantages) and vice versa (gaps)
Manual Override	Update feature support status manually for accuracy

---

🤖 AI Chat Interface

Conversational AI assistant with access to all competitive intelligence data through 14 specialized tools.

Tool	Capability
Search Competitor Pages	Semantic search over all competitor content using pgvector embeddings
Get Competitor Profile	Full profile with positioning, battle card, and threat assessment
Search Content Gaps	Query content gaps with filtering
Get Market Themes	Retrieve market themes with trend direction
Compare Competitors	Side-by-side comparison of competitors on any topic
Get Feature Matrix	Feature comparison data across competitors
Get Recent Changes	Latest content changes and new pages
Get Strategy Context	Your company’s competitive strategy
Generate Briefing	On-demand competitive briefing on any topic or competitor
Search Intel Notes	Query human-submitted intelligence notes
Add Intel Note	Submit new intelligence notes (win/loss data, observations)
Get Threat Assessment	Competitor threat score and strategic assessment
Get Change Events	Recent competitive change events
Get Statistics	Data points and statistics from competitor content

Use Case: Ask “What are Simpplr’s key differentiators versus us in the intranet space?” and get an AI-synthesized answer grounded in actual crawled competitor content, not generic knowledge.

---

🎙️ Brand Voice Detection

Automatically detect and maintain your company’s writing style for consistent content generation.

Feature	Description
Auto-Bootstrap	Analyzes your existing blog posts and case studies to learn your writing style
Sample Selection	Prefers blog posts, case studies, and resources with 200+ words and high quality scores
Style Analysis	Runs AI-powered writing style analysis to create a writing profile
Refresh on Demand	Re-analyze brand voice as your content evolves
Draft Integration	All generated drafts automatically use your detected brand voice

---

🎯 Strategy Management

Define and maintain your competitive positioning to personalize all AI-generated insights.

Feature	Description
AI Strategy Bootstrap	GPT-4o-mini synthesizes your content + competitor landscape into a draft strategy
Company Positioning	Define key differentiators, target segments, and product strengths/weaknesses
Strategic Priorities	Set priorities that influence gap scoring and change event significance
Watch Topics	Define topics to monitor — get high-significance alerts when competitors mention them
OKRs	Set competitive objectives and key results
Stance Suggestions	AI suggests which competitors to displace, co-exist with, monitor, or ignore
LLM Context	Strategy data automatically injected into all AI prompts for personalized analysis

---

🔎 Semantic Similarity Search

pgvector-powered semantic search across all competitor content for finding related content and identifying gaps.

Feature	Description
Text Query Search	Find competitor pages similar to any text query
Blog Post Similarity	Find competitor content similar to your existing blog posts
Page-to-Page Similarity	Find pages similar to any specific CI page
Content Gap Candidates	Automatically find high-quality competitor pages with no similar content from you
Content Clustering	Cluster competitor pages by semantic similarity to find content themes

---

📝 Intel Notes

Capture human intelligence from sales, support, and other teams to enrich AI-generated insights.

Feature	Description
Note Types	Observations, win/loss data, corrections, internal context, market intel
Win/Loss Tracking	Record deal outcomes, decisive factors, deal size, and segment
Confidence Levels	Rate intelligence confidence for AI weighting
Polymorphic Attachment	Attach notes to any CI entity (competitor, page, gap, etc.)
AI Integration	Notes are included in draft generation context and chat agent tools

---

📋 Response Playbooks

AI-generated competitive response playbooks for when competitors make strategic moves.

Feature	Description
Event-Triggered	Generate playbooks from detected change events
Scenario-Based	Pre-built scenarios: new feature, case study, pricing change, new integration, market entry, executive hire
Structured Output	Title, urgency, impact assessment, immediate actions (24–48h), short-term actions (1–2 weeks)
Messaging Guidance	Recommended messaging and talking points for sales and marketing
Watch Indicators	What to monitor after executing the playbook

---

User Roles & Permissions

Role	Capabilities
System Administrator	Full access: configure competitors, run pipeline, view all intelligence, manage strategy, use AI chat, generate content, publish drafts

Note: The Competitive Intelligence module is a system admin tool accessed via the System Admin panel. It is designed for platform-level competitive monitoring across the entire business.

---

How We Compare

See how MangoApps Workforce Competitive Intelligence stacks up against leading CI solutions:

Feature	MangoApps Workforce	Crayon	Klue	Kompyte (Semrush)
Automated Web Crawling	✅	✅	✅	✅
AI Content Analysis	✅	✅	⚡ Limited	⚡ Limited
Battle Card Generation	✅	✅	✅	✅
Content Gap Identification	✅	❌	❌	❌
AI Draft Content Generation	✅	❌	❌	❌
Threat Scoring (0–100)	✅	❌	❌	❌
AI Chat with CI Data	✅	❌	❌	❌
Brand Voice Detection	✅	❌	❌	❌
Semantic Similarity Search	✅	❌	❌	❌
Response Playbook Generation	✅	❌	⚡ Limited	❌
Competitive Digest Reports	✅	✅	✅	✅
Feature Comparison Matrix	✅	❌	✅	❌
Intel Notes (Human Input)	✅	✅	✅	❌
Market Theme Tracking	✅	⚡ Limited	❌	⚡ Limited
One-Click Blog Publishing	✅	❌	❌	❌

Legend: ✅ Included

❌ Not Available

⚡ Limited

Why MangoApps Workforce?

• 🔗 Unified Platform — Competitive Intelligence works seamlessly with Blog, AI Writer, and Writing Profiles in one system
• 💰 No Hidden Costs — All features included, no per-seat CI tool pricing ($30–50/user/month with competitors)
• 🤖 AI-Native — Built-in GPT-4o analysis, not bolted-on AI features
• ✍️ Content Action Loop — Only CI tool that goes from insight → draft → publish in one platform

---

Getting Started

For System Administrators

1. Navigate to System Admin → Competitive Intelligence
2. Seed Competitors — Click “Run Pipeline Action” → “Seed Competitors” to import from the config file
3. Discover & Fetch — Run “Discover URLs” then “Fetch Pages” to crawl competitor content
4. Analyze — Run “Analyze Pages” to process content with AI
5. Generate Embeddings — Run “Generate Embeddings” to enable semantic search
6. Review Dashboard — Check the dashboard for content gaps, market themes, and competitor stats

First-Time Setup (Recommended Order)

1. Add your own site — Ensure your company’s website is included as an “own company” competitor so the system can compare content
2. Run Full Pipeline — Use the “Full Pipeline” action to run all steps in sequence
3. Bootstrap Strategy — Go to “Our Profile” → “Strategy” → “Bootstrap Strategy” to auto-generate competitive positioning
4. Bootstrap Brand Voice — Go to “Our Profile” → “Brand Voice” → “Auto-Detect” to learn your writing style
5. Set Competitor Stances — Classify each competitor as Displace, Co-exist, Monitor, or Ignore
6. Define Watch Topics — Add topics you want to be alerted about in your strategy settings
7. Explore the AI Chat — Ask questions about your competitive landscape using natural language

---

Best Practices

• ✅ Run the pipeline weekly — The scheduled job handles this automatically, but you can trigger manual runs anytime
• ✅ Set competitor stances early — This personalizes threat scoring, change event significance, and pipeline scheduling
• ✅ Define strategic watch topics — Ensures high-significance alerts for topics that matter most to your business
• ✅ Add intel notes regularly — Human intelligence from sales wins/losses enriches AI analysis significantly
• ✅ Review content gaps weekly — Gaps flagged by multiple competitors often represent market-wide trends
• ✅ Bootstrap brand voice before generating drafts — Ensures AI-generated content matches your writing style
• ✅ Use exclude patterns — Filter out irrelevant URLs (login pages, legal pages) to keep analysis focused
• ✅ Pin important battle card fields — Pinned fields persist across profile regeneration cycles

---

Frequently Asked Questions

Q: How does the system know which pages to crawl?
A: The crawler starts by parsing each competitor’s XML sitemap (trying common paths like /sitemap.xml, /sitemap_index.xml). It then applies path filters and exclude patterns you’ve configured to focus on relevant content. Pages are classified by type (blog, case study, product, etc.) using URL pattern matching.

Q: How often is competitor content refreshed?
A: The system uses conditional GET requests (ETag/Last-Modified) to efficiently check for changes. A weekly scheduled pipeline discovers new URLs, fetches changed content, and runs AI analysis. You can also trigger manual runs anytime from the dashboard. Crawl refresh intervals are configurable per competitor via crawl policy settings.

Q: Can I add content that isn’t on a website?
A: Yes. The “Add Content” feature supports manual URL entry, PDF document uploads, and video uploads. These are processed through the same AI analysis pipeline as crawled web pages.

Q: How does the AI chat know about my competitive landscape?
A: The AI chat agent has access to 14 specialized tools that query your actual CI data — competitor profiles, crawled page content via semantic search, content gaps, market themes, feature comparisons, change events, and more. Responses are grounded in real data, not generic AI knowledge.

---

Related Resources

• AI Writer — Generate content using AI with competitive insights
• Writing Profiles — Manage brand voice profiles used by CI draft generation
• Apps Overview — Browse all available MangoApps Workforce apps

---

Stay ahead of the competition — let AI do the watching while you focus on winning.