Insights — Improvement & Growth Plan

Package: capell-app/insights · Kind: extension (admin + frontend) · Tier: premium · Product group: Capell Growth · Bundle: growth · Status: Complete

1. Snapshot

Insights is a first-party, consent-aware web-analytics extension for Capell. The frontend surface ships a vanilla JS tracker and overrideable consent banner injected via a RenderHookLocation::BodyEnd hook (src/Support/RenderHooks/RegisterInsightsTrackerHook.php) plus two public POST endpoints (src/Http/Controllers/InsightsBeaconController.php, InsightsConsentController.php) backed by Actions (RecordInsightsEventsAction, ValidateInsightsBeaconRequestAction, UpdateInsightsConsentAction, CreateInsightsVisitAction, MirrorInsightsConsentToPrivacyCenterAction). The admin surface adds an InsightsPage, seven dashboard widgets, three overview stats, daily aggregate rollups, and a settings schema, fed by Build*QueryAction aggregates over four tables (insights_visits, insights_consents, insights_events, insights_daily_rollups). Deps: lorisleiva/laravel-actions, spatie/laravel-data, spatie/laravel-package-tools; requires capell-app/admin|core|frontend, supports capell-app/privacy-center.

Current marketplace summary (verbatim): “Cookie-light, GDPR-aware web analytics built into your Capell admin — page views, clicks, visitor journeys, and consent, with no third-party scripts and no data leaving your server.” The manifest now lists the extension card, the settings screen, and two styled route-backed public fixture screenshots proving the frontend tracker and packaged consent banner render through the Insights BodyEnd hook. The blank dashboard/widget captures remain committed runner evidence but are no longer promoted as buyer-facing media until analytics demo data produces populated Capell widgets.

2. Improvements (existing functionality)

• Done/Shipped: JS tracker bootstraps a visit on the first recordable hit. The beacon action now creates a visit when no visit_id is supplied, the server-resolved region is outside UK/Europe, consent is not required for all regions, and the batch contains at least one non-ignored event. The events endpoint returns the fresh visit_id, the tracker forces fetch instead of sendBeacon for that first flush, and then stores the id for later beacon batches. Focused coverage documents outside-region creation, consent-required suppression, and the first-flush JS response path. — src/Actions/RecordInsightsEventsAction.php, src/Http/Controllers/InsightsBeaconController.php, resources/js/capell-insights.js — M
• Closed: derive hash_salt from APP_KEY by default — ResolveInsightsHashSaltAction now treats empty, whitespace-only, and legacy public capell-insights salts as placeholders, derives the visitor hash salt from a usable APP_KEY, and falls back to the public legacy salt only when no usable app key exists. InsightsHealthCheck fails that unsafe fallback state without exposing configured secrets in diagnostics. — privacy hardening — src/Actions/ResolveInsightsHashSaltAction.php, src/Actions/CreateInsightsVisitAction.php, src/Actions/UpdateInsightsConsentAction.php, src/Health/InsightsHealthCheck.php — S
• Done/Shipped: ConsentRegionResolver is wired server-side. InsightsConsentController ignores the client-supplied region for trust decisions and passes ResolveConsentRegionAction::run() into UpdateInsightsConsentAction; the beacon endpoint now uses the same server-side region resolution before creating first visits. — fixes attacker-controllable GDPR jurisdiction + removes dead code — src/Http/Controllers/InsightsConsentController.php, src/Http/Controllers/InsightsBeaconController.php, src/Actions/CreateInsightsVisitAction.php, src/Support/Consent/ConsentRegionResolver.php — M
• Done/Shipped: chunked retention purge. PurgeInsightsDataAction::handle now deletes eligible events, consents, and visits through bounded ID batches instead of unbounded table deletes. Batch size defaults to purge_batch_size and can be overridden when the Action is called, keeping the purge command behavior stable while reducing lock/replication risk on high-write installs. — src/Actions/PurgeInsightsDataAction.php, config/capell-insights.php — S
• Done/Shipped: cache dashboard aggregates. Dashboard aggregate Actions now remember materialized results behind stable keys that include locale/window/site/language/limit inputs, using dashboard_cache_ttl_seconds and the insights cache tag when the current store supports tags. Event recording, consent updates, and retention purge writes flush the tagged dashboard cache on tag-capable stores; non-tag stores still get bounded TTL protection. — src/Actions/RememberInsightsDashboardAggregateAction.php, src/Actions/Build*QueryAction.php, src/Actions/RecordInsightsEventsAction.php, src/Actions/PurgeInsightsDataAction.php — M
• Done/Shipped: Reconcile registered vs on-disk migrations — InsightsServiceProvider::configurePackage now registers only the four migration files present in database/migrations/, and InsightsMigrationsTest asserts every registered migration name has a file on disk. — fresh-install hazard closed — src/Providers/InsightsServiceProvider.php, tests/Feature/Database/InsightsMigrationsTest.php — S
• Done/Shipped: Flesh out the health check. InsightsHealthCheck now reports storage-table presence, beacon route registration, frontend tracker render-hook output, monthly retention purge scheduling, and non-default visitor hash secret diagnostics, with pass/fail coverage. — Diagnostics value — src/Health/InsightsHealthCheck.php, tests/Feature/Health/InsightsHealthCheckTest.php — M
• Done/Shipped: session boundaries for journeys. RecordInsightsEventsAction now rotates a persistent visit cookie into a fresh InsightsVisit when last_seen_at is older than session_timeout_minutes (30 minutes by default). The new visit preserves consent region/status and receives the queued cookie, so event sequences restart per session and journey widgets no longer merge returning visitor activity into one historical timeline. — src/Actions/RecordInsightsEventsAction.php, config/capell-insights.php, capell.json — M
• Done/Shipped: daily rollups for long-range reports. The package now owns an insights_daily_rollups table plus RebuildInsightsDailyRollupsAction and insights:rollups:rebuild command, scheduled daily. Popular and trending page reports use day/path/type aggregate rows for day-aligned windows and fall back to raw events until aggregates exist, keeping fresh installs correct while reducing long-range dashboard scans. — database/migrations/2026_06_06_000001_create_insights_daily_rollups_table.php, src/Actions/RebuildInsightsDailyRollupsAction.php, src/Actions/BuildPopularPagesQueryAction.php, src/Actions/BuildTrendingPagesQueryAction.php — L
• Done/Shipped: digest and CSV export seam. BuildInsightsDigestAction assembles overview stats, popular pages, acquisition sources, and funnel conversion data for a window; ExportInsightsDigestCsvAction serializes the digest without reintroducing public render work or admin query coupling. — src/Actions/BuildInsightsDigestAction.php, src/Actions/ExportInsightsDigestCsvAction.php — M

3. Missing Features (gaps)

Tied to capabilities[] = insights, insights-admin, insights-frontend, insights-consent-banner, insights-acquisition-reports, insights-conversion-funnels, insights-traffic-filtering, insights-session-journeys, insights-daily-rollups, insights-privacy-signals, insights-privacy-center-mirror, beacon, cache-blocking.

• Done/Shipped: consent banner / opt-in UI. The BodyEnd hook now renders an overrideable capell-insights::components.consent-banner view by default, gated by consent_banner_enabled. The tracker shows it when no current-policy consent decision is stored, supports accept/reject/granular choices, posts to the consent endpoint, stores the decision locally with policy_version, and hides the banner after a successful response. — resources/views/components/consent-banner.blade.php, resources/js/capell-insights.js, config/capell-insights.php
• Done/Shipped: Referrer & UTM analytics. BuildAcquisitionSourcesQueryAction now surfaces stored utm_source, utm_medium, utm_campaign, and referrer_url data in an Acquisition Sources dashboard widget. The report groups visits by campaign, referrer host, or direct traffic; it is cache-keyed by the dashboard window and declared as insights-acquisition-reports in the manifest. — src/Actions/BuildAcquisitionSourcesQueryAction.php, src/Filament/Widgets/AcquisitionSourcesWidget.php, capell.json
• Done/Shipped: pre-aggregated rollups / retention tiers (differentiator). Daily rollups now aggregate path × day × type counts for popular/trending page dashboards, with raw-event fallback until aggregates exist. Deeper separate retention tiers for raw-vs-aggregate data remain future depth.
• Done/Shipped: funnels & conversions (differentiator). Companion packages can now call RecordConversionAction to record typed conversion events with source package/value/currency metadata, and BuildFunnelConversionReportAction summarizes named event steps by unique visits and conversion rate. The beacon metadata contract also allows conversion metadata keys for browser/custom integrations. — src/Actions/RecordConversionAction.php, src/Actions/BuildFunnelConversionReportAction.php, src/Data/InsightsEventMetadataData.php
• Done/Shipped: bot / self-traffic filtering. RecordInsightsEventsAction now suppresses ignored IPs and user-agent patterns before creating first visits or appending events to existing visits. The package config ships default bot, crawler, monitor, Lighthouse, PageSpeed, Pingdom, GTmetrix, and headless-browser patterns, with ignored_ips available for agency/staff office traffic. — src/Actions/RecordInsightsEventsAction.php, config/capell-insights.php, capell.json
• Visit duration / bounce / entry-exit (table-stakes). started_at/last_seen_at exist on the visit but no engaged-time, bounce-rate, or entry/exit-page metrics are computed.
• Done/Shipped: consent expiry & re-prompt. Browser decisions are stored with policy_version, the packaged banner reappears when the configured policy version changes, and the server refuses analytics recording in consent-required regions when the latest consent is for an old policy or older than consent_expires_days.
• Done/Shipped: server-side event API surface for other packages. RecordConversionAction gives seo-suite/campaign-studio style packages an explicit in-process analytics contract without exposing a public unauthenticated write API. BuildFunnelConversionReportAction provides the matching reporting surface for bundle dashboards.
• Done/Shipped: digest/export reporting. First-party analytics can now be summarized as a typed digest and exported as CSV, giving GA4 Reports, Campaign Studio, Email Studio, and future scheduled report consumers a consistent report seam.
• Done/Shipped: Do-Not-Track / Global Privacy Control honoring and server-side re-consent (differentiator). honor_privacy_signals now defaults on. The browser tracker exits before registering listeners when navigator.globalPrivacyControl, navigator.doNotTrack, or navigator.msDoNotTrack is active, and the beacon endpoint returns no-content without validation/persistence for Sec-GPC: 1, DNT: 1, or X-Do-Not-Track: 1. Consent-required regions now record analytics only when the latest consent matches the configured policy_version and remains inside consent_expires_days. — resources/js/capell-insights.js, resources/views/tracker.blade.php, src/Actions/ValidateInsightsBeaconRequestAction.php, src/Actions/RecordInsightsEventsAction.php

4. Issues / Risks

• Done/Shipped: first-visit data loss is fixed for no-consent-required regions. See §2 item 1. The no-visit beacon path now creates a visit, records the first non-ignored event, returns a visit id, and refuses to create visits when the server region still requires consent.
• Done/Shipped: GDPR region is resolved server-side. InsightsConsentController and the first-visit beacon path both use ResolveConsentRegionAction; client-supplied region remains accepted as request shape only, not as the trust source for consent enforcement.
• Reversible visitor hashes (PII). Closed for default installs: empty or legacy hash_salt values now derive from APP_KEY, and Diagnostics fails if the package would fall back to the public legacy salt. Remaining risk is limited to installs running without a usable APP_KEY and without a private hash_salt.
• Cache-safety of the beacon: OK. The render hook output is static config + a file-read script cached rememberForever keyed by file mtime (src/Actions/GetInsightsTrackerScriptAction.php); the blade does no DB query and leaks no model IDs/labels/permissions (resources/views/tracker.blade.php) — consistent with cacheSafety.cacheable:false, sensitiveOutput:false. The render hook now suppresses tracker output on configured ignored_paths such as /admin*, matching the server-side beacon ignore policy and avoiding admin/Livewire script emission. — src/Support/RenderHooks/RegisterInsightsTrackerHook.php, tests/Feature/Frontend/InsightsRenderHookTest.php
• Closed: migration drift. InsightsMigrationsTest now asserts all registered migration names exist on disk, and the provider no longer references phantom 06/site-FK migrations.
• Closed: core manifest mismatches. capell.json now declares Capell\Insights\Settings\InsightsSettings, the generated Shield page permission View:InsightsPage, and the shipped marketplace screenshot set. ManifestRequirementsTest asserts those declarations and the screenshot files. commands.doctor remains null because no package command ships; Diagnostics consumes the health-check class directly.
• Closed: single-event and legacy import Actions retained intentionally. The single-event family is not on the browser beacon request path, but it remains a supported server-side integration path: RecordConversionAction calls RecordCustomActionAction, docs/tracking-and-consent.md documents custom server-side events, and ImportLegacyPageViewsAction is the migration-owned idempotent importer for legacy page_views. Removing these would regress package integrations and fresh-upgrade migration behavior.
• Performance budget realism. frontendRenderBudgetMs:20 is plausible (cached script), short-TTL dashboard aggregate caching reduces repeated widget scans, and daily rollups now replace raw page-event scans for day-aligned popular/trending dashboards. High write volume on insights_events remains the principal scaling risk for live and custom-event reports.
• i18n. Strings are translated via capell-insights:: keys (good), but the config key track_form-builder contains a hyphen and is mirrored as a setting/label key — brittle and inconsistent with snake_case siblings (config/capell-insights.php, src/Filament/Settings/InsightsSettingsSchema.php:34).
• Test gaps. First-visit auto-creation, consent-required suppression, server-side region resolution, manifest declarations, render-hook banner output, admin-path render-hook suppression, health-check render/schedule probes, JS banner primitives, chunked retention purge behavior, dashboard cache helper behavior, and acquisition-source grouping have focused coverage. Remaining gaps: browser-level consent-banner interaction coverage and deeper long-window rollup coverage.

5. Marketplace & Selling

Critique. The manifest summary and package description now lead with the buyer benefit (first-party, GDPR-aware analytics without third-party scripts). The gallery references the extension card, settings screen, and two styled package-owned public fixture captures for the active tracker and consent-banner flow. Remaining media upside is optional: recapture populated dashboard/widgets before promoting admin analytics screenshots, and promote dark admin alternates only if Marketplace wants explicit dark-mode gallery entries.

Improved 1-sentence summary. “Cookie-light, GDPR-aware web analytics built into your Capell admin — page views, clicks, visitor journeys, and consent, with no third-party scripts and no data leaving your server.”

Improved 3–4 sentence description. “Insights gives Capell sites first-party traffic analytics that respect privacy by default: it records page views, clicks, and visitor journeys server-side, hashes visitor identifiers, and gates UK/EU tracking behind consent. Dashboards for popular pages, trending pages, top actions, and recent journeys live right inside the admin Marketing Studio — no Google Analytics, no cookie soup, no tag manager. Retention windows and an automatic purge keep data lean and compliant. When Privacy Center is installed, consent decisions are mirrored into your central consent ledger automatically.”

Media gaps. Marketplace now references only the buyer-worthy settings and public fixture captures. The overview/dashboard widget PNGs were demoted after visual inspection showed mostly blank Insights admin pages rather than populated analytics widgets. The previous frontend tracker image was demoted and deleted because visual inspection showed a mostly blank Demo page; the replacement fixture is guarded behind CAPELL_INSIGHTS_SCREENSHOT_FIXTURES_ENABLED and is enabled by the screenshot wrapper only for captures. Dark alternates remain available in docs/screenshots/ but are not yet promoted.

Pricing / tier / bundle positioning. Premium tier in the growth bundle is now more defensible because first-visit recording and the packaged consent banner are in place. Insights is the natural data backbone of the bundle: position seo-suite and campaign-studio as dependents that record conversions/funnels through an Insights server-side contract (cross-sell), and lean on the existing supports: capell-app/privacy-center mirror as a compliance upsell. Bundle messaging: “Insights measures it, Campaign Studio acts on it, SEO Suite ranks for it.”

Differentiators / value props / target buyer. Differentiators: privacy-first (no third-party scripts, hashed identifiers, consent-gated, Privacy Center mirror), fully in-admin, multi-site/locale aware. Target buyer: privacy-conscious agencies and site owners on Capell who want GA-style insight without GA’s compliance and cookie overhead.

Keywords/tags (8–12). web-analytics, privacy-friendly-analytics, gdpr, cookie-consent, first-party-tracking, page-views, click-tracking, visitor-journeys, marketing-studio, consent-management, filament, capell.

6. Prioritized Roadmap

Item	Bucket	Effort	Impact	Section ref
Done/Shipped: Fix first-visit visit creation so default traffic is actually recorded	Done	M	High	§2, §4
Done/Shipped: Resolve consent region server-side (wire ConsentRegionResolver)	Done	M	High	§2, §4
Default hash_salt from APP_KEY; enforce non-default in health check	Done	S	High	§2, §4
Reconcile registered-vs-missing migrations and README persistence docs	Done	S	High	§2, §4
Fix manifest: populate settings[], permissions[], and shipped screenshots	Done	S	Med	§4, §5
Done/Shipped: Ship a themable consent banner / frontend component	Done	M	High	§3
Done/Shipped: Chunk the retention purge	Done	S	Med	§2, §4
Done/Shipped: Cache dashboard aggregates (TTL + insights tag)	Done	M	High	§2, §4
Done/Shipped: Expand health check with render-hook and purge-schedule probes	Done	S	Med	§2, §4
Done/Shipped: Add referrer + UTM / channel reports (data already captured)	Done	M	High	§3
Done/Shipped: Add bot + admin/self-traffic filtering	Done	M	Med	§3
Done/Shipped: Per-session journey boundaries (sequence reset)	Done	M	Med	§2
Done/Shipped: Daily rollup table + retention tiers for fast long-range reports	Done	L	High	§3
Done/Shipped: Funnels & conversion reporting + server-side event contract for bundle packages	Done	L	High	§3, §5
Done/Shipped: Add a route-backed public screenshot fixture that renders Insights BodyEnd hooks, then capture/promote tracker and consent-banner PNGs	Done	S	Med	§5
Closed 2026-06-06: confirmed single-event Action family / ImportLegacyPageViews are live integration and migration contracts, not dead code	Done	S	Low	§4
Done/Shipped: Honor DNT / GPC; server-side consent expiry/re-prompt	Done	M	Med	§3
Done/Shipped: Add Insights digest builder and CSV export seam	Done	M	Med	§2, §3
Recapture populated Capell analytics dashboard/widget screenshots before promoting them as buyer-facing media	Future	S	Med	§5

Completion Review

Completed 2026-06-08. Every current manifest-backed roadmap row is closed, including the retained single-event Action family / legacy importer, tracker/banner privacy controls, dashboard aggregate/rollup surfaces, server-side conversion contract, admin-path tracker suppression, and the new digest/export seam. Populated admin dashboard recapture remains optional future media polish rather than an active completion blocker.