# Insights — Improvement & Growth Plan > Package: capell-app/insights · Kind: extension (admin + frontend) · Tier: premium · Product group: Capell Growth · Bundle: growth · Status: Complete ## 1. Snapshot Insights is a first-party, consent-aware web-analytics extension for Capell. The frontend surface ships a vanilla JS tracker and overrideable consent banner injected via a `RenderHookLocation::BodyEnd` hook (`src/Support/RenderHooks/RegisterInsightsTrackerHook.php`) plus two public POST endpoints (`src/Http/Controllers/InsightsBeaconController.php`, `InsightsConsentController.php`) backed by Actions (`RecordInsightsEventsAction`, `ValidateInsightsBeaconRequestAction`, `UpdateInsightsConsentAction`, `CreateInsightsVisitAction`, `MirrorInsightsConsentToPrivacyCenterAction`). The admin surface adds an `InsightsPage`, seven dashboard widgets, three overview stats, daily aggregate rollups, and a settings schema, fed by `Build*QueryAction` aggregates over four tables (`insights_visits`, `insights_consents`, `insights_events`, `insights_daily_rollups`). Deps: `lorisleiva/laravel-actions`, `spatie/laravel-data`, `spatie/laravel-package-tools`; requires `capell-app/admin|core|frontend`, supports `capell-app/privacy-center`. Current marketplace summary (verbatim): _"Cookie-light, GDPR-aware web analytics built into your Capell admin — page views, clicks, visitor journeys, and consent, with no third-party scripts and no data leaving your server."_ The manifest now lists the extension card, the settings screen, and two styled route-backed public fixture screenshots proving the frontend tracker and packaged consent banner render through the Insights BodyEnd hook. The blank dashboard/widget captures remain committed runner evidence but are no longer promoted as buyer-facing media until analytics demo data produces populated Capell widgets. ## 2. Improvements (existing functionality) - **Done/Shipped: JS tracker bootstraps a visit on the first recordable hit.** The beacon action now creates a visit when no `visit_id` is supplied, the server-resolved region is outside UK/Europe, consent is not required for all regions, and the batch contains at least one non-ignored event. The events endpoint returns the fresh `visit_id`, the tracker forces fetch instead of `sendBeacon` for that first flush, and then stores the id for later beacon batches. Focused coverage documents outside-region creation, consent-required suppression, and the first-flush JS response path. — `src/Actions/RecordInsightsEventsAction.php`, `src/Http/Controllers/InsightsBeaconController.php`, `resources/js/capell-insights.js` — M - **Closed: derive `hash_salt` from `APP_KEY` by default** — `ResolveInsightsHashSaltAction` now treats empty, whitespace-only, and legacy public `capell-insights` salts as placeholders, derives the visitor hash salt from a usable `APP_KEY`, and falls back to the public legacy salt only when no usable app key exists. `InsightsHealthCheck` fails that unsafe fallback state without exposing configured secrets in diagnostics. — privacy hardening — `src/Actions/ResolveInsightsHashSaltAction.php`, `src/Actions/CreateInsightsVisitAction.php`, `src/Actions/UpdateInsightsConsentAction.php`, `src/Health/InsightsHealthCheck.php` — S - **Done/Shipped: `ConsentRegionResolver` is wired server-side.** `InsightsConsentController` ignores the client-supplied `region` for trust decisions and passes `ResolveConsentRegionAction::run()` into `UpdateInsightsConsentAction`; the beacon endpoint now uses the same server-side region resolution before creating first visits. — fixes attacker-controllable GDPR jurisdiction + removes dead code — `src/Http/Controllers/InsightsConsentController.php`, `src/Http/Controllers/InsightsBeaconController.php`, `src/Actions/CreateInsightsVisitAction.php`, `src/Support/Consent/ConsentRegionResolver.php` — M - **Done/Shipped: chunked retention purge.** `PurgeInsightsDataAction::handle` now deletes eligible events, consents, and visits through bounded ID batches instead of unbounded table deletes. Batch size defaults to `purge_batch_size` and can be overridden when the Action is called, keeping the purge command behavior stable while reducing lock/replication risk on high-write installs. — `src/Actions/PurgeInsightsDataAction.php`, `config/capell-insights.php` — S - **Done/Shipped: cache dashboard aggregates.** Dashboard aggregate Actions now remember materialized results behind stable keys that include locale/window/site/language/limit inputs, using `dashboard_cache_ttl_seconds` and the `insights` cache tag when the current store supports tags. Event recording, consent updates, and retention purge writes flush the tagged dashboard cache on tag-capable stores; non-tag stores still get bounded TTL protection. — `src/Actions/RememberInsightsDashboardAggregateAction.php`, `src/Actions/Build*QueryAction.php`, `src/Actions/RecordInsightsEventsAction.php`, `src/Actions/PurgeInsightsDataAction.php` — M - **Done/Shipped: Reconcile registered vs on-disk migrations** — `InsightsServiceProvider::configurePackage` now registers only the four migration files present in `database/migrations/`, and `InsightsMigrationsTest` asserts every registered migration name has a file on disk. — fresh-install hazard closed — `src/Providers/InsightsServiceProvider.php`, `tests/Feature/Database/InsightsMigrationsTest.php` — S - **Done/Shipped: Flesh out the health check.** `InsightsHealthCheck` now reports storage-table presence, beacon route registration, frontend tracker render-hook output, monthly retention purge scheduling, and non-default visitor hash secret diagnostics, with pass/fail coverage. — Diagnostics value — `src/Health/InsightsHealthCheck.php`, `tests/Feature/Health/InsightsHealthCheckTest.php` — M - **Done/Shipped: session boundaries for journeys.** `RecordInsightsEventsAction` now rotates a persistent visit cookie into a fresh `InsightsVisit` when `last_seen_at` is older than `session_timeout_minutes` (30 minutes by default). The new visit preserves consent region/status and receives the queued cookie, so event sequences restart per session and journey widgets no longer merge returning visitor activity into one historical timeline. — `src/Actions/RecordInsightsEventsAction.php`, `config/capell-insights.php`, `capell.json` — M - **Done/Shipped: daily rollups for long-range reports.** The package now owns an `insights_daily_rollups` table plus `RebuildInsightsDailyRollupsAction` and `insights:rollups:rebuild` command, scheduled daily. Popular and trending page reports use day/path/type aggregate rows for day-aligned windows and fall back to raw events until aggregates exist, keeping fresh installs correct while reducing long-range dashboard scans. — `database/migrations/2026_06_06_000001_create_insights_daily_rollups_table.php`, `src/Actions/RebuildInsightsDailyRollupsAction.php`, `src/Actions/BuildPopularPagesQueryAction.php`, `src/Actions/BuildTrendingPagesQueryAction.php` — L - **Done/Shipped: digest and CSV export seam.** `BuildInsightsDigestAction` assembles overview stats, popular pages, acquisition sources, and funnel conversion data for a window; `ExportInsightsDigestCsvAction` serializes the digest without reintroducing public render work or admin query coupling. — `src/Actions/BuildInsightsDigestAction.php`, `src/Actions/ExportInsightsDigestCsvAction.php` — M ## 3. Missing Features (gaps) Tied to `capabilities[]` = `insights, insights-admin, insights-frontend, insights-consent-banner, insights-acquisition-reports, insights-conversion-funnels, insights-traffic-filtering, insights-session-journeys, insights-daily-rollups, insights-privacy-signals, insights-privacy-center-mirror, beacon, cache-blocking`. - **Done/Shipped: consent banner / opt-in UI.** The BodyEnd hook now renders an overrideable `capell-insights::components.consent-banner` view by default, gated by `consent_banner_enabled`. The tracker shows it when no current-policy consent decision is stored, supports accept/reject/granular choices, posts to the consent endpoint, stores the decision locally with `policy_version`, and hides the banner after a successful response. — `resources/views/components/consent-banner.blade.php`, `resources/js/capell-insights.js`, `config/capell-insights.php` - **Done/Shipped: Referrer & UTM analytics.** `BuildAcquisitionSourcesQueryAction` now surfaces stored `utm_source`, `utm_medium`, `utm_campaign`, and `referrer_url` data in an Acquisition Sources dashboard widget. The report groups visits by campaign, referrer host, or direct traffic; it is cache-keyed by the dashboard window and declared as `insights-acquisition-reports` in the manifest. — `src/Actions/BuildAcquisitionSourcesQueryAction.php`, `src/Filament/Widgets/AcquisitionSourcesWidget.php`, `capell.json` - **Done/Shipped: pre-aggregated rollups / retention tiers (differentiator).** Daily rollups now aggregate path × day × type counts for popular/trending page dashboards, with raw-event fallback until aggregates exist. Deeper separate retention tiers for raw-vs-aggregate data remain future depth. - **Done/Shipped: funnels & conversions (differentiator).** Companion packages can now call `RecordConversionAction` to record typed conversion events with source package/value/currency metadata, and `BuildFunnelConversionReportAction` summarizes named event steps by unique visits and conversion rate. The beacon metadata contract also allows conversion metadata keys for browser/custom integrations. — `src/Actions/RecordConversionAction.php`, `src/Actions/BuildFunnelConversionReportAction.php`, `src/Data/InsightsEventMetadataData.php` - **Done/Shipped: bot / self-traffic filtering.** `RecordInsightsEventsAction` now suppresses ignored IPs and user-agent patterns before creating first visits or appending events to existing visits. The package config ships default bot, crawler, monitor, Lighthouse, PageSpeed, Pingdom, GTmetrix, and headless-browser patterns, with `ignored_ips` available for agency/staff office traffic. — `src/Actions/RecordInsightsEventsAction.php`, `config/capell-insights.php`, `capell.json` - **Visit duration / bounce / entry-exit (table-stakes).** `started_at`/`last_seen_at` exist on the visit but no engaged-time, bounce-rate, or entry/exit-page metrics are computed. - **Done/Shipped: consent expiry & re-prompt.** Browser decisions are stored with `policy_version`, the packaged banner reappears when the configured policy version changes, and the server refuses analytics recording in consent-required regions when the latest consent is for an old policy or older than `consent_expires_days`. - **Done/Shipped: server-side event API surface for other packages.** `RecordConversionAction` gives seo-suite/campaign-studio style packages an explicit in-process analytics contract without exposing a public unauthenticated write API. `BuildFunnelConversionReportAction` provides the matching reporting surface for bundle dashboards. - **Done/Shipped: digest/export reporting.** First-party analytics can now be summarized as a typed digest and exported as CSV, giving GA4 Reports, Campaign Studio, Email Studio, and future scheduled report consumers a consistent report seam. - **Done/Shipped: Do-Not-Track / Global Privacy Control honoring and server-side re-consent (differentiator).** `honor_privacy_signals` now defaults on. The browser tracker exits before registering listeners when `navigator.globalPrivacyControl`, `navigator.doNotTrack`, or `navigator.msDoNotTrack` is active, and the beacon endpoint returns no-content without validation/persistence for `Sec-GPC: 1`, `DNT: 1`, or `X-Do-Not-Track: 1`. Consent-required regions now record analytics only when the latest consent matches the configured `policy_version` and remains inside `consent_expires_days`. — `resources/js/capell-insights.js`, `resources/views/tracker.blade.php`, `src/Actions/ValidateInsightsBeaconRequestAction.php`, `src/Actions/RecordInsightsEventsAction.php` ## 4. Issues / Risks - **Done/Shipped: first-visit data loss is fixed for no-consent-required regions.** See §2 item 1. The no-visit beacon path now creates a visit, records the first non-ignored event, returns a visit id, and refuses to create visits when the server region still requires consent. - **Done/Shipped: GDPR region is resolved server-side.** `InsightsConsentController` and the first-visit beacon path both use `ResolveConsentRegionAction`; client-supplied region remains accepted as request shape only, not as the trust source for consent enforcement. - **Reversible visitor hashes (PII).** Closed for default installs: empty or legacy `hash_salt` values now derive from `APP_KEY`, and Diagnostics fails if the package would fall back to the public legacy salt. Remaining risk is limited to installs running without a usable `APP_KEY` and without a private `hash_salt`. - **Cache-safety of the beacon: OK.** The render hook output is static config + a file-read script cached `rememberForever` keyed by file mtime (`src/Actions/GetInsightsTrackerScriptAction.php`); the blade does no DB query and leaks no model IDs/labels/permissions (`resources/views/tracker.blade.php`) — consistent with `cacheSafety.cacheable:false, sensitiveOutput:false`. The render hook now suppresses tracker output on configured `ignored_paths` such as `/admin*`, matching the server-side beacon ignore policy and avoiding admin/Livewire script emission. — `src/Support/RenderHooks/RegisterInsightsTrackerHook.php`, `tests/Feature/Frontend/InsightsRenderHookTest.php` - **Closed: migration drift.** `InsightsMigrationsTest` now asserts all registered migration names exist on disk, and the provider no longer references phantom `06`/site-FK migrations. - **Closed: core manifest mismatches.** `capell.json` now declares `Capell\Insights\Settings\InsightsSettings`, the generated Shield page permission `View:InsightsPage`, and the shipped marketplace screenshot set. `ManifestRequirementsTest` asserts those declarations and the screenshot files. `commands.doctor` remains `null` because no package command ships; Diagnostics consumes the health-check class directly. - **Closed: single-event and legacy import Actions retained intentionally.** The single-event family is not on the browser beacon request path, but it remains a supported server-side integration path: `RecordConversionAction` calls `RecordCustomActionAction`, `docs/tracking-and-consent.md` documents custom server-side events, and `ImportLegacyPageViewsAction` is the migration-owned idempotent importer for legacy `page_views`. Removing these would regress package integrations and fresh-upgrade migration behavior. - **Performance budget realism.** `frontendRenderBudgetMs:20` is plausible (cached script), short-TTL dashboard aggregate caching reduces repeated widget scans, and daily rollups now replace raw page-event scans for day-aligned popular/trending dashboards. High write volume on `insights_events` remains the principal scaling risk for live and custom-event reports. - **i18n.** Strings are translated via `capell-insights::` keys (good), but the config key `track_form-builder` contains a hyphen and is mirrored as a setting/label key — brittle and inconsistent with snake_case siblings (`config/capell-insights.php`, `src/Filament/Settings/InsightsSettingsSchema.php:34`). - **Test gaps.** First-visit auto-creation, consent-required suppression, server-side region resolution, manifest declarations, render-hook banner output, admin-path render-hook suppression, health-check render/schedule probes, JS banner primitives, chunked retention purge behavior, dashboard cache helper behavior, and acquisition-source grouping have focused coverage. Remaining gaps: browser-level consent-banner interaction coverage and deeper long-window rollup coverage. ## 5. Marketplace & Selling **Critique.** The manifest `summary` and package description now lead with the buyer benefit (first-party, GDPR-aware analytics without third-party scripts). The gallery references the extension card, settings screen, and two styled package-owned public fixture captures for the active tracker and consent-banner flow. Remaining media upside is optional: recapture populated dashboard/widgets before promoting admin analytics screenshots, and promote dark admin alternates only if Marketplace wants explicit dark-mode gallery entries. **Improved 1-sentence summary.** "Cookie-light, GDPR-aware web analytics built into your Capell admin — page views, clicks, visitor journeys, and consent, with no third-party scripts and no data leaving your server." **Improved 3–4 sentence description.** "Insights gives Capell sites first-party traffic analytics that respect privacy by default: it records page views, clicks, and visitor journeys server-side, hashes visitor identifiers, and gates UK/EU tracking behind consent. Dashboards for popular pages, trending pages, top actions, and recent journeys live right inside the admin Marketing Studio — no Google Analytics, no cookie soup, no tag manager. Retention windows and an automatic purge keep data lean and compliant. When Privacy Center is installed, consent decisions are mirrored into your central consent ledger automatically." **Media gaps.** Marketplace now references only the buyer-worthy settings and public fixture captures. The overview/dashboard widget PNGs were demoted after visual inspection showed mostly blank Insights admin pages rather than populated analytics widgets. The previous frontend tracker image was demoted and deleted because visual inspection showed a mostly blank Demo page; the replacement fixture is guarded behind `CAPELL_INSIGHTS_SCREENSHOT_FIXTURES_ENABLED` and is enabled by the screenshot wrapper only for captures. Dark alternates remain available in `docs/screenshots/` but are not yet promoted. **Pricing / tier / bundle positioning.** Premium tier in the `growth` bundle is now more defensible because first-visit recording and the packaged consent banner are in place. Insights is the natural data backbone of the bundle: position seo-suite and campaign-studio as **dependents** that record conversions/funnels through an Insights server-side contract (cross-sell), and lean on the existing `supports: capell-app/privacy-center` mirror as a compliance upsell. Bundle messaging: "Insights measures it, Campaign Studio acts on it, SEO Suite ranks for it." **Differentiators / value props / target buyer.** Differentiators: privacy-first (no third-party scripts, hashed identifiers, consent-gated, Privacy Center mirror), fully in-admin, multi-site/locale aware. Target buyer: privacy-conscious agencies and site owners on Capell who want GA-style insight without GA's compliance and cookie overhead. **Keywords/tags (8–12).** `web-analytics`, `privacy-friendly-analytics`, `gdpr`, `cookie-consent`, `first-party-tracking`, `page-views`, `click-tracking`, `visitor-journeys`, `marketing-studio`, `consent-management`, `filament`, `capell`. ## 6. Prioritized Roadmap | Item | Bucket | Effort | Impact | Section ref | | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | ------ | ------ | ----------- | | Done/Shipped: Fix first-visit visit creation so default traffic is actually recorded | Done | M | High | §2, §4 | | Done/Shipped: Resolve consent region server-side (wire `ConsentRegionResolver`) | Done | M | High | §2, §4 | | ~~Default `hash_salt` from `APP_KEY`; enforce non-default in health check~~ | Done | S | High | §2, §4 | | Reconcile registered-vs-missing migrations and README persistence docs | Done | S | High | §2, §4 | | Fix manifest: populate `settings[]`, `permissions[]`, and shipped screenshots | Done | S | Med | §4, §5 | | Done/Shipped: Ship a themable consent banner / frontend component | Done | M | High | §3 | | Done/Shipped: Chunk the retention purge | Done | S | Med | §2, §4 | | Done/Shipped: Cache dashboard aggregates (TTL + `insights` tag) | Done | M | High | §2, §4 | | Done/Shipped: Expand health check with render-hook and purge-schedule probes | Done | S | Med | §2, §4 | | Done/Shipped: Add referrer + UTM / channel reports (data already captured) | Done | M | High | §3 | | Done/Shipped: Add bot + admin/self-traffic filtering | Done | M | Med | §3 | | Done/Shipped: Per-session journey boundaries (sequence reset) | Done | M | Med | §2 | | Done/Shipped: Daily rollup table + retention tiers for fast long-range reports | Done | L | High | §3 | | Done/Shipped: Funnels & conversion reporting + server-side event contract for bundle packages | Done | L | High | §3, §5 | | Done/Shipped: Add a route-backed public screenshot fixture that renders Insights BodyEnd hooks, then capture/promote tracker and consent-banner PNGs | Done | S | Med | §5 | | Closed 2026-06-06: confirmed single-event Action family / `ImportLegacyPageViews` are live integration and migration contracts, not dead code | Done | S | Low | §4 | | Done/Shipped: Honor DNT / GPC; server-side consent expiry/re-prompt | Done | M | Med | §3 | | Done/Shipped: Add Insights digest builder and CSV export seam | Done | M | Med | §2, §3 | | Recapture populated Capell analytics dashboard/widget screenshots before promoting them as buyer-facing media | Future | S | Med | §5 | ## Completion Review Completed 2026-06-08. Every current manifest-backed roadmap row is closed, including the retained single-event Action family / legacy importer, tracker/banner privacy controls, dashboard aggregate/rollup surfaces, server-side conversion contract, admin-path tracker suppression, and the new digest/export seam. Populated admin dashboard recapture remains optional future media polish rather than an active completion blocker.