Update test.php

test.php.md

@@ -1 +1,284 @@
-<-- [[Home]]
+<-- [[Home]]
+# BAS Forecast & Dashboard
+
+Python forecasting scripts plus a browser dashboard for validating BACnet trend charts and forecast overlays. Runs **offline first** on sample JS files, then optionally against **live `getData`** from BasData.
+
+---
+
+## On this page
+
+- [Overview & quick start](#overview--quick-start)
+- [Pipeline](#pipeline)
+- [forecast.py](#forecastpy)
+- [forecast_comp.py](#forecast_comppy)
+- [forecast_output.json](#forecast_outputjson)
+- [test.php dashboard](#testphp-dashboard)
+- [Troubleshooting](#troubleshooting)
+
+---
+
+## Overview & quick start
+
+| Component | Description |
+|-----------|-------------|
+| **forecast.py** | Train models, backtest on last 20%, write `forecast_output.json` |
+| **forecast_comp.py** | Compare target-only vs target + extra points (console MAE) |
+| **forecast_output.json** | Predictions + MAE per point for the dashboard |
+| **test.php** | ApexCharts lab: sparks, heatmap, line, phases, HVAC, forecast overlay |
+| **js/test.js** | Chart colors, wordbank, point → chart mapping |
+| **js/veris-sample.js** | Veris meter sample data |
+| **js/rtu-sample.js** | RTU sample data |
+
+### Install & run
+
+    pip install pandas scikit-learn skforecast
+    python forecast.py
+
+### View in browser
+
+    /test.php?sample=1&forecast=1
+
+Or open `test.php`, load sample or live data, then click **Load forecast**.
+
+### Default forecast points (`forecast.py` → `JOBS`)
+
+| Sample file | Points |
+|-------------|--------|
+| `js/veris-sample.js` | `P`, `Ia`, `Ib`, `Ic` |
+| `js/rtu-sample.js` | `ServRmTmp` |
+
+### test.php URL flags
+
+| Parameter | Example | Effect |
+|-----------|---------|--------|
+| `sample=1` | `?sample=1` | Use sample JS; no live fetch |
+| `forecast=1` | `?forecast=1` | Auto-load `forecast_output.json` |
+| `interval=N` | `?interval=24` | Live trend window (8–72 hours) |
+| `keyid=N` | `?keyid=1` | Override automation-server id |
+| `embed=1` | `?embed=1` | Minimal header (iframe) |
+
+### Main BasData app (context)
+
+| Page | Role |
+|------|------|
+| **index.php** | Production viewer + `getData` API |
+| **reports.php** | Report JSON builder |
+| **test.php** | Chart/forecast lab before shipping to reports/index |
+
+---
+
+## Pipeline
+
+    veris-sample.js / rtu-sample.js
+            │
+            ▼
+       forecast.py  ──►  forecast_output.json
+            │
+            ▼
+        test.php  ◄──  (optional) live getData from index.php
+            │
+            ▼
+       ApexCharts (dashed forecast on P line + phase compare)
+
+    forecast_comp.py  →  console MAE only (experiments, no JSON)
+
+---
+
+## forecast.py
+
+Trains one time-series model per BACnet point, evaluates **MAE on the last 20%** of history, predicts **100 steps** ahead, writes **forecast_output.json**.
+
+### Run
+
+    python forecast.py
+
+No CLI args — edit constants at the top of the file.
+
+### Settings
+
+| Constant | Default | Meaning |
+|----------|---------|---------|
+| `FUTURE_STEPS` | `100` | How many future readings to predict |
+| `LAGS` | `25` | How many past readings the model uses each step |
+| `JOBS` | veris + rtu | List of `(js_file, [point_names])` |
+
+Default `JOBS`:
+
+    JOBS = [
+        ("js/veris-sample.js", ["P", "Ia", "Ib", "Ic"]),
+        ("js/rtu-sample.js", ["ServRmTmp"]),
+    ]
+
+### Algorithm (per point)
+
+1. **Load** — Parse `var xxxData = { ... };` from the `.js` file.
+2. **Series** — pandas `Series` indexed by timestamp (`x` = ms, `y` = float).
+3. **Resample** — Even spacing from median step; interpolate gaps.
+4. **Backtest** — First **80%** train, last **20%** predict → **MAE** printed.
+5. **Forecast** — Retrain on **full** series → `FUTURE_STEPS` ahead.
+6. **Write** — `{ "mae", "future": [{x,y},...] }` per point to JSON.
+
+### Model stack
+
+- **skforecast** `ForecasterRecursive`
+- **sklearn** `Ridge`
+- Metric: **mean absolute error** on the hidden 20%
+
+### Console example
+
+    P — 432 readings
+      MAE (last 20%): 11.17
+
+    Saved forecasts for: P, Ia, Ib, Ic, ServRmTmp
+
+---
+
+## forecast_comp.py
+
+Research script: **does adding other points improve predictions for a target?**
+
+Compares two models on the same 80/20 backtest:
+
+| Model | Inputs |
+|-------|--------|
+| **A — alone** | Target point history only |
+| **B — combined** | Target + **WITH_POINTS** at each timestamp |
+
+**Does not write JSON** or change charts.
+
+### Run
+
+    python forecast_comp.py
+
+### Settings (top of file)
+
+| Constant | Example | Meaning |
+|----------|---------|---------|
+| `JS_FILE` | `js/rtu-sample.js` | Sample data source |
+| `TARGET` | `RaTmp` | Point to predict |
+| `WITH_POINTS` | `["FanCmdOvr"]` | Extra columns for model B |
+| `LAGS` | `26` | Lag window |
+
+### Output example
+
+    Backtest on last 20% of sample data:
+      RaTmp only:                    MAE = 1.24
+      RaTmp + [FanCmdOvr]:           MAE = 0.89
+
+    Extra points helped — about 0.35 better on average.
+
+### Quality warnings
+
+Script warns when MAE is misleading:
+
+- Flat target in test slice (MAE ≈ 0, model learned nothing useful)
+- Exog column identical or nearly identical to target
+- Exog fully determines target (lookup table)
+- Flat exog in test slice
+
+**Use `forecast_comp.py` to experiment; use `forecast.py` to ship JSON to the dashboard.**
+
+---
+
+## forecast_output.json
+
+Generated by `forecast.py`. Loaded by `test.php` for dashed forecast overlays.
+
+### Schema
+
+    {
+      "P": {
+        "mae": 11.17,
+        "future": [
+          { "x": 1781713080000, "y": 19.05 },
+          { "x": 1781713200000, "y": 19.04 }
+        ]
+      }
+    }
+
+| Field | Meaning |
+|-------|---------|
+| `mae` | Backtest error on last 20% (same units as point) |
+| `future` | Predicted readings after last historical sample |
+| `x` | Unix time in **milliseconds** (matches sample JS / getData) |
+| `y` | Predicted value (2 decimals) |
+
+### Keys (default `JOBS`)
+
+| Key | Source | Dashboard use |
+|-----|--------|---------------|
+| `P` | veris-sample.js | Power line chart |
+| `Ia`, `Ib`, `Ic` | veris-sample.js | Phase compare |
+| `ServRmTmp` | rtu-sample.js | In JSON; RTU chart overlay TBD |
+
+### How test.php uses it
+
+1. `fetch("forecast_output.json")` → `basForecastData`
+2. `basForecastSeries(actualData, pointName)` → last real point + `future` array
+3. ApexCharts: dashed red series, **"Forecast →"** at last live timestamp
+
+Charts with forecast today: **Line P**, **Phase compare**.
+
+`mae` is stored but not shown in UI yet.
+
+---
+
+## test.php dashboard
+
+**Device Snapshot** — ApexCharts lab for BasData chart prototyping.
+
+### Modes
+
+| Mode | How |
+|------|-----|
+| **Live** | Log in on `index.php` → pick office / building / asp → open `test.php` |
+| **Sample** | `?sample=1` loads `rtu-sample.js` + `veris-sample.js` |
+| **Forecast** | `?forecast=1` or **Load forecast** (needs chart data loaded first) |
+
+Hardcoded devices (for now): **RTU_3**, **VerisMeter**.
+
+### Charts
+
+| Toolbar slot | Chart | Typical points |
+|--------------|-------|----------------|
+| sparks | Fan / room / power mini charts | FanSts, ServRmTmp, P |
+| heatmap | All Veris points | all keys in bundle |
+| lineP | Power trend (+ forecast) | P, Demand |
+| phaseCompare | Ia / Ib / Ic (+ forecast) | Ia, Ib, Ic |
+| hvacMode | HVAC mode timeline | HVACMode |
+| trigger | Fan start counts per bucket | FanSts |
+| radialBar | Temp vs setpoint | ServRmTmp + setpoint pair |
+
+Toggle charts with toolbar checkboxes. Point → slot mapping: `js/test.js` → `BAS_CHART_SLOTS`, `basResolveDashboard()`.
+
+### Live data flow
+
+    test.php  --POST-->  index.php (getData=true&keyid=&name=&interval=&tstamp=)
+             <--JSON--  { PointName: [{x,y}, ...], ... }
+             -->  basApplyDashboard(rtu, veris)
+
+Same contract as the index sidebar.
+
+### Header buttons
+
+| Button | Action |
+|--------|--------|
+| **Load sample data** | Offline JS bundles |
+| **Load forecast** | Fetch JSON, redraw P + phase charts |
+| Status bar | Loading / ok / partial fail / errors |
+
+---
+
+## Troubleshooting
+
+| Problem | Likely cause | Fix |
+|---------|--------------|-----|
+| "No forecast_output.json" | File missing | Run `python forecast.py` from project root |
+| Forecast button, no lines | No chart data | `?sample=1` or log in and load live data first |
+| MAE ≈ 0 in comp script | Flat or duplicate data | Read warnings in console output |
+| Live RTU charts empty | Long `interval`, huge payload | Shorter window; see server memory notes |
+| Point missing from JSON | Not in `JOBS` | Add to `forecast.py` and re-run |
+
+---
+
+*BasData forecast & dashboard wiki — single-page reference.*