Update Home

2026-06-18 20:33:53 +00:00
parent a03bb47e6f
commit 3a04a023bd
1 changed files with 282 additions and 9 deletions
@@ -1,10 +1,283 @@
-# Forecast 
-</br>
-[[forecast.py]]  - Takes a data set and uses 80% of the data to train itself and the last 20% of the data to comapre the accuracy of its results. It also shows the mean error of the data generated.
-</br>
-[[forecast_comp.py]] - Takes a target data set and addes another set to the LLM to give it more data and prodouce accurate results. Gives the Mean error of the single set and the multi set to see if there was improvment. 
-</br>
-[[forecast_output.json]] - output for python script data.
-</br>
+# BAS Forecast & Dashboard

-[[test.php]] - testing charts and methods 
+Python forecasting scripts plus a browser dashboard for validating BACnet trend charts and forecast overlays. Runs **offline first** on sample JS files, then optionally against **live `getData`** from BasData.
+
+---
+
+## On this page
+
+- [Overview & quick start](#overview--quick-start)
+- [Pipeline](#pipeline)
+- [forecast.py](#forecastpy)
+- [forecast_comp.py](#forecast_comppy)
+- [forecast_output.json](#forecast_outputjson)
+- [test.php dashboard](#testphp-dashboard)
+- [Troubleshooting](#troubleshooting)
+
+---
+
+## Overview & quick start
+
+| Component | Description |
+|-----------|-------------|
+| **forecast.py** | Train models, backtest on last 20%, write `forecast_output.json` |
+| **forecast_comp.py** | Compare target-only vs target + extra points (console MAE) |
+| **forecast_output.json** | Predictions + MAE per point for the dashboard |
+| **test.php** | ApexCharts lab: sparks, heatmap, line, phases, HVAC, forecast overlay |
+| **js/test.js** | Chart colors, wordbank, point → chart mapping |
+| **js/veris-sample.js** | Veris meter sample data |
+| **js/rtu-sample.js** | RTU sample data |
+
+### Install & run
+
+    pip install pandas scikit-learn skforecast
+    python forecast.py
+
+### View in browser
+
+    /test.php?sample=1&forecast=1
+
+Or open `test.php`, load sample or live data, then click **Load forecast**.
+
+### Default forecast points (`forecast.py` → `JOBS`)
+
+| Sample file | Points |
+|-------------|--------|
+| `js/veris-sample.js` | `P`, `Ia`, `Ib`, `Ic` |
+| `js/rtu-sample.js` | `ServRmTmp` |
+
+### test.php URL flags
+
+| Parameter | Example | Effect |
+|-----------|---------|--------|
+| `sample=1` | `?sample=1` | Use sample JS; no live fetch |
+| `forecast=1` | `?forecast=1` | Auto-load `forecast_output.json` |
+| `interval=N` | `?interval=24` | Live trend window (8–72 hours) |
+| `keyid=N` | `?keyid=1` | Override automation-server id |
+| `embed=1` | `?embed=1` | Minimal header (iframe) |
+
+### Main BasData app (context)
+
+| Page | Role |
+|------|------|
+| **index.php** | Production viewer + `getData` API |
+| **reports.php** | Report JSON builder |
+| **test.php** | Chart/forecast lab before shipping to reports/index |
+
+---
+
+## Pipeline
+
+    veris-sample.js / rtu-sample.js
+            │
+            ▼
+       forecast.py  ──►  forecast_output.json
+            │
+            ▼
+        test.php  ◄──  (optional) live getData from index.php
+            │
+            ▼
+       ApexCharts (dashed forecast on P line + phase compare)
+
+    forecast_comp.py  →  console MAE only (experiments, no JSON)
+
+---
+
+## forecast.py
+
+Trains one time-series model per BACnet point, evaluates **MAE on the last 20%** of history, predicts **100 steps** ahead, writes **forecast_output.json**.
+
+### Run
+
+    python forecast.py
+
+No CLI args — edit constants at the top of the file.
+
+### Settings
+
+| Constant | Default | Meaning |
+|----------|---------|---------|
+| `FUTURE_STEPS` | `100` | How many future readings to predict |
+| `LAGS` | `25` | How many past readings the model uses each step |
+| `JOBS` | veris + rtu | List of `(js_file, [point_names])` |
+
+Default `JOBS`:
+
+    JOBS = [
+        ("js/veris-sample.js", ["P", "Ia", "Ib", "Ic"]),
+        ("js/rtu-sample.js", ["ServRmTmp"]),
+    ]
+
+### Algorithm (per point)
+
+1. **Load** — Parse `var xxxData = { ... };` from the `.js` file.
+2. **Series** — pandas `Series` indexed by timestamp (`x` = ms, `y` = float).
+3. **Resample** — Even spacing from median step; interpolate gaps.
+4. **Backtest** — First **80%** train, last **20%** predict → **MAE** printed.
+5. **Forecast** — Retrain on **full** series → `FUTURE_STEPS` ahead.
+6. **Write** — `{ "mae", "future": [{x,y},...] }` per point to JSON.
+
+### Model stack
+
+- **skforecast** `ForecasterRecursive`
+- **sklearn** `Ridge`
+- Metric: **mean absolute error** on the hidden 20%
+
+### Console example
+
+    P — 432 readings
+      MAE (last 20%): 11.17
+
+    Saved forecasts for: P, Ia, Ib, Ic, ServRmTmp
+
+---
+
+## forecast_comp.py
+
+Research script: **does adding other points improve predictions for a target?**
+
+Compares two models on the same 80/20 backtest:
+
+| Model | Inputs |
+|-------|--------|
+| **A — alone** | Target point history only |
+| **B — combined** | Target + **WITH_POINTS** at each timestamp |
+
+**Does not write JSON** or change charts.
+
+### Run
+
+    python forecast_comp.py
+
+### Settings (top of file)
+
+| Constant | Example | Meaning |
+|----------|---------|---------|
+| `JS_FILE` | `js/rtu-sample.js` | Sample data source |
+| `TARGET` | `RaTmp` | Point to predict |
+| `WITH_POINTS` | `["FanCmdOvr"]` | Extra columns for model B |
+| `LAGS` | `26` | Lag window |
+
+### Output example
+
+    Backtest on last 20% of sample data:
+      RaTmp only:                    MAE = 1.24
+      RaTmp + [FanCmdOvr]:           MAE = 0.89
+
+    Extra points helped — about 0.35 better on average.
+
+### Quality warnings
+
+Script warns when MAE is misleading:
+
+- Flat target in test slice (MAE ≈ 0, model learned nothing useful)
+- Exog column identical or nearly identical to target
+- Exog fully determines target (lookup table)
+- Flat exog in test slice
+
+**Use `forecast_comp.py` to experiment; use `forecast.py` to ship JSON to the dashboard.**
+
+---
+
+## forecast_output.json
+
+Generated by `forecast.py`. Loaded by `test.php` for dashed forecast overlays.
+
+### Schema
+
+    {
+      "P": {
+        "mae": 11.17,
+        "future": [
+          { "x": 1781713080000, "y": 19.05 },
+          { "x": 1781713200000, "y": 19.04 }
+        ]
+      }
+    }
+
+| Field | Meaning |
+|-------|---------|
+| `mae` | Backtest error on last 20% (same units as point) |
+| `future` | Predicted readings after last historical sample |
+| `x` | Unix time in **milliseconds** (matches sample JS / getData) |
+| `y` | Predicted value (2 decimals) |
+
+### Keys (default `JOBS`)
+
+| Key | Source | Dashboard use |
+|-----|--------|---------------|
+| `P` | veris-sample.js | Power line chart |
+| `Ia`, `Ib`, `Ic` | veris-sample.js | Phase compare |
+| `ServRmTmp` | rtu-sample.js | In JSON; RTU chart overlay TBD |
+
+### How test.php uses it
+
+1. `fetch("forecast_output.json")` → `basForecastData`
+2. `basForecastSeries(actualData, pointName)` → last real point + `future` array
+3. ApexCharts: dashed red series, **"Forecast →"** at last live timestamp
+
+Charts with forecast today: **Line P**, **Phase compare**.
+
+`mae` is stored but not shown in UI yet.
+
+---
+
+## test.php dashboard
+
+**Device Snapshot** — ApexCharts lab for BasData chart prototyping.
+
+### Modes
+
+| Mode | How |
+|------|-----|
+| **Live** | Log in on `index.php` → pick office / building / asp → open `test.php` |
+| **Sample** | `?sample=1` loads `rtu-sample.js` + `veris-sample.js` |
+| **Forecast** | `?forecast=1` or **Load forecast** (needs chart data loaded first) |
+
+Hardcoded devices (for now): **RTU_3**, **VerisMeter**.
+
+### Charts
+
+| Toolbar slot | Chart | Typical points |
+|--------------|-------|----------------|
+| sparks | Fan / room / power mini charts | FanSts, ServRmTmp, P |
+| heatmap | All Veris points | all keys in bundle |
+| lineP | Power trend (+ forecast) | P, Demand |
+| phaseCompare | Ia / Ib / Ic (+ forecast) | Ia, Ib, Ic |
+| hvacMode | HVAC mode timeline | HVACMode |
+| trigger | Fan start counts per bucket | FanSts |
+| radialBar | Temp vs setpoint | ServRmTmp + setpoint pair |
+
+Toggle charts with toolbar checkboxes. Point → slot mapping: `js/test.js` → `BAS_CHART_SLOTS`, `basResolveDashboard()`.
+
+### Live data flow
+
+    test.php  --POST-->  index.php (getData=true&keyid=&name=&interval=&tstamp=)
+             <--JSON--  { PointName: [{x,y}, ...], ... }
+             -->  basApplyDashboard(rtu, veris)
+
+Same contract as the index sidebar.
+
+### Header buttons
+
+| Button | Action |
+|--------|--------|
+| **Load sample data** | Offline JS bundles |
+| **Load forecast** | Fetch JSON, redraw P + phase charts |
+| Status bar | Loading / ok / partial fail / errors |
+
+---
+
+## Troubleshooting
+
+| Problem | Likely cause | Fix |
+|---------|--------------|-----|
+| "No forecast_output.json" | File missing | Run `python forecast.py` from project root |
+| Forecast button, no lines | No chart data | `?sample=1` or log in and load live data first |
+| MAE ≈ 0 in comp script | Flat or duplicate data | Read warnings in console output |
+| Live RTU charts empty | Long `interval`, huge payload | Shorter window; see server memory notes |
+| Point missing from JSON | Not in `JOBS` | Add to `forecast.py` and re-run |
+
+---
+
+*BasData forecast & dashboard wiki — single-page reference.*