Update test.php
+283
@@ -1 +1,284 @@
|
||||
<-- [[Home]]
|
||||
# BAS Forecast & Dashboard
|
||||
|
||||
Python forecasting scripts plus a browser dashboard for validating BACnet trend charts and forecast overlays. Runs **offline first** on sample JS files, then optionally against **live `getData`** from BasData.
|
||||
|
||||
---
|
||||
|
||||
## On this page
|
||||
|
||||
- [Overview & quick start](#overview--quick-start)
|
||||
- [Pipeline](#pipeline)
|
||||
- [forecast.py](#forecastpy)
|
||||
- [forecast_comp.py](#forecast_comppy)
|
||||
- [forecast_output.json](#forecast_outputjson)
|
||||
- [test.php dashboard](#testphp-dashboard)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
|
||||
---
|
||||
|
||||
## Overview & quick start
|
||||
|
||||
| Component | Description |
|
||||
|-----------|-------------|
|
||||
| **forecast.py** | Train models, backtest on last 20%, write `forecast_output.json` |
|
||||
| **forecast_comp.py** | Compare target-only vs target + extra points (console MAE) |
|
||||
| **forecast_output.json** | Predictions + MAE per point for the dashboard |
|
||||
| **test.php** | ApexCharts lab: sparks, heatmap, line, phases, HVAC, forecast overlay |
|
||||
| **js/test.js** | Chart colors, wordbank, point → chart mapping |
|
||||
| **js/veris-sample.js** | Veris meter sample data |
|
||||
| **js/rtu-sample.js** | RTU sample data |
|
||||
|
||||
### Install & run
|
||||
|
||||
pip install pandas scikit-learn skforecast
|
||||
python forecast.py
|
||||
|
||||
### View in browser
|
||||
|
||||
/test.php?sample=1&forecast=1
|
||||
|
||||
Or open `test.php`, load sample or live data, then click **Load forecast**.
|
||||
|
||||
### Default forecast points (`forecast.py` → `JOBS`)
|
||||
|
||||
| Sample file | Points |
|
||||
|-------------|--------|
|
||||
| `js/veris-sample.js` | `P`, `Ia`, `Ib`, `Ic` |
|
||||
| `js/rtu-sample.js` | `ServRmTmp` |
|
||||
|
||||
### test.php URL flags
|
||||
|
||||
| Parameter | Example | Effect |
|
||||
|-----------|---------|--------|
|
||||
| `sample=1` | `?sample=1` | Use sample JS; no live fetch |
|
||||
| `forecast=1` | `?forecast=1` | Auto-load `forecast_output.json` |
|
||||
| `interval=N` | `?interval=24` | Live trend window (8–72 hours) |
|
||||
| `keyid=N` | `?keyid=1` | Override automation-server id |
|
||||
| `embed=1` | `?embed=1` | Minimal header (iframe) |
|
||||
|
||||
### Main BasData app (context)
|
||||
|
||||
| Page | Role |
|
||||
|------|------|
|
||||
| **index.php** | Production viewer + `getData` API |
|
||||
| **reports.php** | Report JSON builder |
|
||||
| **test.php** | Chart/forecast lab before shipping to reports/index |
|
||||
|
||||
---
|
||||
|
||||
## Pipeline
|
||||
|
||||
veris-sample.js / rtu-sample.js
|
||||
│
|
||||
▼
|
||||
forecast.py ──► forecast_output.json
|
||||
│
|
||||
▼
|
||||
test.php ◄── (optional) live getData from index.php
|
||||
│
|
||||
▼
|
||||
ApexCharts (dashed forecast on P line + phase compare)
|
||||
|
||||
forecast_comp.py → console MAE only (experiments, no JSON)
|
||||
|
||||
---
|
||||
|
||||
## forecast.py
|
||||
|
||||
Trains one time-series model per BACnet point, evaluates **MAE on the last 20%** of history, predicts **100 steps** ahead, writes **forecast_output.json**.
|
||||
|
||||
### Run
|
||||
|
||||
python forecast.py
|
||||
|
||||
No CLI args — edit constants at the top of the file.
|
||||
|
||||
### Settings
|
||||
|
||||
| Constant | Default | Meaning |
|
||||
|----------|---------|---------|
|
||||
| `FUTURE_STEPS` | `100` | How many future readings to predict |
|
||||
| `LAGS` | `25` | How many past readings the model uses each step |
|
||||
| `JOBS` | veris + rtu | List of `(js_file, [point_names])` |
|
||||
|
||||
Default `JOBS`:
|
||||
|
||||
JOBS = [
|
||||
("js/veris-sample.js", ["P", "Ia", "Ib", "Ic"]),
|
||||
("js/rtu-sample.js", ["ServRmTmp"]),
|
||||
]
|
||||
|
||||
### Algorithm (per point)
|
||||
|
||||
1. **Load** — Parse `var xxxData = { ... };` from the `.js` file.
|
||||
2. **Series** — pandas `Series` indexed by timestamp (`x` = ms, `y` = float).
|
||||
3. **Resample** — Even spacing from median step; interpolate gaps.
|
||||
4. **Backtest** — First **80%** train, last **20%** predict → **MAE** printed.
|
||||
5. **Forecast** — Retrain on **full** series → `FUTURE_STEPS` ahead.
|
||||
6. **Write** — `{ "mae", "future": [{x,y},...] }` per point to JSON.
|
||||
|
||||
### Model stack
|
||||
|
||||
- **skforecast** `ForecasterRecursive`
|
||||
- **sklearn** `Ridge`
|
||||
- Metric: **mean absolute error** on the hidden 20%
|
||||
|
||||
### Console example
|
||||
|
||||
P — 432 readings
|
||||
MAE (last 20%): 11.17
|
||||
|
||||
Saved forecasts for: P, Ia, Ib, Ic, ServRmTmp
|
||||
|
||||
---
|
||||
|
||||
## forecast_comp.py
|
||||
|
||||
Research script: **does adding other points improve predictions for a target?**
|
||||
|
||||
Compares two models on the same 80/20 backtest:
|
||||
|
||||
| Model | Inputs |
|
||||
|-------|--------|
|
||||
| **A — alone** | Target point history only |
|
||||
| **B — combined** | Target + **WITH_POINTS** at each timestamp |
|
||||
|
||||
**Does not write JSON** or change charts.
|
||||
|
||||
### Run
|
||||
|
||||
python forecast_comp.py
|
||||
|
||||
### Settings (top of file)
|
||||
|
||||
| Constant | Example | Meaning |
|
||||
|----------|---------|---------|
|
||||
| `JS_FILE` | `js/rtu-sample.js` | Sample data source |
|
||||
| `TARGET` | `RaTmp` | Point to predict |
|
||||
| `WITH_POINTS` | `["FanCmdOvr"]` | Extra columns for model B |
|
||||
| `LAGS` | `26` | Lag window |
|
||||
|
||||
### Output example
|
||||
|
||||
Backtest on last 20% of sample data:
|
||||
RaTmp only: MAE = 1.24
|
||||
RaTmp + [FanCmdOvr]: MAE = 0.89
|
||||
|
||||
Extra points helped — about 0.35 better on average.
|
||||
|
||||
### Quality warnings
|
||||
|
||||
Script warns when MAE is misleading:
|
||||
|
||||
- Flat target in test slice (MAE ≈ 0, model learned nothing useful)
|
||||
- Exog column identical or nearly identical to target
|
||||
- Exog fully determines target (lookup table)
|
||||
- Flat exog in test slice
|
||||
|
||||
**Use `forecast_comp.py` to experiment; use `forecast.py` to ship JSON to the dashboard.**
|
||||
|
||||
---
|
||||
|
||||
## forecast_output.json
|
||||
|
||||
Generated by `forecast.py`. Loaded by `test.php` for dashed forecast overlays.
|
||||
|
||||
### Schema
|
||||
|
||||
{
|
||||
"P": {
|
||||
"mae": 11.17,
|
||||
"future": [
|
||||
{ "x": 1781713080000, "y": 19.05 },
|
||||
{ "x": 1781713200000, "y": 19.04 }
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
| Field | Meaning |
|
||||
|-------|---------|
|
||||
| `mae` | Backtest error on last 20% (same units as point) |
|
||||
| `future` | Predicted readings after last historical sample |
|
||||
| `x` | Unix time in **milliseconds** (matches sample JS / getData) |
|
||||
| `y` | Predicted value (2 decimals) |
|
||||
|
||||
### Keys (default `JOBS`)
|
||||
|
||||
| Key | Source | Dashboard use |
|
||||
|-----|--------|---------------|
|
||||
| `P` | veris-sample.js | Power line chart |
|
||||
| `Ia`, `Ib`, `Ic` | veris-sample.js | Phase compare |
|
||||
| `ServRmTmp` | rtu-sample.js | In JSON; RTU chart overlay TBD |
|
||||
|
||||
### How test.php uses it
|
||||
|
||||
1. `fetch("forecast_output.json")` → `basForecastData`
|
||||
2. `basForecastSeries(actualData, pointName)` → last real point + `future` array
|
||||
3. ApexCharts: dashed red series, **"Forecast →"** at last live timestamp
|
||||
|
||||
Charts with forecast today: **Line P**, **Phase compare**.
|
||||
|
||||
`mae` is stored but not shown in UI yet.
|
||||
|
||||
---
|
||||
|
||||
## test.php dashboard
|
||||
|
||||
**Device Snapshot** — ApexCharts lab for BasData chart prototyping.
|
||||
|
||||
### Modes
|
||||
|
||||
| Mode | How |
|
||||
|------|-----|
|
||||
| **Live** | Log in on `index.php` → pick office / building / asp → open `test.php` |
|
||||
| **Sample** | `?sample=1` loads `rtu-sample.js` + `veris-sample.js` |
|
||||
| **Forecast** | `?forecast=1` or **Load forecast** (needs chart data loaded first) |
|
||||
|
||||
Hardcoded devices (for now): **RTU_3**, **VerisMeter**.
|
||||
|
||||
### Charts
|
||||
|
||||
| Toolbar slot | Chart | Typical points |
|
||||
|--------------|-------|----------------|
|
||||
| sparks | Fan / room / power mini charts | FanSts, ServRmTmp, P |
|
||||
| heatmap | All Veris points | all keys in bundle |
|
||||
| lineP | Power trend (+ forecast) | P, Demand |
|
||||
| phaseCompare | Ia / Ib / Ic (+ forecast) | Ia, Ib, Ic |
|
||||
| hvacMode | HVAC mode timeline | HVACMode |
|
||||
| trigger | Fan start counts per bucket | FanSts |
|
||||
| radialBar | Temp vs setpoint | ServRmTmp + setpoint pair |
|
||||
|
||||
Toggle charts with toolbar checkboxes. Point → slot mapping: `js/test.js` → `BAS_CHART_SLOTS`, `basResolveDashboard()`.
|
||||
|
||||
### Live data flow
|
||||
|
||||
test.php --POST--> index.php (getData=true&keyid=&name=&interval=&tstamp=)
|
||||
<--JSON-- { PointName: [{x,y}, ...], ... }
|
||||
--> basApplyDashboard(rtu, veris)
|
||||
|
||||
Same contract as the index sidebar.
|
||||
|
||||
### Header buttons
|
||||
|
||||
| Button | Action |
|
||||
|--------|--------|
|
||||
| **Load sample data** | Offline JS bundles |
|
||||
| **Load forecast** | Fetch JSON, redraw P + phase charts |
|
||||
| Status bar | Loading / ok / partial fail / errors |
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Likely cause | Fix |
|
||||
|---------|--------------|-----|
|
||||
| "No forecast_output.json" | File missing | Run `python forecast.py` from project root |
|
||||
| Forecast button, no lines | No chart data | `?sample=1` or log in and load live data first |
|
||||
| MAE ≈ 0 in comp script | Flat or duplicate data | Read warnings in console output |
|
||||
| Live RTU charts empty | Long `interval`, huge payload | Shorter window; see server memory notes |
|
||||
| Point missing from JSON | Not in `JOBS` | Add to `forecast.py` and re-run |
|
||||
|
||||
---
|
||||
|
||||
*BasData forecast & dashboard wiki — single-page reference.*
|
||||
Reference in New Issue
Block a user