Data Sources
Where the app's market data comes from, what gets cached locally, and how stale a cache is allowed to get before the app falls back to something else.
For backup/restore and disk cleanup of these caches, see Data Management.
Broker APIs (Kite / Breeze / Dhan)
Your connected broker (see Setup) is the source of truth for:
- Live LTP and option-chain quotes — fetched per request, never cached to disk.
- Historical candles — fetched per request for backtests/simulation
seeding. Held in an in-memory cache for the life of the process
(
broker/sim_ltp.py,broker/day_candle_cache.py), not written to disk. - NFO instrument list — the tradable option/future contracts and their
tokens. Fetched once per day on first live use and saved as a dated
snapshot (
strategies/instruments_cache/NFO_<date>.json). Simulations use the newest snapshot on or before the simulated date, so expired contracts stay resolvable. This is the "Cached NFO instruments" line on the Data Management page.
Breeze can't quote index/VIX/currency/commodity instruments itself, and Dhan has similar gaps — that's why the two sources below exist independently of whichever broker is connected.
NSE — trading holiday calendar
holidays.py fetches the NSE trading-holiday list
(nseindia.com/api/holiday-master) the first time a given year is needed,
and caches it to .nse_holidays.json (repo root) indefinitely — trading
holidays for a past year never change. If the fetch fails (NSE blocks a lot
of non-browser traffic), the app falls back to a small bundled list for the
current year that's updated once per release.
Used to gate scheduled jobs and strategy entries to real trading sessions
(is_trading_day()).
Yahoo Finance — market regime classifier
broker/regime.py classifies the current market regime (trend × volatility,
e.g. "Bull Quiet", "Bear Turbulent") from four tickers — NIFTY, India VIX,
USDINR, Crude — pulled from Yahoo Finance's public chart endpoint
(query1.finance.yahoo.com/v8/finance/chart/...) rather than the connected
broker. This keeps the regime badge (top navbar) working identically
regardless of broker or paper mode, and gives Yahoo's date-windowed history
for any as_of date, which NSE's own FII/DII endpoint can't do (no date
param, always "today").
Fetched live, held in an in-process cache for ~15 minutes
(app.py's /api/regime), nothing written to disk per call.
The classifier's thresholds (what counts as "high" volatility, etc.) come
from broker/regime_calibration.json, generated by running
python -m broker.regime_calibrate — a manual, roughly-yearly step that
pulls several years of the same four tickers and writes fresh terciles. If
that file is missing, hardcoded defaults are used instead. A second file,
regime_calibration_backtest.json, holds an out-of-sample calibration for
multi_sim so a backtest never trains on data from inside the window it's
simulating.
Checking freshness
The Data Management page's "External Data Sources" card shows the last-updated date for each cached source above (instrument snapshot, NSE holiday cache, regime calibration). Live/per-request data (quotes, candles) has no "last updated" to show — it's always current as of the last call.