John Mau discusses time series forecasting, arguing it's a neglected but crucial area in data science, distinct from standard predictive modeling and often technically challenging. He critiques the complexity and potential opaqueness of traditional methods like ARMA/ARMAX, emphasizing the importance of out-of-sample evaluation over relying solely on package diagnostics. The talk surveys forecasting methodologies, including ARMA-derived models, brute-force approaches (common with neural networks), and decompositional/hidden state methods (like Facebook Prophet and Stan). Using an artificial example of modeling restaurant diners (with underlying sinx behavior), Mau introduces a key concept: partitioning drivers into unobserved components (e.g., "loyal" customers influenced by auto-regression and durable factors, vs. "transient" customers influenced by temporary events). He demonstrates how Stan, a Bayesian inference tool, can effectively model this hidden structure, providing not just forecasts but also valuable decomposition and explainability by estimating the impact of different types of external regressors. The core message is that for business applications, the ability to model and understand the underlying structure of the time series using tools like Stan or Prophet is often more valuable than the specifics of the forecasting technique itself. Code examples in R and Python are shared via GitHub.
Time Series Forecasting: Understanding Structure with Stan
0:00:40 Introduction to Time Series Forecasting: John Mau introduces time series forecasting as a somewhat neglected, technical, and distinct area within data science.
0:01:46 Outline and Resources: The talk aims to outline an approach, work through an artificial example (fitting sinx), introduce the Stan modeling language, and share extensive code examples (R/Python) via GitHub.
0:02:41 Defining Time Series Forecasting: The core problem is predicting future numerical values based only on past observed values, often visualized with time on the x-axis. Uncertainty intervals are important.
0:03:54 Importance of Prediction Horizon: Crucial to distinguish between predicting one step ahead versus multiple steps ahead, as the latter is often more valuable but harder.
0:04:44 Evaluating Forecasts: Emphasizes the need for out-of-sample evaluation (using hold-out data) rather than relying on model-internal diagnostics to assess prediction quality.
0:06:05 Applications & Example: Used in business (finance, demand forecasting) and science (tides). The talk uses an artificial example: modeling daily diners in a restaurant with sinx patterns.
0:07:07 Critiques of Standard Approaches: Notes the high technical barrier of many time series packages (using Facebook Prophet team's comment) and potential pitfalls like misleading diagnostics or degenerate predictions shown in package documentation.
0:09:23 External Regressors & ARMAX Issues: Discusses incorporating external information ("external regressors") known about the future. Highlights the ambiguity and varying implementations of ARMAX models, making it hard to know the underlying assumptions.
0:11:21 Families of Forecasting Methods: Categorizes methods into:
ARMA-derived (AR, MA, ARIMA, smoothing)
Brute Force (direct prediction for each future step, common in NNs)
Decompositional / Hidden State (Profit, Stan)
0:16:51 Decompositional Methods (e.g., Profit): These methods break down the series into components like trend, seasonality (weekly, yearly), and effects of external regressors, providing explainability alongside forecasts.
0:19:03 Bayesian / Hidden State Inference (Stan): Introduces Stan as a powerful tool for Bayesian inference. It allows specifying a model structure, and Stan computationally infers plausible parameter values based on the data.
0:21:10 Novel Example Structure: Proposes modeling restaurant guests as a sum of two unobserved groups:
Loyal Guests: Influenced by past guest counts (auto-regressive) and durable factors (e.g., reviews, price changes).
Transient Guests: Influenced only by non-durable external factors (e.g., local events, coupons).
0:23:05 Stan Solution for Hidden Structure: Demonstrates how Stan can model this underlabeled problem, successfully inferring the separate contributions of loyal and transient guests from the total count and estimating the distinct impacts of durable vs. transient external regressors.
0:24:26 Benefits of Stan Approach: Provides decomposition, explainability, and accurate coefficient estimates for different types of drivers, which traditional methods struggle with.
0:25:14 Stan Code: Briefly shows Stan code, illustrating it directly translates the conceptual model equations.
0:25:58 Key Takeaway: For business forecasting, defining an actionable structure reflecting domain knowledge (like the loyal/transient split) and using tools like Stan or Prophet to model it offers more value (explainability, targeted interventions) than focusing purely on sophisticated forecasting techniques.
0:26:30 Q&A Highlights: Discusses SK time (another potential package), the artificial nature of the example's transient guest pattern, and how LSTMs (neural nets) compare (good but data-hungry, often use brute-force style prep).