This conference argues that statistics is the fundamental, often invisible, language and art form underpinning Artificial Intelligence (AI) and Machine Learning (ML). Statistics is positioned not as a set of obsolete algorithms, but as an indispensable foundation for hypothesis generation, inference, and data-driven narration. The speaker, Paula Leal Varón, demonstrates this through the application of the six-phase CRISP-DM methodology to a real-world predictive analytics problem: estimating a student's standardized Saber Pro test score based on academic and socioeconomic characteristics. Key statistical concepts—including variable typing (regression vs. classification), data preparation (imputation, outlier handling, unbalanced classes), cross-validation, and model metrics—are asserted as necessary tools for ensuring models are robust, interpretable, and ultimately valuable for organizational decision-making. The discussion concludes that expertise in statistical thinking is essential, as easily generated ML algorithms often lack the required quality control and interpretation necessary for deployment.
Modeling is Telling Stories with Data: The Statistical Art Behind AI
0:00 Introduction: The presentation is part of the 9th Virtual Statistics Seminar, focusing on AI and Data Science. Speaker Paula Leal Varón, a data analyst, presents the concept of "Modeling as telling stories with data."
3:01 Core Thesis: Machine Learning models are not replacing statistics; they are merely a highly developed extension of statistical models. Fundamental concepts like inference, sampling, and randomness are the pillars of AI.
5:04 Methodology: The project uses the six-phase, iterative CRISP-DM (Cross Industry Standard Process for Data Mining) methodology: Business Understanding, Data Understanding, Data Preparation, Modeling, Evaluation, and Deployment.
6:48 Case Study & Problem Framing (Business Understanding): The goal is to predict students' overall Saber Pro standardized test scores (0-300 points) based on characteristics like previous academic scores (Saber 11), socioeconomic status, and grades.
8:57 Statistical Decision Point: The target variable (Saber Pro score) can be treated as continuous (Regression problem) or grouped into ranges (Classification problem). This statistical choice dictates the approach.
11:36 Data Quality Assessment (Data Understanding): Analysis revealed discrepancies between user expectations and data behavior. Key challenges identified statistically include biases (e.g., highly concentrated high scores), missing data, and unbalanced classes (e.g., gender imbalance, 13:13).
14:46 Data Preparation (The Statistical Tool): Statistics is necessary to detect, correct, and transform data. Problems addressed include:
Missing values (requiring careful imputation using techniques like median/mode or context-aware methods, 17:11).
Outliers (requiring analysis to determine if they should be trimmed or capped, 18:21).
Unbalanced classes (requiring resampling techniques or model choice that handles imbalance, 19:10).
16:20 Critique of Algorithm Usage: It is dismissed that easily generated AI/ML code (e.g., via prompt engineering) is sufficient; running a model without deep statistical knowledge often leads to poor, uninterpretable results.
21:30 Modeling Choice: Due to highly unbalanced classes in the score distribution (63% High score, 9% Low score), a classification approach was chosen using models robust to imbalance, such as Random Forest (RF) and SGBoost.
25:33 Hyperparameter Tuning & Cross-Validation: Statistical concepts like cross-validation are essential for tuning hyperparameters (e.g., number and depth of trees in RF) to optimize the model configuration and ensure robust estimates.
28:01 Technical Evaluation Caution: High accuracy metrics (e.g., 90% or 95%) in real-world data should be treated skeptically, as they often indicate the model is "learning by rote" (overfitting). Metrics (F1, Recall, Confusion Matrix) must be used judiciously.
29:40 Functional Evaluation & Usefulness: Model choice must involve both technical and functional teams. The model's value is derived from its ability to provide interpretable insights (e.g., via variable importance analysis) and guide human decisions (e.g., creating early warnings for students, 30:17).
32:38 Final Model Selection: Although metrics for multiple models (RF, SGBoost) were very similar (~63% accuracy), a customized Random Forest was chosen for its stability, low computational cost, and ability to easily analyze variable importance, which adds value to users.
36:02 Conclusion on Necessity: Statistics is the fundamental basis for AI/ML; 70-80% of project time is typically spent on ensuring high data quality. Statistical evaluation provides the necessary confidence and solid arguments for model results.
41:14 Q&A: Subjective Bias: Acknowledgment that functional area knowledge/subjective arguments must be incorporated into variable selection, even if statistical tests suggest a variable is weak, as business relevance is paramount.
45:47 Q&A: Statistics as AI's Eyes: Without foundational statistical concepts (sampling, metrics, data cleaning), machine learning techniques lack stability and interpretability.
50:32 Peer Confirmation: A fellow university professor confirms their own independent project predicting student scores found Random Forest to be among the best models, validating the presenter's methodology and findings (e.g., the importance of quantitative reasoning score and student age).
53:34 Q&A: Safest AI for Data: For security and privacy, users should pay for AI resources. Recommends 'Jeninay' (as far as I can tell, likely Gemini or paid GPT) for statistical analysis and code generation.