An end-to-end architecture for enhancing reproducibility in nowcasting

Yves-Laurent Benichou, Thomas Faria, Antoine Palazzolo

11/03/2025

Context

  • Participation in the 1st round of nowcasting in 2023
  • Five statisticians from Insee across different teams:
    • 2 from economic studies (I. Moutachaker, A. Q-L-T)
    • 1 from short-term statistics (P. Leblanc)
    • 2 from the innovation lab (A. Palazzolo, T. Faria)
  • Economists (models) and data scientists (reproducibility)
  • Side project with limited time allocation

Main objective for Insee

Explore new techniques and assess their feasibility for production implementation.

Data Used

  • Reused the exact same pipeline as in 2023, but with different data
  • Challenge constraint: Only open-source data allowed
  • Four types of data:
    • Macroeconomic data from the Eurostat database
    • Financial data from the Yahoo database
    • Google Trends data
    • Other data sources

Models Used

  • Decision to combine both econometric and ML models:
    • RegARIMA
    • Exponential model
    • Dynamic Factor Models
    • XGBoost
    • Long Short-Term Memory (LSTM) model

A Reproducibility Challenge

  • A reproducibility prize for the challenge

  • Reproducibility of project very important in official statistics

  • Gaining expertise in good computing practices

    • Contribution of the Insee Innovation Lab to the challenge
    • Dissemination within Insee
    • Reuse for internal projects
    • Highlighting open-source projects

🚀 Architecture Built

  • Leveraged SSP Cloud for computation and storage
  • Chose R over Python to enhance reusability within Insee
  • Tech stack designed for maximum reproducibility 📌:
    • 🏗 Git & GitHub for version control
    • 📦 renv to manage R libraries
    • 📁 Structured project workflow with {targets} pipelines
    • 🐳 Docker for environment portability
    • 🤖 Automation using GitHub Actions

Architecture built

🤖 Automation using Github Actions

Final output : an interactive website

  • Making the results available through an interactive website Click Here