testing123/MAKEFILE_TUTORIAL.md

Makefile Tutorial: Generating PASBDC Reports

This project uses a makefile to automate the generation of Section 1 (Network Wide) and Section 3 (Center Specific) reports. The pipeline can process data either from manually exported CSV files or directly from Neoserra via the Export Module (API).

Table of Contents

1. Prerequisites
2. Data Source Modes
3. Running Reports
4. Targeting Specific Fiscal Years
5. Manual Setup for Scorecard Reports (Intervals & Satisfaction)
6. Output Structure

---

Prerequisites

• Python 3.12+: Ensure Python is installed.
• Virtual Environment: The makefile expects a virtual environment at .venv/.

  python -m venv .venv
  source .venv/bin/activate  # Linux/macOS
  # or
  .venv\Scripts\activate     # Windows
  pip install -r requirements.txt
  

---

Data Source Modes

The makefile supports two ways to ingest data from Neoserra.

1. Manual CSV Mode (Default)

This mode uses CSV files stored locally in the csv_files/ directory.

• Requirement: USE_EXPORT_MODULE=false (default).
• File Naming: CSVs must follow a specific naming convention: [type]_raw_[FYXX].csv. Example for FY25:
  • csv_files/trainings_raw_FY25.csv
  • csv_files/client_list_raw_FY25.csv
  • csv_files/nbs_milestones_raw_FY25.csv
  • csv_files/funding_milestones_raw_FY25.csv
  • Plus specialized interval CSVs: days_client_signup_to_start_FY25.csv, etc.

2. Export Module Mode (API)

This mode fetches data directly from Neoserra using pre-configured Export Module URLs.

• Requirement: USE_EXPORT_MODULE=true.
• Usage:

  make all USE_EXPORT_MODULE=true
  

• Note: Counselling Interval Analysis and Satisfaction Survey reports are skipped in this mode. Neoserra does not currently support exporting scorecard/survey data via the Export Module API; these reports require manual CSV exports.

---

Running Reports

Generate All Reports (Current FY)

To generate all Section 1 and Section 3 reports for the current fiscal year:

make all

Clean Previous Runs

To delete the generated reports for the resolved fiscal year:

make clean

---

Targeting Specific Fiscal Years

The makefile automatically calculates the CURRENT and PREVIOUS fiscal years based on the system date (FY starts Oct 1st). You can override this in two ways:

1. Using Helper Targets

make current    # Force current FY
make previous   # Force previous FY
make fy24       # Specific year
make fy23       # Specific year

2. Overriding the Variable

make all FISCAL_YEAR=FY25

---

Manual Setup for Scorecard Reports

Because Neoserra does not support exporting Scorecard data (Satisfaction and Intervals) via the API, these must always be run using CSV files. Critically, you must ensure USE_EXPORT_MODULE=false is set (which is the default).

1. Export Data from Neoserra

Manual exports must be performed in Neoserra for the following categories:

• Satisfaction Survey
• Counselling Intervals (Signup to Start, Start to Counselling, etc.)

2. Place CSVs in csv_files/

Ensure the files are named correctly according to the RESOLVED_FY. If you are running for FY26, your files must be:

• csv_files/satisfaction_survey_raw_FY26.csv
• csv_files/days_client_signup_to_start_FY26.csv
• csv_files/days_client_signup_to_counselling_FY26.csv
• csv_files/days_client_start_to_counselling_FY26.csv
• csv_files/days_initial_to_followup_FY26.csv
• csv_files/days_training_to_counselling_FY26.csv

3. Running the Reports

To generate these reports for a new fiscal year (e.g., FY26), ensure the CSVs are in place and run:

# Explicitly disable export module to ensure these targets are not skipped
make all FISCAL_YEAR=FY26 USE_EXPORT_MODULE=false

4. Overriding Specific Paths (Optional)

If you need to use a non-standard file location:

make all SATISFACTION_SURVEY_CSV=special_exports/survey_data.csv USE_EXPORT_MODULE=false

---

Output Structure

Reports are generated in the reports/ directory, sub-folder by fiscal year:

reports/
└── FY25/
    ├── section_1_trainings_analysis_FY25.docx
    ├── section_1_milestone_analysis_FY25.docx
    ├── section_3_trainings_topics_analysis_FY25.docx
    ├── section_3_trainings_analysis_word/ (Folder containing individual center docs)
    └── ... (folders containing generated graphs)