Skip to content

INTERMITTENT_DS_RUN

Unconstrained Intermittent Generation Forecasts (UIGF) for Dispatch.

Column Schema

DUID

DUID (or Area for non-scheduled) where this forecast applies

Warning

This column may contain funny characters such as / (slash) or # (hash). This may break hive-style partitioning.

For more information, see the dedicated documentation page.

SQL Type: VARCHAR2(20) Polars Type: pl.Categorical

This column is mandatory, and a primary key.

Examples:

  • ADPPV1
  • ALDGASF1
  • ARWF1
  • AVLSF1
  • BALDHWF1
  • BANGOWF1
  • BANGOWF2
  • BANN1
  • BERYLSF1
  • BLUEGSF1

FORECAST_PRIORITY

Unsuppressed forecasts with higher priority values are used in Dispatch in preference to unsuppressed forecasts with lower priority values.

SQL Type: NUMBER(10,0) Polars Type: pl.Int64

This column is mandatory, and a primary key.

Examples: 1, 436

OFFERDATETIME

Date and Time when this forecast submission was loaded.

SQL Type: DATE Polars Type: pl.Datetime()

This column is mandatory, and a primary key.

Example: 2026/08/31 23:55:00

ORIGIN

Origin of this forecast (PARTICIPANTID, AWEFS/ASEFS, or another vendor)

SQL Type: VARCHAR2(20) Polars Type: str

This column is mandatory, and a primary key.

Examples:

  • ALNTARES
  • ARARATWF
  • AWEFS_ASEFS
  • BERBANKW
  • BOCOROCK
  • BODWF1
  • BOMENSF
  • BULGANA
  • BUNGALA1
  • BUNGALA2

RUN_DATETIME

Date and Time where the forecast applies (dispatch interval ending)

SQL Type: DATE Polars Type: pl.Datetime()

This column is mandatory, and a primary key.

Example: 2026/09/01 00:05:00

AUTHORISEDBY

Authorising officer of this forecast (applicable for participant forecasts only). This column is not made available to the public.

SQL Type: VARCHAR2(20) Polars Type: str

This column is not mandatory.

COMMENTS

Comments relating to the forecast. This column is not made available to the public.

SQL Type: VARCHAR2(200) Polars Type: str

This column is not mandatory.

LASTCHANGED

Last date and time the record changed.

For more information, see the dedicated documentation page.

SQL Type: DATE Polars Type: pl.Datetime()

This column is not mandatory.

Example: 2026/08/31 23:55:01

MODEL

Metadata relating to the forecast. This column is not made available to the public.

SQL Type: VARCHAR2(30) Polars Type: str

This column is not mandatory.

PARTICIPANT_TIMESTAMP

Participant can document when the forecast was created

SQL Type: DATE Polars Type: pl.Datetime()

This column is not mandatory.

Example: 2026/08/31 23:55:00

SUPPRESSED_AEMO

Was this forecast suppressed by AEMO? Suppressed = 1,Not suppressed =0

SQL Type: NUMBER(1,0) Polars Type: pl.Int8

This column is not mandatory.

Examples: 0, 1

SUPPRESSED_PARTICIPANT

Was this forecast suppressed by the participant? Suppressed submissions may not be used, Suppressed = 1, Not suppressed =0

SQL Type: NUMBER(1,0) Polars Type: pl.Int8

This column is not mandatory.

Examples: 0, 1

TRANSACTION_ID

Uniquely identifies this interaction

SQL Type: VARCHAR2(100) Polars Type: str

This column is not mandatory.

Examples:

  • 00ac6a4a-e30e-4480-b655-353ba8a8984e
  • 00d462c2-1966-4bb5-bf5f-3d8b9b6e2c9d
  • 01977b93-5f42-46a9-bf78-55636d1a7cb8
  • 019b7398-e968-45d4-b648-0991919500c8
  • 01efd97a-aef3-4807-8a7c-a61df5f9ce7f
  • 023dff65-ef34-44fa-8747-b6356b18e535
  • 02988117-6889-4c45-a156-aa2de0aafa6a
  • 02bee6e9-de6f-421b-96b4-4fb9df827af7
  • 02e2f7a1-fee2-4d31-be4d-d1394a476e8d
  • 03060384-4343-40a6-a3b4-336a423871a6

New Data Timing

Publication of Dispatch solution. Data includes all valid forecasts for that interval

Code Examples

Polars

This table is so large that you should explicitly enumerate and cast every column, which will speed things up and prevents some errors. Although some of the datatypes are inferred, so they may be wrong. In particular, UInt16 vs UInt8 vs Int16 etc. If you get errors, you can swap these for just int.

This code deduplicates for you. Deduplication is normally only required across multiple months.

This code assumes that all columns are present, which may not be true for older files.

import polars as pl
import polars.selectors as cs

# e.g. "2029/03/15 00:05:00"
dt_s_fmt = "%Y/%m/%d %H:%M:%S"

csv_paths = ["SOMEFILE.CSV", "OTHERFILE.CSV"]

lf = (
    pl.concat(
        [
            pl.scan_csv(
                csv_path,  # must be unzipped beforehand
                skip_rows=1,  # skip the header
                infer_schema=False,  # much faster
            )
            .filter(cs.by_index(0) == "D")  # drop the footer
            .select(~cs.by_index(range(4)))  # drop metadata columns
            for csv_path in csv_paths
        ],
        how="diagonal_relaxed",
    )
    .with_columns(
        pl.col(
            "OFFERDATETIME", "RUN_DATETIME", "LASTCHANGED", "PARTICIPANT_TIMESTAMP"
        ).str.strptime(pl.Datetime, dt_s_fmt),
    )
    .cast(
        {
            "DUID": pl.Categorical,
            "FORECAST_PRIORITY": pl.Int64,
            "ORIGIN": str,
            "AUTHORISEDBY": str,
            "COMMENTS": str,
            "MODEL": str,
            "SUPPRESSED_AEMO": pl.Int8,
            "SUPPRESSED_PARTICIPANT": pl.Int8,
            "TRANSACTION_ID": str,
        }
    )
    # Deduplicate
    # These files are so large that this step may cause your laptop to run out of memory.
    # We want one row per ['DUID', 'FORECAST_PRIORITY', 'OFFERDATETIME', 'ORIGIN', 'RUN_DATETIME']
    # Tie-breaker: greatest value of LASTCHANGED
    .sort(LASTCHANGED, descending=False)
    .unique(
        subset=["DUID", "FORECAST_PRIORITY", "OFFERDATETIME", "ORIGIN", "RUN_DATETIME"],
        keep="last",
    )
)

Nemosis

This table is not yet available in Nemosis.

Example File

From PUBLIC_ARCHIVE_INTERMITTENT_DS_RUN_FILE01_202609010000.zip

RUN_DATETIMEDUIDOFFERDATETIMEORIGINFORECAST_PRIORITYAUTHORISEDBYCOMMENTSLASTCHANGEDMODELPARTICIPANT_TIMESTAMPSUPPRESSED_AEMOSUPPRESSED_PARTICIPANTTRANSACTION_ID
2026/09/01 00:05:00ADPPV12026/08/31 23:59:16AWEFS_ASEFS3012026/08/31 23:59:162026/08/31 23:59:02004668178f-da98-466d-bdf9-c4973247e8c6
2026/09/01 00:05:00ADPPV12026/08/31 23:59:16AWEFS_ASEFS3022026/08/31 23:59:162026/08/31 23:59:0300beff2dbd-652a-4ef4-9306-d694d98f12c6
2026/09/01 00:05:00ADPPV12026/08/31 23:59:41AWEFS_ASEFS3012026/08/31 23:59:412026/08/31 23:59:2600697e423c-6e73-4ad1-8567-82fb6e7de31b
2026/09/01 00:05:00ADPPV12026/08/31 23:59:41AWEFS_ASEFS3022026/08/31 23:59:412026/08/31 23:59:26000acd2bb3-9a76-4390-860b-17b940ab8d1d
2026/09/01 00:05:00ALDGASF12026/08/31 23:59:16AWEFS_ASEFS1012026/08/31 23:59:162026/08/31 23:59:0200c142122e-03fe-4b6f-bfdd-2be97085780c
2026/09/01 00:05:00ALDGASF12026/08/31 23:59:16AWEFS_ASEFS1022026/08/31 23:59:162026/08/31 23:59:03000b0d3f07-643c-4785-b2f2-822118fbf9ad
2026/09/01 00:05:00ALDGASF12026/08/31 23:59:41AWEFS_ASEFS1012026/08/31 23:59:412026/08/31 23:59:260052104d5d-5951-4878-bde1-d844bd73c4c9
2026/09/01 00:05:00ALDGASF12026/08/31 23:59:41AWEFS_ASEFS1022026/08/31 23:59:412026/08/31 23:59:2600f22be5ef-847d-42eb-80ca-54a7a11f613a
2026/09/01 00:05:00ARWF12026/08/31 23:58:47ARARATWF1012026/08/31 23:58:47009635f7b3-5255-4217-b848-f2e6d6a19101
2026/09/01 00:05:00ARWF12026/08/31 23:59:16AWEFS_ASEFS1012026/08/31 23:59:162026/08/31 23:59:020075d150f3-427e-453c-adcd-23df37f9ed93
2026/09/01 00:05:00ARWF12026/08/31 23:59:16AWEFS_ASEFS1022026/08/31 23:59:162026/08/31 23:59:03005344a734-496c-4529-9524-ce62aa223dea
...

CSETP.WORLDDVD_INTERMITTENT_DS_RUNAEMOPUBLIC2026/10/0915:42:27001791524547405MONTHLY_ARCHIVE001791524547405
IDEMANDINTERMITTENT_DS_RUN1RUN_DATETIMEDUIDOFFERDATETIMEORIGINFORECAST_PRIORITYAUTHORISEDBYCOMMENTSLASTCHANGEDMODELPARTICIPANT_TIMESTAMPSUPPRESSED_AEMOSUPPRESSED_PARTICIPANTTRANSACTION_ID
DDEMANDINTERMITTENT_DS_RUN12026/09/01 00:05:00ADPPV12026/08/31 23:59:16AWEFS_ASEFS3012026/08/31 23:59:162026/08/31 23:59:02004668178f-da98-466d-bdf9-c4973247e8c6
DDEMANDINTERMITTENT_DS_RUN12026/09/01 00:05:00ADPPV12026/08/31 23:59:16AWEFS_ASEFS3022026/08/31 23:59:162026/08/31 23:59:0300beff2dbd-652a-4ef4-9306-d694d98f12c6
DDEMANDINTERMITTENT_DS_RUN12026/09/01 00:05:00ADPPV12026/08/31 23:59:41AWEFS_ASEFS3012026/08/31 23:59:412026/08/31 23:59:2600697e423c-6e73-4ad1-8567-82fb6e7de31b
DDEMANDINTERMITTENT_DS_RUN12026/09/01 00:05:00ADPPV12026/08/31 23:59:41AWEFS_ASEFS3022026/08/31 23:59:412026/08/31 23:59:26000acd2bb3-9a76-4390-860b-17b940ab8d1d
DDEMANDINTERMITTENT_DS_RUN12026/09/01 00:05:00ALDGASF12026/08/31 23:59:16AWEFS_ASEFS1012026/08/31 23:59:162026/08/31 23:59:0200c142122e-03fe-4b6f-bfdd-2be97085780c
DDEMANDINTERMITTENT_DS_RUN12026/09/01 00:05:00ALDGASF12026/08/31 23:59:16AWEFS_ASEFS1022026/08/31 23:59:162026/08/31 23:59:03000b0d3f07-643c-4785-b2f2-822118fbf9ad
DDEMANDINTERMITTENT_DS_RUN12026/09/01 00:05:00ALDGASF12026/08/31 23:59:41AWEFS_ASEFS1012026/08/31 23:59:412026/08/31 23:59:260052104d5d-5951-4878-bde1-d844bd73c4c9
DDEMANDINTERMITTENT_DS_RUN12026/09/01 00:05:00ALDGASF12026/08/31 23:59:41AWEFS_ASEFS1022026/08/31 23:59:412026/08/31 23:59:2600f22be5ef-847d-42eb-80ca-54a7a11f613a
DDEMANDINTERMITTENT_DS_RUN12026/09/01 00:05:00ARWF12026/08/31 23:58:47ARARATWF1012026/08/31 23:58:47009635f7b3-5255-4217-b848-f2e6d6a19101
DDEMANDINTERMITTENT_DS_RUN12026/09/01 00:05:00ARWF12026/08/31 23:59:16AWEFS_ASEFS1012026/08/31 23:59:162026/08/31 23:59:020075d150f3-427e-453c-adcd-23df37f9ed93
DDEMANDINTERMITTENT_DS_RUN12026/09/01 00:05:00ARWF12026/08/31 23:59:16AWEFS_ASEFS1022026/08/31 23:59:162026/08/31 23:59:03005344a734-496c-4529-9524-ce62aa223dea
...
CEND OF REPORT1003

RUN_DATETIME,DUID,OFFERDATETIME,ORIGIN,FORECAST_PRIORITY,AUTHORISEDBY,COMMENTS,LASTCHANGED,MODEL,PARTICIPANT_TIMESTAMP,SUPPRESSED_AEMO,SUPPRESSED_PARTICIPANT,TRANSACTION_ID
"2026/09/01 00:05:00",ADPPV1,"2026/08/31 23:59:16",AWEFS_ASEFS,301,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:02",0,0,4668178f-da98-466d-bdf9-c4973247e8c6
"2026/09/01 00:05:00",ADPPV1,"2026/08/31 23:59:16",AWEFS_ASEFS,302,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:03",0,0,beff2dbd-652a-4ef4-9306-d694d98f12c6
"2026/09/01 00:05:00",ADPPV1,"2026/08/31 23:59:41",AWEFS_ASEFS,301,,,"2026/08/31 23:59:41",,"2026/08/31 23:59:26",0,0,697e423c-6e73-4ad1-8567-82fb6e7de31b
"2026/09/01 00:05:00",ADPPV1,"2026/08/31 23:59:41",AWEFS_ASEFS,302,,,"2026/08/31 23:59:41",,"2026/08/31 23:59:26",0,0,0acd2bb3-9a76-4390-860b-17b940ab8d1d
"2026/09/01 00:05:00",ALDGASF1,"2026/08/31 23:59:16",AWEFS_ASEFS,101,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:02",0,0,c142122e-03fe-4b6f-bfdd-2be97085780c
"2026/09/01 00:05:00",ALDGASF1,"2026/08/31 23:59:16",AWEFS_ASEFS,102,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:03",0,0,0b0d3f07-643c-4785-b2f2-822118fbf9ad
"2026/09/01 00:05:00",ALDGASF1,"2026/08/31 23:59:41",AWEFS_ASEFS,101,,,"2026/08/31 23:59:41",,"2026/08/31 23:59:26",0,0,52104d5d-5951-4878-bde1-d844bd73c4c9
"2026/09/01 00:05:00",ALDGASF1,"2026/08/31 23:59:41",AWEFS_ASEFS,102,,,"2026/08/31 23:59:41",,"2026/08/31 23:59:26",0,0,f22be5ef-847d-42eb-80ca-54a7a11f613a
"2026/09/01 00:05:00",ARWF1,"2026/08/31 23:58:47",ARARATWF,101,,,"2026/08/31 23:58:47",,,0,0,9635f7b3-5255-4217-b848-f2e6d6a19101
"2026/09/01 00:05:00",ARWF1,"2026/08/31 23:59:16",AWEFS_ASEFS,101,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:02",0,0,75d150f3-427e-453c-adcd-23df37f9ed93
"2026/09/01 00:05:00",ARWF1,"2026/08/31 23:59:16",AWEFS_ASEFS,102,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:03",0,0,5344a734-496c-4529-9524-ce62aa223dea
...

C,SETP.WORLD,DVD_INTERMITTENT_DS_RUN,AEMO,PUBLIC,2026/10/09,15:42:27,001791524547405,MONTHLY_ARCHIVE,001791524547405
I,DEMAND,INTERMITTENT_DS_RUN,1,RUN_DATETIME,DUID,OFFERDATETIME,ORIGIN,FORECAST_PRIORITY,AUTHORISEDBY,COMMENTS,LASTCHANGED,MODEL,PARTICIPANT_TIMESTAMP,SUPPRESSED_AEMO,SUPPRESSED_PARTICIPANT,TRANSACTION_ID
D,DEMAND,INTERMITTENT_DS_RUN,1,"2026/09/01 00:05:00",ADPPV1,"2026/08/31 23:59:16",AWEFS_ASEFS,301,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:02",0,0,4668178f-da98-466d-bdf9-c4973247e8c6
D,DEMAND,INTERMITTENT_DS_RUN,1,"2026/09/01 00:05:00",ADPPV1,"2026/08/31 23:59:16",AWEFS_ASEFS,302,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:03",0,0,beff2dbd-652a-4ef4-9306-d694d98f12c6
D,DEMAND,INTERMITTENT_DS_RUN,1,"2026/09/01 00:05:00",ADPPV1,"2026/08/31 23:59:41",AWEFS_ASEFS,301,,,"2026/08/31 23:59:41",,"2026/08/31 23:59:26",0,0,697e423c-6e73-4ad1-8567-82fb6e7de31b
D,DEMAND,INTERMITTENT_DS_RUN,1,"2026/09/01 00:05:00",ADPPV1,"2026/08/31 23:59:41",AWEFS_ASEFS,302,,,"2026/08/31 23:59:41",,"2026/08/31 23:59:26",0,0,0acd2bb3-9a76-4390-860b-17b940ab8d1d
D,DEMAND,INTERMITTENT_DS_RUN,1,"2026/09/01 00:05:00",ALDGASF1,"2026/08/31 23:59:16",AWEFS_ASEFS,101,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:02",0,0,c142122e-03fe-4b6f-bfdd-2be97085780c
D,DEMAND,INTERMITTENT_DS_RUN,1,"2026/09/01 00:05:00",ALDGASF1,"2026/08/31 23:59:16",AWEFS_ASEFS,102,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:03",0,0,0b0d3f07-643c-4785-b2f2-822118fbf9ad
D,DEMAND,INTERMITTENT_DS_RUN,1,"2026/09/01 00:05:00",ALDGASF1,"2026/08/31 23:59:41",AWEFS_ASEFS,101,,,"2026/08/31 23:59:41",,"2026/08/31 23:59:26",0,0,52104d5d-5951-4878-bde1-d844bd73c4c9
D,DEMAND,INTERMITTENT_DS_RUN,1,"2026/09/01 00:05:00",ALDGASF1,"2026/08/31 23:59:41",AWEFS_ASEFS,102,,,"2026/08/31 23:59:41",,"2026/08/31 23:59:26",0,0,f22be5ef-847d-42eb-80ca-54a7a11f613a
D,DEMAND,INTERMITTENT_DS_RUN,1,"2026/09/01 00:05:00",ARWF1,"2026/08/31 23:58:47",ARARATWF,101,,,"2026/08/31 23:58:47",,,0,0,9635f7b3-5255-4217-b848-f2e6d6a19101
D,DEMAND,INTERMITTENT_DS_RUN,1,"2026/09/01 00:05:00",ARWF1,"2026/08/31 23:59:16",AWEFS_ASEFS,101,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:02",0,0,75d150f3-427e-453c-adcd-23df37f9ed93
D,DEMAND,INTERMITTENT_DS_RUN,1,"2026/09/01 00:05:00",ARWF1,"2026/08/31 23:59:16",AWEFS_ASEFS,102,,,"2026/08/31 23:59:16",,"2026/08/31 23:59:03",0,0,5344a734-496c-4529-9524-ce62aa223dea
...
C,"END OF REPORT",1003

Availability

Visibility: Private & Public Next-Day

This table is in the public monthly MMSDM archive.

I have collated the list of all those files into one place, in this CSV metadata file. (Columns year, month, url, compressed_size in bytes).

The earliest available data is month 8 of 2024. The most recent available data is month 9 of 2026.

Every month in between has a file. (No gaps.)

Data Size

Each year of data is approximately 12 GB when zipped, 122 GB as unzipped CSV.

Tip

This table is large. Simply calling pd.read_csv() might fill up all the memory on your laptop. (As a rule of thumb, you need double the amount of memory as the size of the CSV.) You will need to think hard about memory optimisation. e.g. consider using Polars instead of Pandas, and selecting only the columns you care about.

Deduplication and PDR Loader Configuration

The primary keys are:

The transaction type is "INSERT-UPDATE", which means that you will likely see duplicate data with different values.

You should deduplicate by sorting based on LASTCHANGED, within each group of primary keys.

PDR Loader configuration is:

Report Name Report Subtype Version
DEMAND INTERMITTENT_DS_RUN 1

This means that the CSV files start with metadata columns:

I,DEMAND,INTERMITTENT_DS_RUN,1,...
D,DEMAND,INTERMITTENT_DS_RUN,1,...

ROW_FILTER_TYPE is NONE

This page as JSON (schema documented here, but may change)

Download a JSON Schema specification for this table.

Official AEMO schema documentation

Other Information

Package: DEMAND_FORECASTS