MTPASA_RESERVELIMITSOLUTION

MTPASA_RESERVELIMITSOLUTION is obsolete from 2017 End of Year DM4.27 Release. MT PASA Solution table reporting whether a MT PASA Reserve requirement is binding for each day and LDC block of the run.

Deprecation Warning

This table is in the HISTORICAL TABLES package, so is likely deprecated.

Source

MTPASA_RESERVELIMITSOLUTION is updated each MTPASA run (i.e. weekly).

Volume

400,000 rows per year

Column Schema

DAY

Day this solution is for. Generated from the solution file period id.

SQL Type: DATE Polars Type: pl.Datetime()

This column is mandatory, and a primary key.

Example: 2018/05/06 00:00:00

ENERGYBLOCK

Sunday at start of the week for this solutions energy block. Generated from the solution file energy block.

SQL Type: DATE Polars Type: pl.Datetime()

This column is mandatory, and a primary key.

Example: 2018/05/06 00:00:00

LDCBLOCK

Load Duration Curve block this solution is for. Generated from the solution file period id.

SQL Type: NUMBER(3,0) Polars Type: pl.Int16

This column is mandatory, and a primary key.

Example: 1

RESERVELIMITID

The unique identifier of the MT PASA LRC Reserve Requirement.

SQL Type: VARCHAR2(20) Polars Type: str

This column is mandatory, and a primary key.

Examples:

MTPASAMRL01_NSW1
MTPASAMRL01_QLD1
MTPASAMRL01_TAS1
MTPASAMRL01_VICSA_01
MTPASAMRL01_VICSA_02
MTPASAMRL01_VICSA_03
MTPASAMRL01_VICSA_04
MTPASAMRL01_VICSA_05
MTPASAMRL01_VICSA_06

RUN_DATETIME

Date processing of the run begins

SQL Type: DATE Polars Type: pl.Datetime()

This column is mandatory, and a primary key.

Example: 2018/05/01 12:00:00

RUN_NO

Unique run ID. Generated from the solution file Case ID.

SQL Type: NUMBER(3,0) Polars Type: pl.Int16

This column is mandatory, and a primary key.

Example: 20

RUNTYPE

Type of run. Values are RELIABILITY_LRC and OUTAGE_LRC

SQL Type: VARCHAR2(20) Polars Type: pl.Categorical

This column is mandatory, and a primary key.

Example: RELIABILITY_LRC

LASTCHANGED

Timestamp the record was last modified.

For more information, see the dedicated documentation page.

SQL Type: DATE Polars Type: pl.Datetime()

This column is not mandatory.

Example: 2018/05/01 12:31:15

MARGINALVALUE

Marginal Value of the Reserve Requirement Constraint. A non-zero value indicates that the reserve requirement is binding.

SQL Type: NUMBER(16,6) Polars Type: pl.Float64

This column is not mandatory.

Examples: -75.493070, 0, -14491.154403

New Data Timing

On MTPASA solution published to the market (usually once per week)

Code Examples

Polars

Casting is enums, categoricals etc is not essential, but may speed up parsing. Swapping int for pl.UInt16, pl.Int32 would also speed up parsing. However some of the datatypes are inferred, so they may be wrong.

This code deduplicates for you. Deduplication is normally only required across multiple months.

This code assumes that all columns are present, which may not be true for older files.

import polars as pl
import polars.selectors as cs

# e.g. "2029/03/15 00:05:00"
dt_s_fmt = "%Y/%m/%d %H:%M:%S"

csv_paths = ["SOMEFILE.CSV", "OTHERFILE.CSV"]

lf = (
    pl.concat(
        [
            pl.scan_csv(
                csv_path,  # must be unzipped beforehand
                skip_rows=1,  # skip the header
                infer_schema=False,  # much faster
            )
            .filter(cs.by_index(0) == "D")  # drop the footer
            .select(~cs.by_index(range(4)))  # drop metadata columns
            for csv_path in csv_paths
        ],
        how="diagonal_relaxed",
    )
    .with_columns(
        pl.col("DAY", "ENERGYBLOCK", "RUN_DATETIME", "LASTCHANGED").str.strptime(
            pl.Datetime, dt_s_fmt
        ),
    )
    .cast(
        {
            "LDCBLOCK": int,  # or pl.Int16
            "RESERVELIMITID": str,
            "RUN_NO": int,  # or pl.Int16
            "RUNTYPE": pl.Categorical,
            "MARGINALVALUE": pl.Float64,
        }
    )
    # Deduplicate
    # We want one row per ['DAY', 'ENERGYBLOCK', 'LDCBLOCK', 'RESERVELIMITID', 'RUN_DATETIME', 'RUN_NO', 'RUNTYPE']
    # Tie-breaker: greatest value of LASTCHANGED
    .sort(LASTCHANGED, descending=False)
    .unique(
        subset=[
            "DAY",
            "ENERGYBLOCK",
            "LDCBLOCK",
            "RESERVELIMITID",
            "RUN_DATETIME",
            "RUN_NO",
            "RUNTYPE",
        ],
        keep="last",
    )
)

Nemosis

This table is not yet available in Nemosis.

Example File

From PUBLIC_DVD_MTPASA_RESERVELIMITSOLUTION_201805010000.zip

Table without MetadataTable with MetadataRaw Text without Metadata

RUN_DATETIME	RUN_NO	RUNTYPE	ENERGYBLOCK	DAY	LDCBLOCK	RESERVELIMITID	MARGINALVALUE	LASTCHANGED
2018/05/01 12:00:00	20	RELIABILITY_LRC	2018/05/06 00:00:00	2018/05/06 00:00:00	1	MTPASAMRL01_NSW1	0	2018/05/01 12:31:15
2018/05/01 12:00:00	20	RELIABILITY_LRC	2018/05/06 00:00:00	2018/05/06 00:00:00	1	MTPASAMRL01_QLD1	0	2018/05/01 12:31:15
2018/05/01 12:00:00	20	RELIABILITY_LRC	2018/05/06 00:00:00	2018/05/06 00:00:00	1	MTPASAMRL01_TAS1	0	2018/05/01 12:31:15
2018/05/01 12:00:00	20	RELIABILITY_LRC	2018/05/06 00:00:00	2018/05/06 00:00:00	1	MTPASAMRL01_VICSA_01	0	2018/05/01 12:31:15
2018/05/01 12:00:00	20	RELIABILITY_LRC	2018/05/06 00:00:00	2018/05/06 00:00:00	1	MTPASAMRL01_VICSA_02	0	2018/05/01 12:31:15
2018/05/01 12:00:00	20	RELIABILITY_LRC	2018/05/06 00:00:00	2018/05/06 00:00:00	1	MTPASAMRL01_VICSA_03	0	2018/05/01 12:31:15
2018/05/01 12:00:00	20	RELIABILITY_LRC	2018/05/06 00:00:00	2018/05/06 00:00:00	1	MTPASAMRL01_VICSA_04	0	2018/05/01 12:31:15
2018/05/01 12:00:00	20	RELIABILITY_LRC	2018/05/06 00:00:00	2018/05/06 00:00:00	1	MTPASAMRL01_VICSA_05	0	2018/05/01 12:31:15
2018/05/01 12:00:00	20	RELIABILITY_LRC	2018/05/06 00:00:00	2018/05/06 00:00:00	1	MTPASAMRL01_VICSA_06	0	2018/05/01 12:31:15
2018/05/01 12:00:00	20	RELIABILITY_LRC	2018/05/06 00:00:00	2018/05/07 00:00:00	1	MTPASAMRL01_NSW1	0	2018/05/01 12:31:15
2018/05/01 12:00:00	20	RELIABILITY_LRC	2018/05/06 00:00:00	2018/05/07 00:00:00	1	MTPASAMRL01_QLD1	0	2018/05/01 12:31:15
...

C

SETP.WORLD

DVD_MTPASA_RESERVELIMITSOLUTION

AEMO

PUBLIC

2018/06/07

02:35:04

0000000295553422

0000000295553313

I

MTPASA

RESERVELIMITSOLUTION

1

RUN_DATETIME

RUN_NO

RUNTYPE

ENERGYBLOCK

DAY

LDCBLOCK

RESERVELIMITID

MARGINALVALUE

LASTCHANGED

D

MTPASA

RESERVELIMITSOLUTION

1

2018/05/01 12:00:00

20

RELIABILITY_LRC

2018/05/06 00:00:00

1

MTPASAMRL01_NSW1

0

2018/05/01 12:31:15

D

MTPASA

RESERVELIMITSOLUTION

1

2018/05/01 12:00:00

20

RELIABILITY_LRC

2018/05/06 00:00:00

1

MTPASAMRL01_QLD1

0

2018/05/01 12:31:15

D

MTPASA

RESERVELIMITSOLUTION

1

2018/05/01 12:00:00

20

RELIABILITY_LRC

2018/05/06 00:00:00

1

MTPASAMRL01_TAS1

0

2018/05/01 12:31:15

D

MTPASA

RESERVELIMITSOLUTION

1

2018/05/01 12:00:00

20

RELIABILITY_LRC

2018/05/06 00:00:00

1

MTPASAMRL01_VICSA_01

0

2018/05/01 12:31:15

D

MTPASA

RESERVELIMITSOLUTION

1

2018/05/01 12:00:00

20

RELIABILITY_LRC

2018/05/06 00:00:00

1

MTPASAMRL01_VICSA_02

0

2018/05/01 12:31:15

D

MTPASA

RESERVELIMITSOLUTION

1

2018/05/01 12:00:00

20

RELIABILITY_LRC

2018/05/06 00:00:00

1

MTPASAMRL01_VICSA_03

0

2018/05/01 12:31:15

D

MTPASA

RESERVELIMITSOLUTION

1

2018/05/01 12:00:00

20

RELIABILITY_LRC

2018/05/06 00:00:00

1

MTPASAMRL01_VICSA_04

0

2018/05/01 12:31:15

D

MTPASA

RESERVELIMITSOLUTION

1

2018/05/01 12:00:00

20

RELIABILITY_LRC

2018/05/06 00:00:00

1

MTPASAMRL01_VICSA_05

0

2018/05/01 12:31:15

D

MTPASA

RESERVELIMITSOLUTION

1

2018/05/01 12:00:00

20

RELIABILITY_LRC

2018/05/06 00:00:00

1

MTPASAMRL01_VICSA_06

0

2018/05/01 12:31:15

D

MTPASA

RESERVELIMITSOLUTION

1

2018/05/01 12:00:00

20

RELIABILITY_LRC

2018/05/06 00:00:00

2018/05/07 00:00:00

1

MTPASAMRL01_NSW1

0

2018/05/01 12:31:15

D

MTPASA

RESERVELIMITSOLUTION

1

2018/05/01 12:00:00

20

RELIABILITY_LRC

2018/05/06 00:00:00

2018/05/07 00:00:00

1

MTPASAMRL01_QLD1

0

2018/05/01 12:31:15

...

C

END OF REPORT

1003

RUN_DATETIME,RUN_NO,RUNTYPE,ENERGYBLOCK,DAY,LDCBLOCK,RESERVELIMITID,MARGINALVALUE,LASTCHANGED
"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_NSW1,0,"2018/05/01 12:31:15"
"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_QLD1,0,"2018/05/01 12:31:15"
"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_TAS1,0,"2018/05/01 12:31:15"
"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_01,0,"2018/05/01 12:31:15"
"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_02,0,"2018/05/01 12:31:15"
"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_03,0,"2018/05/01 12:31:15"
"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_04,0,"2018/05/01 12:31:15"
"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_05,0,"2018/05/01 12:31:15"
"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_06,0,"2018/05/01 12:31:15"
"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/07 00:00:00",1,MTPASAMRL01_NSW1,0,"2018/05/01 12:31:15"
"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/07 00:00:00",1,MTPASAMRL01_QLD1,0,"2018/05/01 12:31:15"
...

Raw Text with Metadata

C,SETP.WORLD,DVD_MTPASA_RESERVELIMITSOLUTION,AEMO,PUBLIC,2018/06/07,02:35:04,0000000295553422,,0000000295553313
I,MTPASA,RESERVELIMITSOLUTION,1,RUN_DATETIME,RUN_NO,RUNTYPE,ENERGYBLOCK,DAY,LDCBLOCK,RESERVELIMITID,MARGINALVALUE,LASTCHANGED
D,MTPASA,RESERVELIMITSOLUTION,1,"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_NSW1,0,"2018/05/01 12:31:15"
D,MTPASA,RESERVELIMITSOLUTION,1,"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_QLD1,0,"2018/05/01 12:31:15"
D,MTPASA,RESERVELIMITSOLUTION,1,"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_TAS1,0,"2018/05/01 12:31:15"
D,MTPASA,RESERVELIMITSOLUTION,1,"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_01,0,"2018/05/01 12:31:15"
D,MTPASA,RESERVELIMITSOLUTION,1,"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_02,0,"2018/05/01 12:31:15"
D,MTPASA,RESERVELIMITSOLUTION,1,"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_03,0,"2018/05/01 12:31:15"
D,MTPASA,RESERVELIMITSOLUTION,1,"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_04,0,"2018/05/01 12:31:15"
D,MTPASA,RESERVELIMITSOLUTION,1,"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_05,0,"2018/05/01 12:31:15"
D,MTPASA,RESERVELIMITSOLUTION,1,"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/06 00:00:00",1,MTPASAMRL01_VICSA_06,0,"2018/05/01 12:31:15"
D,MTPASA,RESERVELIMITSOLUTION,1,"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/07 00:00:00",1,MTPASAMRL01_NSW1,0,"2018/05/01 12:31:15"
D,MTPASA,RESERVELIMITSOLUTION,1,"2018/05/01 12:00:00",20,RELIABILITY_LRC,"2018/05/06 00:00:00","2018/05/07 00:00:00",1,MTPASAMRL01_QLD1,0,"2018/05/01 12:31:15"
...
C,"END OF REPORT",1003

Availability

Visibility: Public

This table is in the public monthly MMSDM archive.

I have collated the list of all those files into one place, in this CSV metadata file. (Columns year, month, url, compressed_size in bytes).

The earliest available data is month 1 of 2015. The most recent available data is month 5 of 2018.

Every month in between has a file. (No gaps.)

Data Size

Each year of data is approximately 508 KB when zipped, 5 MB as unzipped CSV.

Deduplication and PDR Loader Configuration

The primary keys are:

DAY
ENERGYBLOCK
LDCBLOCK
RESERVELIMITID
RUN_DATETIME
RUN_NO
RUNTYPE

PDR Loader configuration is:

Report Name	Report Subtype	Version	Undisclosed
MTPASA	RESERVELIMITSOLUTION	1	⚠️

Warning

Some of these file mappings are not in the published PDR Loader configuration file, even though files for this table exist with those metadata values. Mappings marked with ⚠️ have been added by me after manually inspecting those files.

This means that the CSV files start with metadata columns:

I,MTPASA,RESERVELIMITSOLUTION,1,...
D,MTPASA,RESERVELIMITSOLUTION,1,...

Links

This page as JSON (schema documented here, but may change)

Download a JSON Schema specification for this table.

Official AEMO schema documentation

Other Information

Package: HISTORICAL TABLES