Skip to content

BIDPEROFFER

BIDPEROFFER shows period-based Energy and Ancillary Service bid data. BIDPEROFFER is a child table of BIDDAYOFFER.

Deprecation Warning

This table is in the HISTORICAL TABLES package, so is likely deprecated.

The new ancillary service arrangements require availability and prices for each Frequency Control Ancillary Service to be bid on a similar basis to energy. Three new tables facilitate ancillary service bidding. The new tables (BIDOFFERFILETRK, BIDDAYOFFER and BIDPEROFFER) are similar in structure to energy bidding tables (OFFERFILETRK, DAYOFFER and PEROFFER). The significant differences with the new tables are:

· The OFFERDATE field reflects the time the bid was loaded and this field alone provides the key for versioning of bids. The VERSIONNO field is retained for participant use as information only.

· The new tables support bids for multiple services. The BIDTYPE field defines the service to which the bid applies.

· There are no default bids. In the absence of a bid for a specific settlement date, the latest bid submitted for a previous settlement date applies.

BIDPEROFFER data is confidential to the submitting participant until made public after 4am the next day.

Source

BIDPEROFFER updates as energy and ancillary service bids are processed. BIDPEROFFER includes all accepted energy and ancillary service bids.

Volume

Approximately 72,000,000 records per year

Column Schema

BIDTYPE

Bid Type Identifier

The SQL type is VARCHAR2(10). This is actually a factor/enum, with values:

  • ENERGY

  • LOWERREG

  • LOWER60SEC

  • RAISE5MIN

  • LOWER5MIN

  • LOWER6SEC

  • RAISE60SEC

  • RAISE6SEC

  • RAISEREG

  • RAISE1SEC

  • LOWER1SEC

This column is mandatory, and a primary key.

DUID

Dispatchable Unit identifier

Warning

This column may contain funny characters such as / (slash) or # (hash). This may break hive-style partitioning.

For more information, see the dedicated documentation page.

SQL Type: VARCHAR2(10) Polars Type: pl.Categorical

This column is mandatory, and a primary key.

Examples:

  • AGLHAL
  • W/HOE#2

OFFERDATE

Offer date

Warning

Despite what the name suggests, this is a datetime, not a date.

SQL Type: DATE Polars Type: pl.Datetime()

This column is mandatory, and a primary key.

Example: 2021/01/21 09:57:06

PERIODID

Period ID

SQL Type: NUMBER(22,0) Polars Type: pl.Int128

This column is mandatory, and a primary key.

Examples: 1, 48

SETTLEMENTDATE

Market date starting at 04:05

SQL Type: DATE Polars Type: pl.Date()

This column is mandatory, and a primary key.

Example: 2021/02/02 00:00:00

Note that AEMO represents dates as datetimes with a midnight time component.

BANDAVAIL1

Availability at price band 1

SQL Type: NUMBER(22,0) Polars Type: pl.Int128

This column is not mandatory.

Examples: 0, 29, 32

BANDAVAIL2

Availability at price band 2

SQL Type: NUMBER(22,0) Polars Type: pl.Int128

This column is not mandatory.

Example: 0

BANDAVAIL3

Availability at price band 3

SQL Type: NUMBER(22,0) Polars Type: pl.Int128

This column is not mandatory.

Example: 0

BANDAVAIL4

Availability at price band 4

SQL Type: NUMBER(22,0) Polars Type: pl.Int128

This column is not mandatory.

Example: 0

BANDAVAIL5

Availability at price band 5

SQL Type: NUMBER(22,0) Polars Type: pl.Int128

This column is not mandatory.

Example: 0

BANDAVAIL6

Availability at price band 6

SQL Type: NUMBER(22,0) Polars Type: pl.Int128

This column is not mandatory.

Example: 0

BANDAVAIL7

Availability at price band 7

SQL Type: NUMBER(22,0) Polars Type: pl.Int128

This column is not mandatory.

Examples: 29, 60

BANDAVAIL8

Availability at price band 8

SQL Type: NUMBER(22,0) Polars Type: pl.Int128

This column is not mandatory.

Example: 0

BANDAVAIL9

Availability at price band 9

SQL Type: NUMBER(22,0) Polars Type: pl.Int128

This column is not mandatory.

Example: 0

BANDAVAIL10

Availability at price band 10

SQL Type: NUMBER(22,0) Polars Type: pl.Int128

This column is not mandatory.

Examples: 163, 195

ENABLEMENTMAX

Maximum Energy Output (MW) at which this ancillary service can be supplied (AS Only)

SQL Type: NUMBER(6,0) Polars Type: pl.Int32

This column is not mandatory.

Example: 0

ENABLEMENTMIN

Minimum Energy Output (MW) at which this ancillary service becomes available (AS Only)

SQL Type: NUMBER(6,0) Polars Type: pl.Int32

This column is not mandatory.

Example: 0

FIXEDLOAD

Fixed unit output MW (Energy Bids Only) A value of zero means no fixed load so the unit is dispatched according to bid and market (rather than zero fixed load)

SQL Type: NUMBER(12,6) Polars Type: pl.Float64

This column is not mandatory.

Example: 0

HIGHBREAKPOINT

Maximum Energy Output (MW) at which the unit can provide the full availability (MAXAVAIL) for this ancillary service (AS Only)

SQL Type: NUMBER(6,0) Polars Type: pl.Int32

This column is not mandatory.

Example: 0

LASTCHANGED

Last date and time record changed

For more information, see the dedicated documentation page.

SQL Type: DATE Polars Type: pl.Datetime()

This column is not mandatory.

Example: 2021/01/21 09:57:06

LOWBREAKPOINT

Minimum Energy Output (MW) at which the unit can provide the full availability (MAXAVAIL) for this ancillary service (AS Only)

SQL Type: NUMBER(6,0) Polars Type: pl.Int32

This column is not mandatory.

Example: 0

MAXAVAIL

Maximum availability for this BidType in this period

SQL Type: NUMBER(12,6) Polars Type: pl.Float64

This column is not mandatory.

Examples: 174, 205

MR_CAPACITY

Mandatory Restriction Offer amount

SQL Type: NUMBER(6,0) Polars Type: pl.Int32

This column is not mandatory.

PASAAVAILABILITY

Allows for future use for energy bids, being the physical plant capability including any capability potentially available within 24 hours

SQL Type: NUMBER(12,0) Polars Type: pl.Int64

This column is not mandatory.

Example: 205

ROCDOWN

MW/Min for lower (Energy Bids Only)

SQL Type: NUMBER(6,0) Polars Type: pl.Int32

This column is not mandatory.

Example: 12

ROCUP

MW/min for raise (Energy Bids Only)

SQL Type: NUMBER(6,0) Polars Type: pl.Int32

This column is not mandatory.

Example: 12

VERSIONNO

Version number of offer

SQL Type: NUMBER(22,0) Polars Type: pl.UInt128

This column is not mandatory.

Examples: 1, 10

New Data Timing

When energy and ancillary service bids are processed

Code Examples

Polars

This table is so large that you should explicitly enumerate and cast every column, which will speed things up and prevents some errors. Although some of the datatypes are inferred, so they may be wrong. In particular, UInt16 vs UInt8 vs Int16 etc. If you get errors, you can swap these for just int.

This code deduplicates for you. Deduplication is normally only required across multiple months.

This code assumes that all columns are present, which may not be true for older files.

import polars as pl
import polars.selectors as cs

# e.g. "2029/03/15 00:05:00"
dt_s_fmt = "%Y/%m/%d %H:%M:%S"

# Dates are represented as datetimes at midnight

d_fmt = "%Y/%m/%d 00:00:00"

csv_paths = ["SOMEFILE.CSV", "OTHERFILE.CSV"]

lf = (
    pl.concat(
        [
            pl.scan_csv(
                csv_path,  # must be unzipped beforehand
                skip_rows=1,  # skip the header
                infer_schema=False,  # much faster
            )
            .filter(cs.by_index(0) == "D")  # drop the footer
            .select(~cs.by_index(range(4)))  # drop metadata columns
            for csv_path in csv_paths
        ],
        how="diagonal_relaxed",
    )
    .with_columns(
        pl.col("OFFERDATE", "LASTCHANGED").str.strptime(pl.Datetime, dt_s_fmt),
        pl.col("SETTLEMENTDATE").str.strptime(pl.Date, d_fmt),
    )
    .cast(
        {
            "BIDTYPE": pl.Enum(
                [
                    "ENERGY",
                    "LOWERREG",
                    "LOWER60SEC",
                    "RAISE5MIN",
                    "LOWER5MIN",
                    "LOWER6SEC",
                    "RAISE60SEC",
                    "RAISE6SEC",
                    "RAISEREG",
                    "RAISE1SEC",
                    "LOWER1SEC",
                ]
            ),
            "DUID": pl.Categorical,
            "PERIODID": pl.Int128,
            "SETTLEMENTDATE": pl.Date(),
            "BANDAVAIL1": pl.Int128,
            "BANDAVAIL2": pl.Int128,
            "BANDAVAIL3": pl.Int128,
            "BANDAVAIL4": pl.Int128,
            "BANDAVAIL5": pl.Int128,
            "BANDAVAIL6": pl.Int128,
            "BANDAVAIL7": pl.Int128,
            "BANDAVAIL8": pl.Int128,
            "BANDAVAIL9": pl.Int128,
            "BANDAVAIL10": pl.Int128,
            "ENABLEMENTMAX": pl.Int32,
            "ENABLEMENTMIN": pl.Int32,
            "FIXEDLOAD": pl.Float64,
            "HIGHBREAKPOINT": pl.Int32,
            "LOWBREAKPOINT": pl.Int32,
            "MAXAVAIL": pl.Float64,
            "MR_CAPACITY": pl.Int32,
            "PASAAVAILABILITY": pl.Int64,
            "ROCDOWN": pl.Int32,
            "ROCUP": pl.Int32,
            "VERSIONNO": pl.UInt128,
        }
    )
    # Deduplicate
    # These files are so large that this step may cause your laptop to run out of memory.
    # We want one row per ['BIDTYPE', 'DUID', 'OFFERDATE', 'PERIODID', 'SETTLEMENTDATE']
    # Tie-breaker: greatest value of ['LASTCHANGED']
    .sort(["LASTCHANGED"], descending=False)
    .unique(
        subset=["BIDTYPE", "DUID", "OFFERDATE", "PERIODID", "SETTLEMENTDATE"],
        keep="last",
    )
)

Nemosis

This table is not yet available in Nemosis.

Example File

From PUBLIC_DVD_BIDPEROFFER_202102010000.zip

DUIDBIDTYPESETTLEMENTDATEOFFERDATEPERIODIDVERSIONNOMAXAVAILFIXEDLOADROCUPROCDOWNENABLEMENTMINENABLEMENTMAXLOWBREAKPOINTHIGHBREAKPOINTBANDAVAIL1BANDAVAIL2BANDAVAIL3BANDAVAIL4BANDAVAIL5BANDAVAIL6BANDAVAIL7BANDAVAIL8BANDAVAIL9BANDAVAIL10LASTCHANGEDPASAAVAILABILITYMR_CAPACITY
AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:061120501212000000000060001952021/01/21 09:57:06205
AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:062120501212000000000060001952021/01/21 09:57:06205
AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:063120501212000000000060001952021/01/21 09:57:06205
AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:064120501212000000000060001952021/01/21 09:57:06205
AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:065120501212000000000060001952021/01/21 09:57:06205
AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:066120501212000000000060001952021/01/21 09:57:06205
AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:067120501212000000000060001952021/01/21 09:57:06205
AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:068120501212000000000060001952021/01/21 09:57:06205
AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:069120501212000000000060001952021/01/21 09:57:06205
AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:0610120501212000000000060001952021/01/21 09:57:06205
AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:0611120501212000000000060001952021/01/21 09:57:06205
...

CSETP.WORLDDVD_BIDPEROFFERAEMOPUBLIC2021/03/0615:10:0500000003377733280000000337773314
IOFFERBIDPEROFFER1DUIDBIDTYPESETTLEMENTDATEOFFERDATEPERIODIDVERSIONNOMAXAVAILFIXEDLOADROCUPROCDOWNENABLEMENTMINENABLEMENTMAXLOWBREAKPOINTHIGHBREAKPOINTBANDAVAIL1BANDAVAIL2BANDAVAIL3BANDAVAIL4BANDAVAIL5BANDAVAIL6BANDAVAIL7BANDAVAIL8BANDAVAIL9BANDAVAIL10LASTCHANGEDPASAAVAILABILITYMR_CAPACITY
DOFFERBIDPEROFFER1AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:061120501212000000000060001952021/01/21 09:57:06205
DOFFERBIDPEROFFER1AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:062120501212000000000060001952021/01/21 09:57:06205
DOFFERBIDPEROFFER1AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:063120501212000000000060001952021/01/21 09:57:06205
DOFFERBIDPEROFFER1AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:064120501212000000000060001952021/01/21 09:57:06205
DOFFERBIDPEROFFER1AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:065120501212000000000060001952021/01/21 09:57:06205
DOFFERBIDPEROFFER1AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:066120501212000000000060001952021/01/21 09:57:06205
DOFFERBIDPEROFFER1AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:067120501212000000000060001952021/01/21 09:57:06205
DOFFERBIDPEROFFER1AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:068120501212000000000060001952021/01/21 09:57:06205
DOFFERBIDPEROFFER1AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:069120501212000000000060001952021/01/21 09:57:06205
DOFFERBIDPEROFFER1AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:0610120501212000000000060001952021/01/21 09:57:06205
DOFFERBIDPEROFFER1AGLHALENERGY2021/02/02 00:00:002021/01/21 09:57:0611120501212000000000060001952021/01/21 09:57:06205
...
CEND OF REPORT1003

DUID,BIDTYPE,SETTLEMENTDATE,OFFERDATE,PERIODID,VERSIONNO,MAXAVAIL,FIXEDLOAD,ROCUP,ROCDOWN,ENABLEMENTMIN,ENABLEMENTMAX,LOWBREAKPOINT,HIGHBREAKPOINT,BANDAVAIL1,BANDAVAIL2,BANDAVAIL3,BANDAVAIL4,BANDAVAIL5,BANDAVAIL6,BANDAVAIL7,BANDAVAIL8,BANDAVAIL9,BANDAVAIL10,LASTCHANGED,PASAAVAILABILITY,MR_CAPACITY
AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",1,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",2,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",3,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",4,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",5,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",6,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",7,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",8,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",9,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",10,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",11,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
...

C,SETP.WORLD,DVD_BIDPEROFFER,AEMO,PUBLIC,2021/03/06,15:10:05,0000000337773328,,0000000337773314
I,OFFER,BIDPEROFFER,1,DUID,BIDTYPE,SETTLEMENTDATE,OFFERDATE,PERIODID,VERSIONNO,MAXAVAIL,FIXEDLOAD,ROCUP,ROCDOWN,ENABLEMENTMIN,ENABLEMENTMAX,LOWBREAKPOINT,HIGHBREAKPOINT,BANDAVAIL1,BANDAVAIL2,BANDAVAIL3,BANDAVAIL4,BANDAVAIL5,BANDAVAIL6,BANDAVAIL7,BANDAVAIL8,BANDAVAIL9,BANDAVAIL10,LASTCHANGED,PASAAVAILABILITY,MR_CAPACITY
D,OFFER,BIDPEROFFER,1,AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",1,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
D,OFFER,BIDPEROFFER,1,AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",2,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
D,OFFER,BIDPEROFFER,1,AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",3,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
D,OFFER,BIDPEROFFER,1,AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",4,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
D,OFFER,BIDPEROFFER,1,AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",5,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
D,OFFER,BIDPEROFFER,1,AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",6,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
D,OFFER,BIDPEROFFER,1,AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",7,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
D,OFFER,BIDPEROFFER,1,AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",8,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
D,OFFER,BIDPEROFFER,1,AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",9,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
D,OFFER,BIDPEROFFER,1,AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",10,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
D,OFFER,BIDPEROFFER,1,AGLHAL,ENERGY,"2021/02/02 00:00:00","2021/01/21 09:57:06",11,1,205,0,12,12,0,0,0,0,0,0,0,0,0,0,60,0,0,195,"2021/01/21 09:57:06",205,
...
C,"END OF REPORT",1003

Availability

Visibility: Private & Public Next-Day

This table is in the public monthly MMSDM archive.

I have collated the list of all those files into one place, in this CSV metadata file. (Columns year, month, url, compressed_size in bytes).

The earliest available data is month 1 of 2015. The most recent available data is month 2 of 2021.

Warning

There are some months with no files.

Warning

The filenames of zips may not correspond to the tables. These can be mixed up with BIDOFFERPERIOD. Check column 2 and 3 inside the CSV against REPORT_NAME and REPORT_SUBTYPE to be sure. I have already done this for you, and listed all URLs in this metadata file.

Data Size

Each year of data is approximately 1 GB when zipped, 11 GB as unzipped CSV.

Tip

This table is large. Simply calling pd.read_csv() might fill up all the memory on your laptop. (As a rule of thumb, you need double the amount of memory as the size of the CSV.) You will need to think hard about memory optimisation. e.g. consider using Polars instead of Pandas, and selecting only the columns you care about.

Deduplication and PDR Loader Configuration

The primary keys are:

Tip

Bid deduplication is particularly tricky. See the guide.

The transaction type is "insert", which means that you will usually not see duplicate data. If you do, you can deduplicate based on the primary keys, without sorting first. Rows from different files with the same primary keys should have the same values.

PDR Loader configuration is:

Report Name Report Subtype Version
OFFER BIDPEROFFER 1

This means that the CSV files start with metadata columns:

I,OFFER,BIDPEROFFER,1,...
D,OFFER,BIDPEROFFER,1,...

Indexes

This page as JSON (schema documented here, but may change)

Download a JSON Schema specification for this table.

Official AEMO schema documentation

Other Information

Package: HISTORICAL TABLES