Skip to content

Tables

The Tables

AEMO's dataset contains many different 'tables'. They are called this because AEMO expects market participants to load the data into tables in a SQL database. (If you are an analyst querying CSV/parquet files with Pandas, these may each be a different dataframe. The concept is the same.) There are hundreds of tables available to market participants. Some of those are not available to the public (e.g. settlement data), but most are. The most important ones are:

  • DUDETAILSUMMARY contains some static data about each generator (e.g. which region they are in)
  • DISPATCHPRICE - for energy and ancillary prices
  • P5MIN_REGIONSOLUTION - for short term price and energy forecasts
  • DISPATCH_UNIT_SCADA - per-generator power output at the start of each 5 minute interval
  • DISPATCHLOAD - despite the name, this is for both generators and loads. This is per-generator power (both actual, and what they were supposed to do), ancillary service dispatch (what they were supposed to be able to provide if called upon, but not whether they were actually called upon)
  • DISPATCHREGIONSUM - This contains region-level power, both actual and planned. This is for both generated, consumed and imported/exported power, as well as ancillary services. A few of the columns about total renewable output/capacity are always empty, unfortunately.
  • BIDDAYOFFER, BIDOFFERPERIOD, BIDPEROFFER_D, BIDDAYOFFER_D, BIDPEROFFER: Raw bids by generators. These are very large, up to 2 TB uncompressed if you download 10 years of data. They are also very complex to understand. A dedicated section explains them further down.
  • There is 4 second granularity power data for every generator, as well as some other data (such as grid frequency). This is exceptionally large, and in a different location and format to the other data.

Some other important tables are listed in the Nemosis wiki.

There are conceptually related groups of tables.

  • DISPATCH* refers to the actual decisions from NEMDE's market clearing. This includes the actual price (could be energy or FCAS), power targets from AEMO ("cleared") per generator, or per region, interconnector flows between regions etc.
  • PREDISPATCH* and P5* refer to AEMO's predictions for the near term. (Each revision of these predictions is stored in perpetuity, so you can see how predictions change over time for a given interval.)
  • ANCILLARY* refers to FCAS.
  • PASA, STPASA, MTPASA refer to medium term (multi-day) to long term (multi-month) predictions
  • SETTLEMENT* refers to invoicing data, which is typically private
  • GBB is about gas
  • CONF* is about configuration for PDR Batcher and Loader
  • FORCE_MAJEURE and "market suspension" are about black swan events (e.g. a few times per decade)
  • PRUDENTIALS refers to a concept about the credit-worthiness of market participants. (e.g. retailers may need to prepay for their customers' consumption during an exceptionally high-priced week)