Data documentation

Argus provides the following subscriber documentation files in CSV format:

• latestCodes.csv: A list of all commodities Argus publishes or has published with their properties such as publishing frequency, timing information, measurement units, etc. The code is a basic assessment descriptor and it does not include any information about types of datapoints assessed for a commodity, or when those assessments are made, etc.
• latestModules.csv: The list of modules Argus publishes with their descriptions, short names and locations.
• latestPricetype.csv: A list of descriptors that clarifies the type of value published, for example High, Low, Average etc. Also known as PriceType or PT.
• latestTimestamp.csv: List of time periods Argus publishes its assessments. Also known as TimeStamp or TS.
• latestQuotes.csv: A list of all Code-TS-PT-CFWD(CFWD = continuous forward - explained below) combinations Argus ever assessed with their start and end dates and other parameters. In conjunction with date that combination represents a unique datapoint key.
• latestModuleDetails.csv: describes the content of each module in a form of Code-TS-PT-CFWD combination (note, that it is possible to have daily and weekly, weekly and monthly, etc published data within one module, so not every csv file contains all assessments only when they are due).
• latestCategory.csv: A list of all codes mapped to their specific categories, displayed in a tree format.
• latestDoc.csv: A combined list of all the documentation above in a decoded format.
• latestUnits.csv: A list of units of measurements and currencies
• latestRVP_Code_reference.csv: This file is specific to Argus US Products gasoline and represents the mapping between the price series and RVP series.

The relationships between tables are shown here

The file latestDoc.csv contains details of the folder and filename you will find on the FTP site. As an example, you can find the Coal API2 index (see 2nd row of the file below) in a folder called "DAMCOAL" (see column C) and in a file whose name contains "dcm" (see column D and the naming convention outlined below). All related data that is published together is contained in a single file.

The file above also indicates the frequency of publication of each piece of data (see column V).

The file latestModules.csv contains details of the time at which each file is published. Please note that all file timestamps are in UTC.

The naming convention for the files is as follows (where FFF is replaced by the Filename in column D of the file latestDoc.csv):

In the screenshot below you will see an example of the DAMCOAL folder after publication on 9th September 2013. This FTP instance is hosted on a server that runs EST time zone so the timestamps are 5 hours behind UK time. Argus hosts FTP servers in different time zones. To avoid ambiguity we recommend to use MDTM FTP command as described in RFC-3659.

The DAMCOAL folder contains two files, the "dcm" file contains daily and weekly published prices (e.g. API2) and the "dcm2" file contains weekly and monthly prices so it is updated once a week, in this case on a Friday.

DATA CSV file format

The CSV data files contain considerably more information than only name, date, value and a timestamp for each assessment. The extra information relates to the timing of the assessment and the basis from which it is priced. For example, Argus North Sea Dated is calculated from a forward month North Sea the csv file will include data to say which month it is priced from. Some series are both forward prices and based on another series and the csv file will have all this timing information.

In addition (using the same example), all North Sea months (currently there are four Argus forward North Sea months) are given the same code, so the csv file has a "continuous forward" field. The continuous forward value holds the sequence number of each forward period. For example, Argus posts North Sea month 1, North Sea month 2, North Sea month 3 and North Sea month 4 prices the forward period might be 10,11,12,1 (Oct, Nov, Dec, Jan) but the continuous forward periods will always be 1,2,3,4 for 1st, 2nd, 3rd and 4th month.

Used in conjunction with the date, the forward period and continuous forward fields allow for databases to extract data by effective month, ie August or by First month depending on what the user requires. The Year allows these fields to be used to produce an August 2003 query.

A final field indicates whether a data record is new or a correction. This allows Argus to place corrections into a regular daily data file along side the new data. This allows user to access new data and corrections in one download, which should ensure that user databases are kept as up to date as the Argus master database.

Example of CSV file:

Note: the first line contains the field headings.

Code - column 1

A character string field. This is the identifier of the commodity. The identifier links to the codes description table giving information such as commodity, units, etc.

Time Stamp Type - column 2

A numerical integer field. TS (Time Stamps) Type is the time at which the assessment is fixed, the value is an identifier which is assigned to a specific time. A Code can have multiple time stamps, for example Argus produces North Sea month crude assessments at London midday and London close (Please refer to the relevant methodology for exact time values). By using one code and multiple time stamp modifiers all the data (in this example North Sea Dated) is kept together. Download the list of Time Stamps

Price Type Code - column 3

A numerical integer field. PT (Price Type), is an identifier that describes the type of price that the assessment is. A Code can have multiple Price Types, for example Argus produces North Sea Dated crude assessments for Bid, Ask and Midpoint. By using one code and multiple price type modifiers all the data (in this example North Sea Dated) is kept together. Download the list of Price Types

Date - column 4

A date field in DD-MMM-YYYY format. A self-explanatory field - the assessment date.

Value - column 5

A non-integer numerical field. This is the assessment value, again a self-explanatory field. The code description table describes the units of this value.

Forward Period - column 6

A numerical integer field. The Forward (Fwd) Period, (sometimes referred to as delivery period) describes the period of time the assessment applies to. The value of this field can represent years, months, weeks, days depending on the definitions in the latestQuotes.csv file.

Diff Base Roll - column 7

A numerical integer field. The Differential (Diff) Base Roll is primarily a field used by crude assessments (but not exclusively) that are assessed relative to another crude (or commodity if not a crude). Argus North Sea Dated is a crude assessment that is assessed relative to a forward month of North Sea forward months series. To be able to keep track of the basis month the Diff Base Roll stores the month value. Diff Base Roll are month values in range 1 to 12.

Year - column 8

This is the effective year of the assessment. If the series is a prompt or dated price then the year is the same as the year of the assessment date in column 4. If the series is for a forward delivery or future price, or even, previous time period, then this year value is for the actual year of the forward period. In the sample file above there are 10 rows of data for code PA0002731, which is Natural gas Bacton forward months. This represents 2004 at the moment because the prices are for Feb 2004 through to June 2004. Around August, the data might look like:

or tabulated for better representation: 

For the last two rows, the year has rolled around to 2005. This is because the Forward Period of this price is now 1 (January) for this code, which takes the Forward Period into the next year (2005).

Cont forward - column 9

Closely related to the Forward Period and year is the cont (continuous) period. Continuing the example from above, the code PA0002731 has 10 rows of data. There are five pairs of data. The differencies between each pair are the price types, values, forward periods and continuous periods. The Price Types indicate that these are bid/ask pairs. The Forward Periods indicate that these are for months September through to January. The continuous periods show that these are month 1, month 2, month 3, month 4 and month 5 forward prices.

Status - column 10

The status flag indicates if the price is a new data point or a corrected data point. The values are either 'N' for new, 'C' for correction, 'H' for historically corrected data (see below) or 'D' for deleted price record. Corrections sometimes provide data which was omitted previously. This flag highlights that the data is extra to the normal daily data because of a previous problem with that particular record. The flag can be used to locate an existing record to overwrite/delete data based on Code-TS Type-PT Code-Cont fwd-Date key. If you are processing multiple csv files at the same time, the files must be processed in chronological order.

Argus continually cleans and audits its time series data and corrects discrepancies when they are discovered. If the data is “old” (i.e. after 6 months) then no new correction would normally be issued, unless the number was a significant index such as “North Sea Dated” on which a large number of other prices are based. Argus may issue corrections for “old” data at its discretion.