IOSpec Format

The IOSpec contains:

  • Information about inputs and outputs

    • Their names (match the FQIR’s inputs and output names)

    • Their sizes (number of elements, before and after padding)

    • Low-level information like “mailbox id” for inputs

      • This is no longer required if using sequence-centric IO in SPU library (i.e. SpuWrite/ReadData())

    • For convenience, information like the quantization scale and the original (unpadded) data length is included as well

  • Information about “sequences”

    • Which inputs must be supplied before which each output is produced

    • Ordering information: the order of the sequences and their inputs and outputs

Sequences

The IOSpecs describe the “sequences” of input-to-output relationships in the program. For each SPU output, there is a set of inputs which must be provided before the output will be produced. For example, a network that simply performs A = B + C will have a single sequence that has output [A] and inputs [B, C].

The IOSpec for this simple model might look like:

inputs:
  B:                     # the input variable's name
    type: input
    varname: B           # the input variable's name (again)
    length: 60           # original size in the model
    padded_length: 64    # this is what is compiled/what FW handles
    length_64b_words: 16 # equivalent to padded_length, number of 64b words
    precision: 16        # 16b-element vector
    quantization:        # information about model quantization
      scale: 1.0
      zero_pt: 0.0
    core_id: 0           # what core this variable is received on
    pc: 0                # what pc value is associated with this variable
                         #   (mailbox_id is similar, for outputs)
    comments:            # misc information
      latched: false     # hint added by compiler,
                         #   whether this input is used in a "latched" sequence

  C:
    type: input
    varname: C
    # ... other information similar to A's

outputs:
  A:
    type: output
    varname: A
    # ... other information

simple_sequences:
  main_seq:
    # once B and C have been provided, A will be output
    type: simple_sequence
    outputs:
    - A
    inputs:
    - B
    - C

complex_sequences: {}

It is illegal to break the strict sequence order given by the IOSpec. For example, providing B twice before providing C will break the SPU’s internal control. Similarly, providing, B, then C, then B again before receiving A will also cause problems. In FW, the SPU library will emit errors (e.g. kSpuSequenceError) if the order is not respected. Similarly, the IOSpec’s rules will be checked when running the hardware or simulation using Femtodriver.

The order of the inputs and outputs within the sequence is important when using the SPU library. In the above, B comes before C in the main_seq. The SPU library’s SpuWriteData() requires the vectors to be written in the order in the associated sequence. Similarly, the order that data is returned from SpuReadData() will match the order in the sequence (e.g. if there were two outputs, X followed by Y, X will come out first).

Complex sequences are sequences with non-1:1 relationships between inputs and outputs, e.g. a complex sequence would be used if a particular output was only generated every other time an input was sent. This is not currently used by any model and is not yet supported by SPU library FW.

Multiple Sequences

With the exception of latched sequences (see below), multiple sequences are not yet supported.

The sequence_id provided to SpuWrite/ReadData() in SPU library is the order that the sequence occurs in the IOSpec file. For example, in the latched inputs example below, main_seq occurs first and would be sequence id 0, and latched_seq is sequence id 1.

Latched Inputs

Latched inputs are simply inputs that are part of a sequence with no associated output. Suppose we wanted to do the same math as before: A = B + latchedC, but we only wanted to update latchedC infrequently, producing A whenever B is provided (maybe latchedC is like a model tuning parameter, and we do not want to send a new value every frame).

When constructing the FQIR, the user will have specified that latchedC is a latched variable. The following IOSpec might be produced:

inputs:
  B:
    type: input
    varname: B
    # ... other information

  latchedC:
    type: input
    varname: C
    # ... other information
    comments:
      latched: true      # useful hint, but redundant with the sequence info

outputs:
  A:
    type: output
    varname: A
    # ... other information

simple_sequences:
  main_seq: # sequence id 0 in SPU library FW
    # when B (and only B) is provided, A will be output
    type: simple_sequence
    outputs:
      - A
    inputs:
      - B # no entry for latchedC!

  latched_seq: # sequence id 1 in SPU library FW
    type: simple_sequence
    # the latched input sequence for latchedC (no output)
    outputs: []
    inputs:
    - latchedC

complex_sequences: {}

With the above, it is perfectly valid to provide B only, without providing C, each time we drive inputs to the model. For example:

  • write latched_C = 1, write B = 1

  • read A = 2

  • write B = 2

  • read A = 3

  • write B = 3

  • read A = 4

  • write latched_C = 2, write B = 4

  • read A = 6

Is a valid ordering of transactions.

Latched inputs will be initialized to 0, override their values before starting the model, if necessary.