femtodrive CLI

Femtodriver provides a Command Line Interface (CLI) called femtodrive. To show all options:

femtodrive --help

This document will cover the most common use cases for femtodriver but the full list of capabilities is listed in the help.

Generate program files from an FQIR pickle

You can generate SD programming files from a previously saved FQIR pickle. You can pickle femtocrux’s input, the FQIR graph, with torch.save() (In the PyTorch femtocrux walkthroughs, this variable is called fqir_graph).

This way, femtodrive can use femtocrux to compile the model and emit program binaries, as is done directly in the notebooks.

Example:

femtodrive my_model.pt

As a “hello world” you can invoke:

femtodrive LOOPBACK

This will call femtodrive on an “identity” network that is installed with the package. As before, output will be put in model_datas/<stem of pickle filename>/. Notice the images.zip that appears and was unpacked to docker_data/.

(Pickles are notoriously unportable. Ideally, any pickling/unpickling is done on one machine, but failing that, try to ensure the pickle is unpacked using the same package versions it was generated with)

Generate program files from a memory image zip

You can generate SD programming files from a previously generated Femtocrux memory image zip:

femtodrive <path-to-zipfile_from_femtocrux> # general example
femtodrive bitfile.zip # specific example

This will create model_datas/<stem of zipfile path>/. Inside, along with other information, there will be a io_records/apb_records. This holds the 0PROG_A and 0PROG_D files which can be downloaded to the SD card. Note that future firmware might allow multiple models to coexist on the SD card. The leading ‘0’ indicates that this is the first model. In some cases, with multiple models loaded, you may need to edit the number in the filename.

Historical note: this replaces sd_from_femtocrux.py in femtodriverpub.

Simulation With Femtodriver via Femtocrux

When the FQIR pickle is supplied, it is also possible to simulate the model using femtodriver. In this case, pass “fasmir” to the --runners argument.

femtodrive ../models/my_model.pt --runners="fasmir"

See femtodrive --help for the options related to passing inputs and retreiving outputs.

Run Audio Through Model

You can also run audio through a model using the --input-file flag.

femtodrive my_model.pt --runners="fasmir" --input-file example_audio.wav --input-period 0.016

This will reshape the audio into the correct dimensions of (frames, features/samples) to run through the model for simulation. The --input-period is the input period time to process a single frame of audio. The more time, the lower the power consumption but there is tradeoff with latency.

Comparing Runners

Femtodriver allows you to pass inputs through “runners” to get outputs. We have the following representations of the model:

  • FQIR (quantized torch-like)

  • FASMIR (high-level assembly)

  • femtofiles (binaries) are increasingly-low level representations of the program

They all specify exactly the same arithmetic. When the model is compiled the execution stays exactly the same. Each program representation can have one or more “runners” associated with it. The runner could be a software simulation or it could be drivers controlling real SPU hardware. We can construct one or more runners that execute a single program, executing each of its representations.

This is useful in a number of ways:

  • By comparing the output of each runner, for the same inputs, we verify that the model will perform on hardware as the ML developer expects

  • By running simulation of the hardware (FASMIR’s runner), we can extract power and resource estimates

  • For models where only binaries are available, we can stream data through the model using a PC

  • We can use the hardware runner to get power and latency estimates

You can compare the hw runner with the fqir or fasmir runners to ensure the software simulator outputs match the outputs on the hardware.

One important thing to note is that all the runners are only available when you start from FQIR files. If you start from femtofiles, you can only use the hw runner as we are already at a lower level in the process.

From the command line you can do:

# Compare simulated hardware runner with simulated fasmir
femtodrive my_model.pt --hardware=fakezynq --runners=hw,fasmir
# Compare evk2 SPU-001 hardware runner with fqir runner
femtodriver my_model.pt --hardware=evk2 --runners=hw,fqir

Cleaning up Zombie Docker Containers

If for some reason you exit the an invocation of femtodrive with a ctrl-c and femtodrive cannot clean up a docker container that it started, you can always clean up all the femtocrux docker containers with:

femtodrive --cleanup-docker