Troubleshooting#

“Have you tried turning it off and on again?” - The IT Crowd

Please refer to the table below for common issues and their potential solutions for Qblox Cluster. A list of what status the LED indicates can be found at Frontpanel LEDs. In case your problem is not listed or you are unable to resolve it, kindly contact us through the designated Slack support channel. If there is a problem accessing Slack, you could use support@qblox.com to reach us.

Starting Troubles#

Problem-Solution Reference#
Problem	Solutions
The status (S) LED is not green or white.	If the LED is red, it indicates an error that needs to be resolved. Query `get_system_state()` to see what the problem is. If the LED is orange, the LED indicates a critical warning, that an error has occurred but has been resolved. Query `get_system_state()` to see what the problem is. If the LED is yellow, the device is booting. If this state persists for more than 2 minutes, try power-cycling the device manually. At most two power cycles should be needed to boot the device.
The reference clock (R) LED is not green.	If the LED is red, no reference clock is found. Most likely you have selected the external input as reference source, but have not connected the reference. If the LED is blue, the internal reference clock is selected. Use the parameter `reference_source()` to change this if necessary.
The channel (I/O) LEDs are not green.	If the LED is red, an error has occurred in one or more of the connected sequencers. Query `get_sequencer_state()` to see what the problem is. If the LED is purple or blue, one or more of the connected sequencers are armed or running. Query `get_sequencer_state()` to see what the sequencers are doing. If the LED is orange, one or more of the connected sequencers are causing the output to clip. Query `get_sequencer_state()` to see what the sequencers are doing.
I cannot connect to the instrument.	Make sure that the Ethernet cables are firmly inserted into the instrument and host PC (see section Connecting to a Cluster). Make sure that the instrument and host PC are within the same subnet of the same network (see section Connecting to a Cluster). Make sure that you are using the IP address of the instrument. If you do not know the IP address or are not sure you are using the correct one, see section Finding the IP address of a module. If all else fails: 1) connect to the instrument with your computer directly, and disconnect everything else (WiFi etc. too), to make sure you’re really only connected to the offending device; 2) set your IP address to 192.168.0.200, with netmask 255.255.255.0 (see section Connecting to a Cluster); 3) run `qblox-pnp recover-device`; 4) the device should now reboot (if it doesn’t come back up within 1-2 minutes, remove and then reapply power); 5) you should now be able to connect to the instrument via its default IP address, namely 192.168.0.2.
When I connect to the instrument, I get an error that the driver and firmware are incompatible.	This indicates that the version of your installed Qblox instruments package and the firmware of the instrument you are trying to connect to is not compatible (see sections Setup and Updating). If all else fails, the instrument can be instantiated in debug mode where this error is bypassed. However, this is highly discouraged and no guarantees can be given that the instruments will function properly. Please see `Cluster`.
When I connect a reference clock to REF ⁱⁿ and configure the instrument to use an external reference clock, the reference clock LED (R) turns red.	Ensure that the reference clock source is outputting the clock as a 10 MHz, 1 Vpp signal. The REF ^out from another Qblox instrument can be used for this (see section Synchronization).
I cannot get the SYNQ to work.	Make sure that the SYNQ is enabled by setting the `Sequencer.sync_en()` parameter. Make sure that the sequencers are used to play waveforms and start acquisitions and that these sequencers are running the wait_sync instruction (see section Q1 Sequence Processor). When using multiple clusters: Make sure that you are using a USB-C compliant cable to connect the instruments using the SYNQ connectors (see section Mainframe & CMM). Make sure that the SYNQ cables are firmly inserted into the instrument. A distinct click should be felt when inserting them into the instruments.
I cannot get the trigger to work.	Make sure the trigger source is connected to the TRIG ⁱⁿ SMP connector (see section Mainframe & CMM). Make sure that the trigger source is outputting a trigger. This can be easily verified using an oscilloscope. Make sure that the sequencers are used to play waveforms and start acquisitions and that these sequencers are running the wait_trigger instruction (see section Instructions).
The output voltage of the instrument is twice as high as expected.	Make sure that you terminate the output signal with 50 Ω.
I updated the firmware or IP address using the Qblox Configuration Manager and the LEDs turned yellow.	This is normal behavior and indicates that the instrument has stopped its internal processes and is trying to reboot (see Updating). The process should be finished within 2 minutes.

Experiments and Measurements#

Issues	Troubleshooting Steps
No QCM-RF/QRM-RF Output on Scope (Or Gibberish Output)	Ensure LO is powered ON. Enable modulation (`mod_en_awg()` = True) and set NCO frequency > 10 MHz (since IF path is AC coupled to the upconverting mixer). Verify ON status of output RF switches (`marker_ovr_en()` or `set_mrk` Q1ASM commands). Refer to 6. Marker output channels for more info.
Cannot run Basic Sequencing/ Advanced Sequencing/ Acquisition / Multiplexed Sequencing Tutorial on RF modules	Please try the steps outlined in the previous question.
Binned Data Contains NaN	Ensure sufficient sequence wait time (`get_sequencer_state()` and `get_acquisition_state()`). Make sure you are retrieving acquisitions from the correct index key. Ensure that the Q1ASM program is populating all the bins as specified in the `num_bins` key of the acquisition dictionary.
Unexpected Q1ASM Results	Insert `nop` between consecutive commands on the same register. Make sure your classical/parameter instructions are updated using `upd_param` or `play` or `acquire` commands. Adhere to 4 ns grid for NCO commands. Maintain ≥ 8 ns gap between frequency updates.
Stuck Sequencers (Blue LEDs)	Verify proper wait_sync command placement. Review sync_en settings for unused sequencers.
Output Amplitude Mismatch on Scope	Verify impedance compatibility (module output impedance = 50 Ohms) with oscilloscope input impedance. Check your gain/offset values specified in the Q1ASM (if any) and make sure the values set are proportionally adjusted for the dynamic range of the module’s output DAC. `set_awg_offs 32767, 32767` will set a 2.5V DC offset on QCM (dynamic range 5Vpp) and 1V offset on QRM). Check your gain/attenuation settings. If modulation is ON, the output signal amplitude is smaller by a factor of \(\frac{1}{\sqrt{2}}\). Please check the NCO Control Tutorial for more info.
Oscillations on Frequency Sweep	Firmware ≤ 0.5.0: Oscillations of 6-7 MHz/oscillation may stem from fixed quantization noise during phase acquisition. We strongly recommend updating the firmware. If this is not possible, the oscillations can be reduced by enabling phase compensation with `nco_prop_delay_comp_en()`. Oscillations at NCO frequency with decaying oscillation amplitude: Likely due to input ADC offsets (which get demodulated and integrated by the hardware). Please refer to the Binned Acquisition tutorial (last section) to see how one can calibrate away these offsets. Oscillations can also be caused by other tones (e.g. multiplexed readout or input ADC offsets). If the phase of readout pulses is not fixed, the integrated magnitude of these spurious tones will depend on the frequency (as the phase of these tones at the start of integration depends on the frequency). This will contribute to an oscillating noise on top of the signal. You can use `set_ph` or `reset_ph` to eliminate this effect
DC Offset on Scope Acquisition	Due to thermal/manufacturing effects ADC may show a small DC offset on top of your sampled signals. Please refer to the Binned Acquisition tutorial (last section) to see how one can calibrate away these offsets. For RF modules, spurious tones at the frequency of oscillator (LO) can give rise to these offsets. You can use the calibration routine mentioned in the above point to get rid of these offsets.
`get_sequencer_state()` returns a FLAG	`DISARMED`: Sequencer was disarmed. `FORCED_STOP`: Sequencer was stopped while still running. `SEQUENCE_PROCESSOR_Q1_ILLEGAL_INSTRUCTION`: Classical sequencer part executed an unknown instruction. `SEQUENCE_PROCESSOR_RT_EXEC_ILLEGAL_INSTRUCTION`: Real-time sequencer part executed an unknown instruction. `SEQUENCE_PROCESSOR_RT_EXEC_COMMAND_UNDERFLOW`: Real-time sequencer part command queue underflow. `AWG_WAVE_PLAYBACK_INDEX_INVALID_PATH_0`: AWG path 0 tried to play an unknown waveform. Please ensure that the index referenced in your waveform dictionary for the sequence aligns with the one specified in your Q1ASM program. `AWG_WAVE_PLAYBACK_INDEX_INVALID_PATH_1`: AWG path 1 tried to play an unknown waveform. Please ensure that the index referenced in your waveform dictionary for the sequence aligns with the one specified in your Q1ASM program. `ACQ_WEIGHT_PLAYBACK_INDEX_INVALID_PATH_0`: Acquisition path 0 tried to play an unknown weight. `ACQ_WEIGHT_PLAYBACK_INDEX_INVALID_PATH_1`: Acquisition path 1 tried to play an unknown weight. `ACQ_SCOPE_DONE_PATH_0`: Scope acquisition for path 0 has finished. `ACQ_SCOPE_OUT_OF_RANGE_PATH_0`: Scope acquisition data for path 0 was out-of-range. `ACQ_SCOPE_OVERWRITTEN_PATH_0`: Scope acquisition data for path 0 was overwritten. `ACQ_SCOPE_DONE_PATH_1`: Scope acquisition for path 1 has finished. `ACQ_SCOPE_OUT_OF_RANGE_PATH_1`: Scope acquisition data for path 1 was out-of-range. Please make sure that the input values are within the dynamic range of the ADC. `ACQ_SCOPE_OVERWRITTEN_PATH_1`: Scope acquisition data for path 1 was overwritten. Please make sure that the input values are within the dynamic range of the ADC. `ACQ_BINNING_DONE`: Acquisition binning completed. `ACQ_BINNING_FIFO_ERROR`: Acquisition binning encountered internal FIFO error. This can be due to insufficient time between two consecutive acquisitions. `ACQ_BINNING_COMM_ERROR`: Acquisition binning encountered internal communication error. `ACQ_BINNING_OUT_OF_RANGE`: Acquisition binning data out-of-range. Please make sure that your average bin value * number of averages is less than \(2^{32}\). `ACQ_INDEX_INVALID`: Acquisition tried to process an invalid acquisition. Please double check that your index from the acquisition dictionary matches the one mentioned in your Q1ASM program. `ACQ_BIN_INDEX_INVALID`: Acquisition tried to process an invalid bin. Please double check that your index from the acquisition dictionary matches the one mentioned in your Q1ASM program. `TRIGGER_NETWORK_CONFLICT`: Trigger network has encountered a conflict. `TRIGGER_NETWORK_MISSED_INTERNAL_TRIGGER`: Trigger network missed an internal trigger. `OUTPUT_OVERFLOW`: Output overflow. Please make sure that the output range (from all the sequencer outputs combined) is within the dynamic range of your DAC. `CLOCK_INSTABILITY`: Clock source instability occurred. `ACQ_INTEGRATOR_OUT_OF_RANGE_PATH_0`: Acquisition integration input data for path 0 was out-of-range. `ACQ_INTEGRATOR_OUT_OF_RANGE_PATH_1`: Acquisition integration input data for path 1 was out-of-range.

FAQs#

Q1ASM#

Q: How do I set negative numbers to a register?

A. Registers within the sequence processor store unsigned integers. To represent signed integers (including negative numbers), you can use the concept of 2’s complement. In Q1ASM, you can achieve this by performing subtraction using the sub command. Here’s how:

Initialize a register to zero using: move 0, R0.

Use the sub command to subtract the desired value from the zero-initialized register. For example, to set -3 to a register R_dest, you can use: sub 3, R0, R_dest.

Q: What is the minimum delay for back-to-back acquisitions?

A. Acquisitions can be gapless.

Q: What is the maximum rate of acquisitions?

A. The maximum sustainable rate for back-to-back acquisitions depends on which sequencers are used. When using at least one, three or five sequencers, the maximum sustained rate is one acquisition on each sequencer per 300, 600 or 900 ns, respectively. Due to the built-in buffer, it is possible to exceed the maximum sustained rate for up to 7 acquisitions on each sequencer.

Q: How many averages can you do in a single sequence for a particular bin?

A. The sequence processor accumulates the measured value in a 32-bit register before dividing it by the number of repetitions.For binning it is \(2^{16}\). with inputs at maximum amplitude and maximum integration time and up to \(2^{32}\). with inputs at lower amplitudes or shorter integration times, but the number of averages then depends on the input signal strength and integration time. If the maximum is exceeded, the sequencer will raise the ACQ_BINNING_OUT_OF_RANGE flag.

Q: How many averages can you do for scope acquisition?

A. \(2^{32}\).

Q. What is the total propagation delay contributed by the input and output path of a module?

A. For a baseband module this is around 146 ns whereas for RF modules this is around 156 ns. One can accurately measure and compensate the total delay/time-of-flight during measurements using the steps outlined in the “TOF Calibration Section” of this tutorial.

Modules#

Q. How do we set a DC offset on the output of RF modules?

A. The RF Modules (QCM-RF and QRM-RF) can only output in the range of 1.5-18.5 GHz. Therefore, it is not possible to set a DC offset on the final output of RF modules.

Q. How do I calibrate the up-converting mixer to get rid of spurious tones at LO and sideband?

A. Please refer to the Mixer Calibration section in the RF Control Tutorial. You should be able to reach a spurious free dynamic range of upto 55dBc or more easily.

Q. The I/O LEDs are stuck at red/orange from a previous experiment. How do I clear them?

A. Uploading a new sequence to all 6 sequencers of a module clears any flags. Alternatively, you can do a reset() to reset the flags.