X6-400M Clocking Issues for OCT Imaging Applications

[potsaid]

We have been using the previous generation X5-400M for OCT imaging applications using several different laser sources to externally clock the X5-400M. In several years of testing, we have not seen any glitches with the X5-400M.

We are noticing that the X6-400M behaves differently than the X5-400M. Using the X6-400M, we are not able to complete acquisitions when the card is externally clocked by the laser source.

It looks like there is a small band of input voltage amplitudes that cause the X6-400M to generate an "Input FIFO overrun 0x13030001" error. It is very repeatable. If we clock with fixed amplitude and fixed frequency sine waves of particular voltages peak-to-peak, the card generates the error and halts acquisition. See the attached pdf on slides 13-16 for details.

Above the troublesome voltage band, the card clocks OK. Below the voltage band, the card does nothing (as expected because clock signal is too small).

We think that the OCT clock signal transitions through that small band of input voltages as the clock starts and stops, causing the error.

We are curious if you have any ideas on why this might be occurring (maybe the voltage level brings the clocking circuit to a threshold where noise or internal circuit oscillation is interpreted as clock signal and overclocks the A/D), and also if you have any ideas on how to prevent the error.

Thanks again,

- Ben

The attached pdf provides an overview of the clock source characteristics, a comparison of X5 vs. X6, our testing data and testing results. Slide 16 maps the X6-400M clocking behavior.

[dlewis]

Hello Ben,

Thank you for your post.

We would like to better understand this. I will be looking into this, and will be trying some experiments and comparisons.

My first thought was the clocking circuit input is diffferent for the X5-400M and the X6-400M. The X5-400M has a true 50 Ohm DC termination to ground prior to AC coupling, while the X6-400M is AC coupled to 50 Ohms. If it is not trouble, could you please provide information on the output of the interferometer that connects to the clock input?



Regards

Dave

[potsaid]

Dave,

That is a very interesting thought about the 50 Ohm termination.

With that in mind, we did a bit more testing. We used several different sources (see attached pdf):

* Commercial Swept Laser Source (Slides 3-5)

* Agilent N5181A 3GHz analog signal generator (6-9)

* Prototype Thorlabs/MIT circuit (Slides 11-13)

The drive circuit for the Commercial Swept Laser Source and the Agilent analog signal generator are not known. But, the trace as captured from a Tektronix scope with 50 Ohm termination looks clean as shown in the attached slides.

The drive circuit for the Thorlabs/MIT circuit consists of an op-amp followed by a 47 Ohm resistor to the SMA cable connection.

We use 50 Ohm SMA cables for all connections.

The other thing we tried was to overclock the X6-400M in order to see if we could generate that "Input FIFO overrun error (see slide 10). We overclocked to 500MHz and were not able to generate the error. This means that if indeed the FIFO error is from overclocking due to noise or internal circuit oscillation, it is of higher frequency than 500MHz.

Thanks for looking into this. We are ready to test or try things that you might suggest.

- Ben

[potsaid]

Hi.

We wanted to try one more external clocking experiment before sending the board back for the trigger resistor swap.

A 200MHz low pass filter was inserted between the commercial swept laser clock output and the X6-400M clock input (see attached pdf for details).

When the source is directly connected to the X6-400M, we get a "Input FIFO overrun..." error and the acquisition halts.

When the 200MHz low pass filter is used, the X6-400M produces a different error message of "Input overrange...". The message continuously scrolls during the acquisition. The input overrange was unexpected because we did not have anything connected to the AD0 channel and we don't get the error message when internal clocking.

We are not sure exactly what this means, but perhaps the X6-400M was being overclocked by transients or internal oscillations. We are trying to figure out if we can get a source to you to test with.

Thanks again,

- Ben

[jhenderson]

Ben -

We suspect that the new clock circuit design on the X6-400M is the source of the problem.

The bandwidth of the new circuit is much higher than the X5-400M. Consequently, as the clock signal changes through the transition band (the voltage considered on vs off), the any noise present on your clock source will be interpreted as clock transitions. If you idle in this condition, then the X6-400M will receive a blizzard of sample clocks, resulting in the data overrun.

We are attempting to reproduce this phenomenon here.

[dlewis]

Hello Ben,

We have modified an X6-400M. We modified the clock input hardware to match an X5-400M, and tested it with very low levels of clock input. I believe this is the best way to support you.

Looking at the swept laser source's data sheets online, the laser source's clock output is specified as "LVDS/TTL". The excellent data you took does not show common TTL levels (even low voltage 3 to 3.6 Volt TTL has a 0.8V Vil and 1.5V Vth). All the data was less than 1.2V, most "high" levels were only about 0.8V well below the typical TTL threshold level. With the shifting threshold level seen in the data, LVDS does not fit that data well either, though I suppose it could be argued to be one half of an LVDS differential pair with a shifting common level.

The type of clock input used in the X6-400M is often advertised by others as a "universal input". It is similar to an AC coupled low threshold differential input, where the inverted differential input is held at one half the supply voltage and almost any well behaved clock can drive it.

Regarding the excellent experiments you did with the Agilent signal source, I had limited success repeating these (I could sometimes get an error with a smaller clock signal, more often I got what I believe were clock slipping issues seen in the data without an error). However I believe I understand some of what is happening. With fast logic, low thresholds, and a low clock level; noise, interference and waveform shape can potentially cause multiple and/or intermittent (slipping) clocking. We are working to minimize the potential for this.

However, supporting small clock amplitudes can be double edged. To allow a >0.1Vpp clock specification we accept there will be clocking well below 0.1Vpp to guarantee 0.1Vpp works worst case. Also somewhere further below 0.1Vpp we will be on the verge of operation, perhaps intermittent or false clocking with noise, etc... before the clock gets so small as to have no effect.

Regarding the prototype circuit, I am less sure. If it is not trouble (no presentation needed) the part number of the OP-AMP and what else it is connected too (feedback or another load where there is a "T" in the output in your presentation) may help.

Since we had something that worked well for you in the X5-400M, we tried to copy the clock input circuit to a X6-400M for you. I would like to send this for you to try, if that is OK.


To clarify you use/desire DC coupled DACs and ADCs?
And we will be sure what we send is otherwise the latest revision.

Regards

Dave

[dlewis]

HI Ben,

Perhaps also check the grounding between the equipment as the threshold level seems to move when using the laser source clock, sorry if this is naive but it is better to mention it....

Dave

[potsaid]

Dave,

This is fantastic. We would love to try the modified card and can test right away.

Yes, we do use DC coupling on both the A/D and D/A.

We just shipped the X6-400M that needed the trigger resistor swap. We'll be sure to check the ground when testing with the new card.

Let me get the op-amp part numbers to you shortly...


Thanks again. Can't wait to see how it works. We really appreciate it.

- Ben

[potsaid]

Here is a bit more on the clocking.

The differences in the latest card we tested are that the external trigger now works because of the resistor swap and the X6-400M had an X5 clocking circuit.

This card is improved, but it looks like the improvement mostly came from the external trigger allowing us to start the acquisition after the troublesome voltages on the clock signal that caused the FIFO overrun (see slides 6-7).

We also have to be very careful to select a frame length that is shorter than the OCT data - this is to be sure that the A/D stops clocking before the signal dips down to the troublesome voltage range on the clock input. Otherwise, we see the same FIFO overun error (see slides 8-9).

We also tried putting a 200MHz low pass filter in the clock signal to try to reduce any high frequency noise near that troublesome voltage range from causing overclocking of the A/D.

The surprising outcome of this low pass filter experiment was that the error changed to an "Input Overrange" Error (see slide 2 in second attachment). This was surprising because there was no electrical connection to the AD0 input (and we did not see this error when internally clocking).

Might this unexpected "Input Overrange" error point to something strange happening in the FPGA code? If so, then maybe the fix could be code related.

We are continuing to try different things to get the card to work through the troublesome voltages. If we have success, we will let you know. If you have any thoughts on your end, we would love to hear.

Thanks again,

- Ben

[smoses]

I noticed in the attached screen capture of the Stream application, the Reference Source clock was set to "Internal" 10.0 MHz and the Clock Source set to "External" 330.0 MHz.

When in external clock mode, if the supplied clock is at the desired sampling rate (330MHz in this case), both of the Reference Source and Clock Source should be set to "External" and their frequencies should be set to 330.0 MHz since the Reference Source Frequency defines the rate of the clock that is connected to the SMA connector and the Clock Source Frequency defines the sampling clock rate.

Could you please re-test with the new clock setup?

Regards,
Shant

[potsaid]

Shant,

Thanks for the suggestion. We gave it a try and saw the same FIFO Overrun error that we had seen before.

For the OCT imaging application, it is important to bypass the PLL entirely. The clock frequency nominally changes from about 50MHz to 330MHz at a rate of about 200kHz for this imaging scenario. As the clock frequency changes, the delay associated with a PLL tracking the changing clock frequency would degrade the image quality.

We are trying to set up for the X6 clocking scenario shown in the attached figure from the X6-400M user manual. This configuration looks to bypass the PLL and clock the A/D directly.

There is a note in the user manual to set the clock frequency (even if the not using the internal clock) because the software uses this to estimate the data rate and size buffers appropriately. We set the rate to 330MHz because this is the peak clocking frequency.

We had left the ref input at the default 10MHz because we did not think the PLL would be used as part of the clocking. We reset this to 330MHz and rested:

As you can see in the results below, we are getting similar outcomes as before.

A quick question: Does setting the Clock Source to external fully bypass the PLL, regardless of the Ref Source Internal/External Selection?


--- Configuration 1 ---
Ref Source: Internal, 400MHz
Clock Source: Internal, 400MHz
Velo Packet Size: 0x300000
Trigger: Software
Mode: Unframed
Data Logging Ceiling: 512000000
RESULT: Card clocks to completion with no errors, showing that Packet sizes and bus transfer capability are OK with 817.4MB/s Rx Rate. Received 345 Blocks in 1.282 seconds.

--- Configuration 2 ---
Ref Source: Internal, 330MHz
Clock Source: External, 330MHz
Velo Packet Size: 0x300000
Trigger: Software
Mode: Unframed
Data Logging Ceiling: 512000000
RESULT: Card does 1 of 3 things with a pretty even chance of each outcome:

1. Input FIFO overrun error with 882 or 1226 or 879, etc. Blocks received.

2. Card hangs after 508 or 361 or 501 or 44, etc. Blocks received.

3. Card clocks to completion with 1038 Blocks received.

--- Configuration 3 ---
Same as Configuration 2, but with
Ref Source: External, 330MHz
RESULT: Looks similar to configuration 2 with:

FIFO overrun with 6 or 1237 Blocks received.

Card hangs after 77, 260 Blocks received.

Card clocks to completion with 518, 1038, 1039 Blocks received.

[smoses]

Setting the Clock Source to external fully bypasses the PLL. Only the output dividers of the clock distribution tree are active in this case. The software uses the Reference Clock Source frequency and the sampling clock frequency to determine how to set these dividers. Therefore it is important to have the Reference Clock Source be set to "External" and its frequency set to the sampling rate to bypass the output dividers. The PLL chip output clocks setting can be read by pressing the "Clock info" button on the Debug tab.

I'm thinking of instrumenting a logic version with chipscope and if possible remote accessing into your system to diagnose the problem.

I have another question: Is it possible to run a test with a fixed clock frequency and a sine wave applied to the ADC input? I would like to see how the captured wave looks hoping to help me in finding out whether the cause of the problem is the changing clock rate or the absence of clock.

Regards,
Shant

[potsaid]

Shant,

That is neat about the clock configuration. Thanks, I understand now.

You had suggested running a fixed frequency external clock and measuring a fixed frequency sine wave at the A/D. I am pretty sure that this would work fine. We had done quite a few experiments using an Agilent Analog Signal Generator before as the clock source. We found that clocking worked well, except for when the clock voltage was of a certain small amplitude. Some of this testing is summarized on slide 16 in the pdf in the first post of this discussion. Here we see that when the clock amplitude passes through a particular voltage region, the card issues the FIFO Overrun. We think that our clock signal passes through this voltage region, causing the error.

Similar to what you describe, though, we did an experiment where we connected the swept light source to the external clock and saved the data using the Stream.exe application. It is this frequency varying clock with dead times in the clock signal that is causing the FIFO Overrun problems for us.

First, we used internal 400MSPS clocking and found that a 1MHz signal at the A/D looks fine in BinView and there were no acquisition errors.

Second, we used external clocking from the swept laser source. Indeed, there was a FIFO Overrun and the data file is consequently smaller in size.
The files are large, so I uploaded a zip of them to google docs at this link:

https://sites.google.com/site/potsaiddatashare/

You may or may not see anything interesting in the data. If not, we should definitely consider getting some hardware out there for you to test with your chipscope setup. Let me know if you think this is a good next step and we can see what we can do.

Thanks again,

-Ben

[smoses]

Ben,

So I was able to get the FIFO overrun alerts and data flow to stop with a gated external clock.
The clock that I'm using is 330MHz gated by a 100Hz 80% duty cycle clock. I will try to find what's causing the overruns and let you know my findings.

Shant

[smoses]

Ben,

I found the cause of the FIFO overflow and data stoppage. When the sampling clock stops, the logic that runs on this clock starts misbehaving, specially the part of the logic that is responsible for constructing VITA packets.

I modified the logic by moving the sampling clock -> system clock synchronization earlier in the pipe and running the VITA framer at the system clock domain. The new logic flows data continuously without stopping or getting overflow alerts. However, I noticed the presence of bad samples in the stream which I believe is happening when the sampling clock is stopping or starting. The attached zip folder has a screen shot of the captured data during the absence of the sampling clock. I also attached the bit file that has the logic workaround.

Could you please download the attached bit file in your board and run your tests?

Regards,
Shant