Another piece of my ongoing FPGA stereo-vision project. This board is, as the name suggests, a breakout board for Aptina’s excellent MT9V032 1/3″ VGA image sensor.
The board’s main purpose in life is to connect the LVDS output of the MT9V032 sensor to my FMC-LPC to SATA adapter board, which would then route the LVDS data into one of Xilinx’s Spartan-6 FPGA development boards. Multiple camera boards would be connected to support stereo vision.
There’s more to the board than simple signal breakout, however.
There are a few big reasons for selecting the MT9V032 – but the biggest, by far, is its global shutter (“TrueSNAP” in Aptina’s marketing parlance). The vast majority of CMOS image sensors use a simpler rolling shutter, which leads to all sorts of troublesome visual artifacts. The MT9V032, on the other hand, is able to expose all of its pixels for the exact same slice of time, making it immune to many of those artifacts. That doesn’t, of course, do any good for more conventional artifacts – like motion blur; only shorter exposures can help with that.
Which brings up another point: the MT9V032 (for its size) has very large and sensitive pixels. It only has 752×480 of them (wide-VGA), but they’re each 6µm x 6µm – compared to, for example, the diminutive 1.75µm photo-sites in Aptina’s 2592×1944 (5 Mpx) MT9P013. The upshot being that the MT9V032 can take usable images in lower light conditions, and it can use higher shutter speeds to reduce motion blur.
Beyond raw sensitivity, the MT9V032 supports a form of high-dynamic-range exposures. By progressively reducing the sensitivity of its pixels over the course of an exposure, the sensor is able to approximate a non-linear response. This significantly increases the dynamic range that the sensor can capture in a single exposure – making it much easier to operate in environments with wildly varying lighting conditions (e.g. outside on a sunny day).
For a mobile robot operating in an unstructured outdoor environment, each of those features is pretty compelling (perhaps even mandatory). Finding them all in one device is even better. Those features come at a (monetary) cost, however – in small quantities, it’s easily twice the price of less-functional VGA sensors. That’s a trade-off that I’m willing to make.
The MT9V032 has one other interesting feature: it supports outputting all of its image data over a single high-speed (320 Mbps) LVDS pair. In a stereo vision setup, it even supports cascading two image sensors so that they can share the same LVDS pair.
LVDS doesn’t offer any direct benefit to the robot’s capabilities, but it does dramatically reduce the amount of wiring required to connect to each image sensor – which has the potential benefit of allowing remote sensors to be placed in more ideal locations. This breakout board capitalizes on the LVDS capability of the MT9V032 by allowing its entire interface to be handled by a single 4-signal SATA cable (with the exception of power, which is supplied separately).
That 4-signal interface is the main reason for the board’s additional complexity. 2 of the signals are immediately used for the LVDS video stream. 1 signal is needed to convey the 26.67 MHz reference clock to the image sensor – which can’t be locally generated, since the standard (non-MGT) deserializer blocks in the Spartan-6 FPGA are unable to perform embedded clock recovery (furthermore, precisely synchronizing multiple cameras in a stereo pair would be considerably trickier without a common reference clock). That leaves just 1 signal to configure and control the sensor.
The sensor uses an I2C interface for most configuration, and has a handful of additional control signals that may come in handy for certain applications (low-power standby controls, external exposure triggers, etc.). Placing a small 8-bit AVR microcontroller (an ATmega168) on the board was an easy way to provide local control over all of the MT9V032’s signals. To communicate back to the main FPGA, that 1 remaining signal is used as a bidirectional half-duplex serial bus (it’s hooked to both the transmit and receive lines of the AVR’s UART).
The image sensor and the AVR are nominally 3.3V devices with 3.3V I/Os (except for the MT9V032’s LVDS output – which is, well, LVDS). The Spartan-6 FPGA on the other side of the SATA cable is, however, setup for 2.5V signaling (which Xilinx FPGAs require for LVDS transmission). As a result, some level translation is also needed on this board – and it needs to be pretty fast, too, since both the fast ~27 MHz (54 Mbps) clock and the slow serial bus need to be translated. TI has a whole slew of nice voltage translator – including the SN74LVC1T45 dual-supply bidirectional translator, which I’ve used on this board. It’s rated for well over 100 Mbps.
The translators don’t have automatic direction sensing (a feature which typically imposes a significant performance penalty). The AVR is, thus, tasked with manually controlling the direction of each translator. Normally, the clock signal will always be received by the board – still, there are provisions for (temporarily) co-opting the clock line as an additional bidirectional communication line with the FPGA. The serial line’s direction must, of course, be suitably driven by the AVR depending on whether or not it is actively transmitting or receiving (a trivial burden, as the AVR must already do this with its own TX/RX pins).
2.5V power for the translators (they’re the only devices on the board that need it) comes from a small SOT-23 linear regulator (a MCP1700). 3.3V power for the rest of the logic is also via a linear supply (a larger, SOT-223 packaged, ZLDO1117). The analog supplies to the MT9V032 are run through ferrites, to remove some of the noise introduced by the high-frequency digital logic on the board (the AVR and the MT9V032’s own digital logic). The input to the board is nominally 5V – but, since this supply only drives the 3.3V regulator, it can tolerate significant variation (subject to the ZLDO1117’s input voltage and power dissipation limits). Current draw of the board when running flat-out (60 FPS with LVDS enabled) is on the order of 100mA, so keeping the input voltage low reduces power loss in the linear regulator.
Having a dedicated microcontroller on the board opens up additional possibilities, as well. The AVR’s two unused ADC/GPIO pins are brought out to an expansion header for future use. One could, for example, use a lens with an integrated variable iris/aperture – which the AVR could dynamically adjust in response to extreme changes in ambient lighting (e.g. transitioning between deep shade into direct sunlight).
Regarding lenses: the board is designed to work with two different lens holders from Sunex. The CMT821 (seen in the photo) and the CMT103 are both supported. They’re both M12x0.5 (S-mount) lens holders, with the main difference between them being their height (lenses of different focal lengths and construction may require different mounting heights to achieve critical focus).
Currently, a DSL903B 6.6mm lens is attached (largely because Sunex was offering them at a hefty discount when I originally bought the lens holders). On the MT9V032, this works out very nearly to a normal focal length (~50mm in 35mm format terms). In practice, a wider angle lens may be needed to give a wide enough field of view for robot navigation (at the cost of reduced stereoscopic depth resolution).
Layout was easy; space wasn’t at a premium, and the amount of components to place and signals to route was few (owing in no-small-part to the MT9V032’s LVDS interface obviating the need for routing any of the sensor’s parallel outputs). The majority of the signals are on the top layer, with just a few breaking up the otherwise continuous ground fill on the bottom layer. The sensor’s LVDS output is routed as directly as possible to the SATA connector, and the 27 MHz clock is routed so-as to minimize reflection-inducing stubs.
The board has mounting holes for the camera lens holder, and also has four well-defined mounting holes with 1″ spacings – making it easy to mount to some sort of development structure, for experimenting with various multi-camera setups.
The board was fabbed through Laen’s excellent 2-layer PCB service (just under $15 for 3 copies of this 1.7″x1.7″ board!). Assembly wasn’t a problem. I used a laser-cut Kapton stencil from OHARARP to apply solder paste to the board, loaded all of the parts, and then reflowed the whole thing in my modified toaster oven (as I have done before).
The firmware for this board doesn’t have to do a whole lot at this point. Currently, its primary task is reconfiguring the MT9V032 in order to enable the LVDS output. The hardest part in that was tracking down a minor problem on the PCB: I accidentally swapped the data and clock lines for the I2C bus in the schematic (one of two known issues with the board – the other being a typo on the bottom silkscreen). This prevented me from using the AVR’s built-in I2C peripheral, so I had to resort to writing a simple bit-banged I2C routine. The AVR has plenty of cycles to spare, so there isn’t much concern over the efficiency lost from not using the hardware peripheral.
Eventually, the firmware will be enhanced to give the FPGA some amount of control over the image sensor. For stereo-vision, you (ideally) want to have all cameras perfectly synchronized and with matched exposures. Synchronization is easy; if configured identically, clocked from the same source, and removed from reset simultaneously, the image sensors will operate in lock-step. Normally, exposure is automatically adjusted by the MT9V032; by giving the FPGA control over this, however, it should be possible to achieve slightly better correlation between multiple sensors.