Post a Comment On: Ken Shirriff's blog

.hilite {cursor:zoom-in} a:link img.hilite, a:visited img.hilite { color: #fff;} One of the under-appreciated aspects of the Apollo launches to the Moon is how much testing was required. I recently came across an item that was part of this testing: the Computer Buffer Unit. It is essentially a 16-bit shift register that interfaced test equipment to the Apollo Guidance Computer. While a shift register is a trivial circuit nowadays, back then it took a box full of transistors that weighed about 5 pounds. In this blog post, I look inside this unit, describe its unusual packaging and circuitry, and explain how it works. [Image]The Computer Buffer Unit is a 4"×6"×6" box. The three electrical connectors on the left are covered by protective covers. It has a humidity indicator and pressurization valve at the bottom. Testing for the Apollo missions The Apollo spacecraft required extensive testing even while it was sitting on the launch pad. Thousands of different spacecraft components needed to be activated and analyzed for various tests. Since the control room was miles away from the launch pad, it wasn't practical to run separate wires to each component. Instead, NASA invented (and patented) a complex digital test system that communicated efficiently between the control room and the rocket. This test system sent digital commands to the launch site, where racks of control and interface units were wired to the spacecraft components. These units decoded the commands and performed the specified operation. Massive quantities of measurement data from the spacecraft were encoded digitally and serialized for communication back to the control room. The complexity of testing is illustrated by the control room below.2 This is not Mission Control, but a separate control room specifically for testing, called ACE-S/C (Acceptance Checkout Equipment-Spacecraft). These consoles were crammed with control switches, tape readers, CRT displays, chart recorders, and status panels for conducting tests and recording results. The ACE-S/C system supported manual, semiautomatic, and automatic testing, driven by two minicomputers1. [Image]ACE control room. From Applicability of Apollo Checkout Equipment. All parts of the spacecraft were tested, including the fuel cells, cryogenic fuel storage, communications, and environmental control. For this blog post, the relevant subsystem is "Guidance and Navigation", responsible for determining the Apollo spacecraft's position in space using inertial navigation and guiding it on the proper trajectory including the landing on the Moon's surface. The key to Guidance and Navigation was the Apollo Guidance Computer, 70-pound computers onboard the Lunar Module and the Command Module. [Image]The Apollo Guidance Computer that we restored, next to a replica DSKY. In space, astronauts operated the Apollo Guidance Computer through the Display/Keyboard (DSKY), a box (above) with keys, indicator lights, and numeric displays. But for ground testing, there needed to be a way to feed commands into the Apollo Guidance Computer from the testing system. The solution was the Computer Buffer Unit, the box that I'm examining. To operate the Apollo Guidance Computer remotely, the ACE test system encoded each DSKY keypress as a 16-bit command3 and sent it to the Buffer Unit. The Buffer Unit converted the message to serial, transferring one bit at a time to the Apollo Guidance Computer, which then processed the desired keypress.4 Thus, the Apollo Guidance Computer could be controlled remotely for testing, providing control over the Guidance and Navigation system, and the Computer Buffer Unit was the interface with the Apollo Guidance Computer. Inside the Computer Buffer Unit Next, I'll discuss the physical construction of the Computer Buffer Unit. Removing the lid reveals the components inside.5 The main circuitry consists of six horizontal circuit boards wired into a vertical backplane board; the top board is visible below. One unusual feature is the bag of desiccant inside the unit, zip-tied to the right side of the case. The designers of the unit were worried about Florida humidity and the risk of corrosion.6 To guard against damp air, the unit has a valve on the front so it can be pressurized with dry nitrogen. On the front of the unit, you can see a humidity sensor that changes color to indicate 10%, 20%, and 30% humidity. If the internal humidity exceeded 30%, the desiccant needed to be replaced, as described by the warning label. [Image]The Buffer Unit with the lid removed. I removed the circuit boards with some difficulty, as they fit tightly. The photo below shows the stack of six printed circuit boards wired into the vertical backplane. The wires from the connectors are soldered directly to the backplane. [Image]With the circuit boards pulled out of the unit, the wiring to the backplane is visible. The circuit boards can be opened up like a book to provide access to the inner boards. The boards are not soldered directly to the backplane, but are connected by short, flexible wires, allowing them to swing apart. To prevent short circuits between the boards, they are separated by white sheets of (probably) silicone. [Image]After removing six screws, the boards can be unfolded like a book. The circuitry is constructed in a very unusual way that I haven't seen before. Instead of mounting components directly on the circuit boards, components are mounted on small boards, each forming a module with a logic gate or two. These smaller modules are then soldered on pins above the main circuit boards, forming two-layer boards. Essentially they built pseudo-integrated-circuits on small boards, and then constructed the circuitry from these modules. [Image]Closeup of logic modules mounted on the circuit board. A blue resistor is visible on the underside of the module. It is difficult to see the components sandwiched between the main board and the smaller modules, but the side view below shows some of the components. The two boards are connected by the vertical pins. A tiny glass diode is visible towards the left. The longer components are resistors. The shiny metal-can transistors are in the middle of the module and harder to see. [Image]This side view shows a latch module (bottom) attached to the circuit board (top). The diodes, resistors, and transistors of the latch module are visible. One question is why the circuitry is implemented with small circuit boards attached to the larger circuit board, instead of mounting the components directly on the circuit board. This approach seems overly complex and makes the boards twice as thick. One advantage, though, is that the separate logic modules could be manufactured, testing, and repaired separately, important in an era when semiconductors were less reliable. Second, the main boards and the logic modules are different types of printed circuit boards: four-layer circuit boards with widely-spaced traces versus single-sided but dense boards. Logic gates The circuitry is implemented with a logic family called Diode-Transistor Logic (DTL). This type of logic was used in the early 1960s as it only required one (expensive) transistor per gate, using cheaper diodes where possible. As transistor prices dropped, Transistor-Transistor Logic (TTL) became more popular because of its better performance. Nowadays fast, low-power CMOS logic is used in most integrated circuits. I reverse-engineered the schematic below, which shows a NOR gate from this unit. This gate has two inputs, as well as two outputs (for reasons that will be explained below). If both inputs are low (0), the transistor will be turned off. As a result, the resistors pull the outputs high, producing 1 outputs. [Image]The NOR gate with both inputs low, outputs high. If an input is high, the circuit behaves as shown below. Current flows from the input pull-up resistor through the diodes and the transistor's base, turning the transistor on. As a result, current flows from the outputs, through the transistor to ground, pulling the outputs low. Thus, the circuit implements a NOR gate: the output is 1 if all inputs are low, and 0 otherwise. [Image]The NOR gate with a high input, outputs high. The reason for multiple outputs is clever. If you connect the outputs from multiple gates together, this combined output will be pulled low if any output is low (i.e. the transistor is turned on), and otherwise will be pulled high by the resistor.7 This logic is equivalent to an AND gate. Note that the AND gate is implemented "for free" by wiring outputs together, without requiring additional logic; this is called wired-AND. However, you can't use a gate's output in two different wired-AND gates, since everything will be shorted together. Instead, a gate provides multiple outputs that can be wired independently; the diodes keep the outputs isolated from each other. The board I examined has 5 different types of logic module8, from an inverter with 1 input and 8 outputs to a module with two 2-input, 5-output NOR gates. These modules follow the circuit above, but with different numbers of inputs and outputs. Implementation of the shift register The idea behind a shift register is to store multiple bits in a row. Each time a clock signal is activated, the bits are shifted by one position. Shift registers can be used to store data, convert parallel data to serial, or convert serial data to parallel. In this Buffer Unit, the shift register converted a 16-bit parallel value from the test equipment into a serial stream of bits for the Apollo Guidance Computer.9 The board implements four bits of the 16-bit shift register. The schematic below shows the circuitry for a one-bit stage of the shift register. There's a lot going on, but I'll try to explain it. The heart of the stage consists of two latches, which store one bit. A bit is stored by first updating the primary latch, and then the secondary latch. (Each latch consists of two cross-coupled NOR gates, and can hold either a 0 or a 1.) The shift out lines are the outputs from the shift register stage, a regular output and an inverted output. [Image]One stage of the shift register. It can read the bits in parallel, or shift a bit from one stage to the next. Each shift out line is fed to the shift in lines of the next stage, allowing the bits to be transferred from stage to stage through the shift register. The shift and load control lines, along with the AND gates, select the input to each stage. With shift high, the input will be the shift out from the previous stage. With load high, the input reads the external bit in lines. This allows a 16-bit data word to be read into the shift register in parallel. (I'm not sure what the clear bit function is used for.) After a bit has been loaded into the primary latch, the clock line is activated to load the bit into the secondary latch, completing the shift or load cycle. An interesting function of the unit is that after loading, the value in the latch is compared to the input value, to make sure that the circuit is operating correctly. If there is a mismatch, a compare AND gate will activate, clearing the match line. (A compare AND gate will activate if the input bit is 1 and the latch bit is 0, or vice versa.) This circuit also detects a fault in the bit input wires. Each bit is provided over two wires: one with the bit value and one with the inverted bit value. If a wire is broken or affected by noise, the comparison will fail.10 [Image]This diagram shows the functions of the gates. Note that the circular golden transistors are faintly visible through the circuit boards. The board above11 contains four of these shift-register stages. The photo above shows how these stages map onto the hardware. The external signals (4 pairs of bit lines) enter at the bottom of the board, and pass through the input inverters. The 8 primary latch NOR gates implement four primary latches. Four secondary latch modules implement the four secondary latches, since each module contains two NOR gates. The clock driver, load driver, and shift driver provide 8 copies of the clock, load, and shift signals for the circuitry. Finally, the two match NOR gates combine the 8 match signals. (Note that since the AND gates are implemented with wired-AND, they don't use additional circuitry and do not appear in this diagram.) I/O and power I'll wrap up with a few comments on the I/O and power supply for the Buffer Unit. The unit has three military-style connectors on the front. At the top is a 61-pin connector for receiving the parallel data and control signals from ground equipment. (The pin count is larger than you might expect because each bit uses two wires as discussed earlier. Also, many of the 61 pins are unused.) [Image]The unit has three connectors. The unit receives parallel data from ground equipment through the 61-pin connector at the top. The middle connector communicates the serial data to the Apollo Guidance Computer. The unit is powered with 28 volts through the bottom connector, which has larger pins for the high-current supply. The middle connector has four pins that provide the serial data stream to the Apollo Guidance Computer. The wiring is a bit unusual. Instead of transmitting data over one serial line, the unit uses two pairs of lines: one to transmit "0" bits and one to transmit "1" bits. To provide electrical isolation between the unit and the Apollo Guidance Computer, these signals are transmitted via two small pulse transformers, shown below. When a pulse is fed into a pulse transformer, a similar pulse is produced on the output. (In modern equipment, an optoisolator provides similar functionality.) [Image]Two pulse transformers on the top circuit board. Each small transformer is about 1 cm in diameter. The bottom connector on the unit has two thick pins to provide 28 volts to the unit. This view inside the unit shows the power converter, a sealed black box. I believe this is a switching power supply module that converted the 28-volt input into the lower voltage required by the logic circuitry. It also provided electrical isolation from the power supply. The smaller black box on the right is an EMI filter on the power input; the Apollo ground test equipment encountered faults from voltage transients and electrical noise, so they added filtering. [Image]The power supply components are sealed in black plastic. Conclusion This Computer Buffer Unit was built in 1965, a time when the industry was shifting from transistors to integrated circuits. This may explain the Unit's unusual construction technique, small circuit-board modules that are like integrated circuits built from discrete components.12 Interestingly, Motorola built a similar Buffer Unit for NASA that used integrated circuits (but was just as large),13 illustrating that transistors and integrated circuits were both viable approaches in 1965. This box also illustrates the rapid pace of integrated circuit technology since the 1960s. The first commercial MOS integrated circuit was a 20-bit shift register introduced in 1964 and by 1970, Intel was producing a 512-bit shift register. In 1971, Western Digital was selling a UART chip, putting a complete parallel-to-serial and serial-to-parallel communication system onto a chip. Thus, it took 6 years to shrink the complex shift-register box down to a single chip (more or less). Nowadays, this functionality forms a tiny part of a complex chip. Coincidentally, Moore's Law, describing the exponential growth of integrated circuits, was published in 1965, the same year this box was manufactured. I announce my latest blog posts on Twitter, so follow me @kenshirriff. (The Twitter thread corresponding to this blog post is here.) I also have an RSS feed. Thanks to Steve Jurvetson for letting me examine this artifact. A video tour of his space museum is here. Thanks to Mike Stewart for providing documents and extensive information on this box. Notes and references The photo below shows the ACE computer room that supported ACE testing. The system was controlled by two 13-bit CDC 160-G minicomputers. Strangely, the CDC 160-G minicomputers were 13-bit computers, with 13-bit addresses, 13-bit registers, and 13-bit arithmetic. The earlier CDC 160 computer was 12 bits, and CDC improved the 160-G model by adding one more bit. The CDC 160 was designed by Seymour Cray, reportedly over a weekend. [Image]"An ACE Station with twin Control Data computers." From Computers in Spaceflight.  ↩ There were about 10 ACE installations for testing at various sites. ACE testing was performed at contractor sites, as well as at the launch pad. ↩ To send a DSKY keypress through the testing system, each keypress was encoded as 5 bits as shown below. The 16-bit message consisted of a 1 bit followed by three copies of the 5-bit keypress, with the middle copy inverted. (Sending the keypress in triplicate detected communication errors.) [Image]The encoding of keys when communicating with the Apollo Guidance Computer. From ACE-S/C Operator's Manual.  ↩ The serial protocol used by the Apollo Guidance Computer is a bit unusual compared to modern serial protocols. Instead of a single serial line, it used two pairs of wires: one to receive a 1 bit and one to receive a 0 bit. This worked well with the Apollo Guidance Computer hardware, which included a feature for incrementing and decrementing counters in response to interrupts. In particular, a serial input 0 triggers a SHINC instruction (shift left), while a serial input 1 triggers a SHANC (shift and increment by 1) instruction. (The interrupt-triggered counter mechanism worked well except during the Apollo 11 landing, when the power supply for the Apollo Guidance Computer and the power supply for the rendezvous radar had a phase difference. For complex reasons, this resulted in a high rate of interrupts, overloading the Apollo Guidance Computer and causing restarts. This was indicated by the famous 1201 and 1202 program alarms during the landing.) [Image]The K-START (Keyboard - Selections To Actuate Random Testing) panel is used to send commands to the Apollo Guidance Computer. From ACE-S/C Operator's Manual. In the ACE testing control room, DSKY keypresses were entered on a panel called K-START (Keyboard - Selections To Actuate Random Testing), shown above. The keyboard corresponds to the keyboard on the DSKY, while it has other switches specific to testing. These key entries could also be recorded on perforated tape and played back at high speed. ↩ Another interesting feature of the unit is how it is mounted on a rack. The back of the unit has two Teflon-lined holes. Two "dagger pins" from the rack fit into these holes. On the front, the unit has two small hold-down hooks; a knob on the rack engages with the hook to hold the unit in place. The mounting hooks are type NAS 622, an aerospace standard. The hold-down mechanism is described here. [Image]Back of the Buffer Unit with identifying label and two holes for dagger pins. The labels say "Unit, Computer Buffer Guidance & Navigation. NAA/S & ID Control No. ME901-0271-0002. Stock No. Contract No. M5H3XA-450001. NAA/S & ID Inspection Serial No. Control Data Corporation MFGR Part No. 106068-0002. Mfgr Serial No. 10136SA08185. US Nov 19 1965.  ↩ The document Acceptance Checkout Equipment for the Apollo Spacecraft discusses the corrosion problems encountered by the test equipment due to humidity and insufficient air conditioning. The specifications don't discuss pressurization of the unit, but I'm assuming they used nitrogen based on other items I've studied. ↩ One subtlety with the wired-AND gate is that connecting multiple outputs together will result in multiple pull-up resistors in parallel, which may provide too much pull-up current. The solution is that some gates have outputs without pull-up resistors, so each wired-AND output has a single pull-up. The wired-AND isn't entirely free, since the multiple outputs require multiple diodes, but diodes are inexpensive compared to transistors. I should admit that I'm not 100% sure of the circuitry. Since the components are all hidden underneath the module, I had to deduce the circuitry by probing it from above. There were a few inputs that didn't seem to have connectivity; perhaps there are capacitors to make these inputs pulse-based. ↩ The board I examined uses the following types of modules:
2304: 1-in, 8-out inverter
2309: 3-in, 4 out NOR
2311: 4-in, 2-out NOR
2319: 1-in, 4-out inverter
2314: dual 2-in, 5-out NOR (larger than the other modules) ↩ The specifications for the Buffer Unit describe its purpose: "This specification covers the requirements for a Guidance and Navigation Computer Buffer Unit, hereinafter referred to as the G&N buffer. The G&N buffer shall form a part of the Digital Test Command System (DTCS) which is the up-link portion of the Automatic Checkout Equipment (ACE). The ACE will be used as ground support equipment for the Apollo space craft. The G&N buffer shall receive remotely generated digital test commands from the control room via the DTCS and shall store, verify, and shift out G&N data in appropriate format to the G&N on-board computer." [Image]Functional diagram of the Buffer Unit. Image from Specification MC 901-0666 courtesy of Mike Stewart. The specifications for the Computer Buffer Unit can be viewed online: MC901-0666, ME901-0666, ME901-0271, ME476-0070. The unit includes more functionality than just a shift register (but not much more). As shown in the functional diagram above, the unit also includes the clock oscillator that controls the timing of the serial pulses. Second, it contains a control circuit to handle loading the bits in parallel and then shifting them out serially. Third, for reliability reasons, it has a comparator circuit to check that the bits loaded into the shift register match the input bits. ↩ Modern systems often use differential signaling, using two complementary signals for a bit. Looking at the difference between the two signals provides noise immunity, since electrical noise will often affect both signals equally, and thus will be canceled out. Although the Buffer Unit uses two complementary signals, it doesn't provide this noise immunity, since the two signals are processed independently rather than differentially. ↩ I only reverse-engineered one of the boards, since I didn't want to risk more disassembly, and one board is enough to understand the basic logic. I studied board 6 of the unit, which implements bits 15 through 18 of the shift register. Board 3 implements bits 3-6, board 4 implements bits 7-10, as well as mode bits 1 and 2, board 5 implements bits 11 through 14, and board 6 (the one I examined) implements bits 15 through 18. Boards 2 and 4 implement control logic, while board 1 has the output driver transformers. [Image]With board 6 folded down, board 5 is visible. The photo above shows board 5. Note that the circuit layout is entirely different from board 6. I thought that the unit might consist of four identical 4-bit shift register boards, but it turns out that the boards are optimized for particular roles. ↩ In the context of "not-quite-integrated circuits", I should mention IBM's use of hybrid modules (called SLT) for the System/360 mainframes. These small aluminum-cased modules contained a few transistor or diodes as silicon dies, mounted on a ceramic substrate along with thick-film resistors. These modules were not quite integrated circuits, since they were built from discrete (but unpackaged) components. But they were closer to integrated circuits than the modules in the Buffer Unit, which used packaged transistors, resistors, and diodes on a printed circuit board. ↩ Motorola made a similar Buffer Unit, but they used integrated circuits, specifically Motorola's line of high-speed ECL chips, introduced in 1962. Since each chip is a few gates, it still took multiple boards to build the unit. Apollo Guidance Computer expert Mike Stewart has photos of the Motorola box here, as well as reverse-engineered schematics. The functionality of the Motorola box is nearly identical, except it has separate inputs for the 16-bit compare value. It is built with chips such as the MC308 flip flop and MC 309 dual NOR gate, described here. [Image]A board from the Motorola version of the Buffer Unit. Each metal can is an integrated circuit. Photo courtesy of Mike Stewart.  ↩