Gen 3.2 Evaluation
- General Microservers, Vexcel Microservers, Quick Reference, Data Acquisition, Middleware
- Components GPS, SBC
- Configuration Microserver, SBC, SD Card, Power
- Operation Communication Protocol (SBC <--> uc), SBC Operations, Agent
- Operation (background) Task Manager "milo", config files, MACRO
- Microcontroller Microcontroller, Firmware, Skeleton firmware
- Evaluation 2008 Cairn relay failure evaluation, Lab evaluation, Eval-Firmware, BPMS, Source notes
- PCS Board PCS Board Design, Voltage Monitoring Circuit
- Gen 3.1 Kernel Upgrade and Field Notes (Spring 2007)
- Gen 3.1 Microserver, README, Schematics
- Gen 3.1 Task Manager "milo", GPS, Firmware
- 1 Introduction
- 2 Enclosure / Overall Issues
- 3 PCS
- 4 Communication / Software / Firmware / Operation
- 5 Programming Cables
- 6 Operate / Hibernate cycle testing
- 7 Potential Problems / Hangs
- 8 Operational Timing
- 9 Sleep
- 10 Watchdog Timer (WDT)
- 11 Version 3 Description (Functionality / items not implemented )
- 12 Limitations / Issues / Not Implemented
- 13 Power consumption
- 14 GPS notes
- 15 DIO cables
- 16 Battery Monitor Circuit
- 17 Prototype and Build Boards
- 18 Documentation
- 19 ICD programming text
- 20 edip functionality
The prototype Gen 3.2 Power Conditioning Subsystem (PCS) boards evaluated well enough (winter 2008) to merit a production run of 16 boards with two "scab" modifications made to each. This build was accompanied by the purchase of corresponding Single Board Computers (Technologic Systems TS-7260) and 16 Microserver Enclosures (everything else). This page documents the evaluation of the PCS boards (circa April 2008) and in the bigger context the Gen 3.2 Microserver (Vus) as a whole. uc refers to the Microcontroller, SBC to the Single Board Computer.
There are three code firmware versions relevant here (indexed by the myVersion variable):
- Version 1: Pre-production run code
- Version 2: Production run testing phase
- Version 3: Production run first operational software version
These notes include parenthetical indicators like "not tested yet" = (...?...) and Good! = (+) and Warning! = (!)
Enclosure / Overall Issues
- Replaced metal screws with nylon in battery clips to prevent short.
- Battery clips should NEVER EVER BE USED WITH METAL JACKETED D-CELL BATTERIES because of the danger of shorting.
- Needed: Double-ended threaded standoffs (size?) to emplace base station mote; an inquiry has been sent to KimCo.
- 1/3 watt is burned by the system passively when sleeping (low power state). This is due to vampire draw from the off-board power components, the SunSaver-6 charge controller and the 12V voltage reg (enclosure floor Northwest). For a fix-plan see the dedicated page: Gen 3.2 BIG POWER MOSFET SWITCH. (An initial try at building a bypass failed due to part overheating.)
- The enclosure wall N-type connector will require a pigtail to interface to the mote board connector.
- Need spec for tapped holes (mote mount) in base plate. "#1-72" screw works, I think, but verify/expand this.
There are two types of PCS modifications: Those incorporated as scab-on during the Build 1 (March 2008) production of 16, and those that can be deferred would be fixed (together with the scabs) on a board re-spin. In addition here are some notes on procedure and quality control here as well.
- Do not use a meter probe or other metal near the PCS board unless taking extreme caution. The board is not designed to withstand accidental contact shorts and will try and melt/destroy itself given any opportunity.
- The RJ45 uc programming connector should be examined to ensure the internal connecting pins are ok.
- To avoid shorts and other damage to the (all-too-fragile) PCS board: Suggest inserting an insulator cover, even insulating foam. Hot pins coming off the PCS, e.g. JP7 and JP8 connectors would be protected.
- RESPIN The JP9 test header has 5 test voltages that all read == uC supply voltage, not their intended supply voltages.
- RESPIN The supply voltages all read correctly but several when turned off decay (cap) rather than drop to zero. This should be fixed by putting a load on them (can see this by loading with an 8kohm resistor).
- RESPIN RJ45 connector is of a poor make on at least one PCS board, making it impossible to program the uC without some hard torqueing on the RJ45 spring-wires.
- RESPIN Re-order JP9 RB pins and A/GP pins
- RESPIN The RJ45 programming connector is side-justified rather than centered.
- Correct: Power up with only internal batteries attached: No voltages appear on JP set (except maybe transients; it would be a good idea to set everything Lo explicitly on start.)
- RESPIN Power up with only internal and the Reset button does nothing; only the power slider can reset.
- RESPIN Power up with external supply connected: 3.3V appears on JP8 with the slider power switch off. Correct voltages appear on other JPs intermittently.
- RESPIN A connected external supply makes the slider switch stop working; insulates the PCS from the power/reset switch combination; there is no obvious way to reset the PCS short of disconnecting the external power.
- RESPIN Pulldown resistors needed on the power supply lines (current slow decay on several).
- RESPIN The ref voltage part REF198E needs two caps Gd--OUT to stabilize the otherwise oscillatory reference voltage.
- RESPIN JP9:1--4 is supposed to be passive A/GPIO; JP9:5--9 is supposed to show internal/external supply voltages. On P4 it looks like JP9:1, 2, 4 are ok but 3, 5--9 are all at Vdd (the uC supply voltage). Checks on JP7, JP8, and J7 and on P5 all check out fine so this would appear to be a JP9 wiring issue (and I hope not a leak).
- RESPIN With external power attached to the PCS the slider switch / reset button does not work (it has been circumvented).
- SCAB A 1.0 and 0.1 uF cap were soldered in parallel across U17 pin 6 (Vref-out) to Gd. Caps are necessary to remove high-freq oscillation from the reference voltage signal.
- SCAB The 5V-B supply voltage reg U11 was replaced with a minilynx switching VReg to cut power waste (heat). KimCo chose a negative-logic unit so the prototype boards (which shouldn't use U11 anyway) and the Build 1 boards will always differ in firmware on that PORTD pin set/clear. This scab modification is somewhat apparent in the following image (blue wires etc at U11).
Communication / Software / Firmware / Operation
- There is still message latency due to a factor on the uc side and a factor on the SBC side.
- uc side: Use a 1-second delay polling loop which is reasonably well-calibrated to real time. However this means the uc is checking for new messages only once per second. Reducing the polling loop wait to milliseconds makes the message passing go much faster but messes up the uc sense of time so a big-picture cure is needed, not a quick-fix issue.
- SBC side: Putting a system("date"); call in front of every DIO port read has some "buffer flush" effect because without this the DIO gets really really bogged down. In the latter case a 6-Signal Message can require upwards of 15 seconds to get across.
- Verified: JP9 Pin 21 is the correct clock signal
- Verified: All power supply lines are at the correct voltages
- Verified: Power supply controls on uC work (except pulldowns needed as noted).
- At about 8.5V the off-board 12.08V Vreg drops out. This means it can bench-press up to its standard 12.08V output
even when the supply voltage is well below 12V. Of course this draws more current as the supply voltage drops. Do not let the device operate below 9V external supply, or select a higher voltage threshold to be more safe.
- Note Prototype 4 has ip 192.168.1.4 and its Bridge has ip 192.168.1.204
- Lab transfer rate: 270 kbps
- 12V-Internal JP7 Slow Decay (5 sec from 12.054 to 11.972)
- 3.3V-Internal JP8 Faster Decay (5 sec from 3.30 to 1.78)
- J7 East pin 12V-External Slow Decay to 11.964 as above Internal
- J7 Adjacent-to-East pin 5V-External Rapid decay reaches zero in about 3 sec acceptable
- J7 Adjacent-to-West pin 3.3V-External Faster Decay as above, reaches 1.8V in 5 sec
- Timing pretty accurate but still can be refined.
- Open firmware issue: Startup code pre-edip switch: Must ensure RE0:2 (configured using ADCON1) stay DIO for edip-switch purposes.
- The system voltage switches over to external about 2 seconds after PowerOn.
- Can an Ethernet cable come in from outside and go directly into the SBC?
- External USB cable: Can use as a serial data drain for example?
- Need to demonstrate that the 3.3V supply can drive an internally mounted Tmote Sky
- Need to establish serial interfaces to SBC: GPS and COM2
The MPLAB ICD (In-Circuit-Debugger) is a hockey-puck sized piece of plastic used to program the microcontroller (uc) on the PCS board. This device is first connected to a PC running the MPLAB IDE (download for free from http://www.microchip.com) where the current Workspace encapsulates the Gen-3.1 or Gen-3.2 firmware.
The ICD connection to the PC is made by means of a USB cable that comes with the ICD. The ICD is also powered by a 9V supply. Finally the ICD connects to the RJ45 connector in the center of the PCS board by means of a 6-conductor programming cable. The RJ45 connector in the PCS board has 8 signal lines. The programming cable connects to the ICD by means of an RJ12 connector. This 6-to-8 arrangement is unfortunate as there is some room for error. These notes explain how to make sure the connection is ok.
First, the programming cable must be as short as possible (less than 10cm is a good rule of thumb) to avoid lossiness in the programming process. Second, the programming cable must conform to the following:
- Gen 3.1 programming cable: Use RJ12 to RJ12 with connector clip on the same side of the cable at the two ends. The RJ12 will fit into the center of the PCS RJ45 and this is correct; no off-center bias is necessary.
- Gen 3.2 programming cable: Use an RJ12 connector on the 6-conductor cable at the ICD end, crimping this in normally. Place an RJ45 connector on the other end of the programming cable where the connector clip is on the same side of the cable at both ends. However WARNING the six conductors must be sideways-justified before crimping down the RJ45, as follows: Orient the cable so that the connector-clips are on the down side (so you are looking at the cable from above and do not see the connector clips). Put the RJ45 side of the cable facing away from you (so the RJ12 is closest to you). In this orientation, justify the 6 conductors as they go into the RJ45 connector all the way to the left. Crimp that and verify that the cable works properly by programming the PCS board.
Operate / Hibernate cycle testing
- (+) 339 up-down cycles over the weekend of 4/18--4/21 (Fri afternoon to Monday morning) using no apparent hangs or other problems.
Potential Problems / Hangs
- (!) Halting updown during its "pause" interval is ok. Halting during a Msg can potentially hang the uc.
- (+) There are two uc communication-hang modes that can be fixed manually (or automatically) from the SBC:
- SBC Message Send halts before completion: Can use "bc" to toggle the lead bit one Signal at a time until clear.
# bc 142 128 142 # bc 0 142 0 0 # bc 0 128 128 # bc 0 128 0 0 # bc 0 128 128 # bc 0 128 0 128
In this example the uc stopped at 142 in Receive mode and was waiting for PORTB MSB to go to Zero. This was done manually on the next run of bc. When bc is run with no args it sends the Rest State value of 128 which toggles the MSB back to Hi. This series of alternating Hi / Lo / Hi / Lo pulled the uc all the way through its Signal Series back to the Rest State (where it did not respond to the last bc 0 because that is not recognized as an Alert value.
- SBC Message Receive halts before completion: Can use "bc" to echo what the uc is sending until clear. When the above method fails to produce changes in the uc, it is very likely that the uc was in Send mode so bc can be used to echo the PORTC value, similarly stepping the uc through all the Signals it wants to send as part of the current Message.
- Note that if the WDT is enabled at the time of these hangs, the system will reset in 120 seconds so you have that long to fix the issue. There is no clrwdt (Watchdog Timer reset) inside the uc MSB-toggle polling loops.
- (-) Could bring myDropDeadSeconds into the game to make timing calibration more accurate.
- (-) Periodic Messages from SBC will throw off the timing (time will pass more slowly in the uc)
- (...?...) Version 3 has a "decent" clock but it needs a proper benchmark.
- (+) Sleep recalibrated to be accurate to within a couple seconds over 40 minutes (see remarks in code)
- (+) Sleep current draw = 24 mA without BPMS (perhaps 100 days on a 100 Amp-Hour supply)
- (+) Ext-power disconnect during sleep does not appear to damage the sleep timing
Watchdog Timer (WDT)
- The WDT can be enabled or disabled by the SBC
- The WDT acts as a failsafe measure against hang states
- The most common would be a communication failure between uc and SBC
- This could lead to a polling loop race condition on the uc side
- The WDT consists of a timer that counts down to zero over about two minutes
- The WDT is reset (normally) about once every second
- Hence the timeout interval never elapses during normal operation
- In the event of a communication hang, the reset will not execute and the WDT will timeout.
- If this happens the uc will reset.
- This will also reset the SBC
- The SBC will restart and require a file system check on boot
- This approximately doubles the SBC start interval to about two minutes
- The WDT is initially disabled in the uc for two minutes after reset
- After that (in Version 3) the WDT is only enabled by the SBC, not automatically
- A future version could be "more" failsafe by having the uc self-enable the WDT
- (+) 121 seconds required to WDT timeout
- (+) WDT reset seems to restart the SBC (a good thing)
- (...?...) Watchdog restarts system off a hung uc???
Version 3 Description (Functionality / items not implemented )
- (+) Checks external voltage before powering up the SBC else sleep for 1 hour
- (-) Checksum not implemented (simple echo instead)
- (-) No self-halt on low voltage; this must be done by Agent polling
- (-) Slow-ish comm interface time (fix: careful rewrite/recalibrate countdown etc)
Limitations / Issues / Not Implemented
- Not tested: Ext-power off during operation: Does this reset the uc timing vars
- Query 32 reads AN0: Not tested
- Query 33 reads AN1: Not tested
- Not tested: After an AN0/1 query: Does voltage query work again?
- On SBC: system("date") is a cheesy DIO flush... with cheesy workaround code in updown.c
- Cassandra it would be nice to check if V-ext enable will result in Vdd < uc-V-brownout
- V3 does not have "myCaptureFailed" implemented (both Send and Receive)
- This would be done in PollB7Clear and PollB7Set
- The timeout must happen faster than a WDT timeout (120 sec)
- Need to document real expected boot times for the SBC with and without fsck
- "Ignore PORTB for 2 minutes" is implemented but could be timed better
- Agent must not expect any response during this "ignore" interval
- mySBCIsAwake state variable is neither set nor used (1100 exchange)
- myBattMonNotConnected state variable not implemented
We have an internal supply (D-cells), an external supply, and inferred current/power draws from individual components.
- Use 6V as baseline voltage (LiSO2 D-cell block Qty 2 in parallel, 1 or 2 blocks in series)
- 0.54 mA: uc Sleep (no noticeable flickers on meter for Countdown once/minute)
- 0.1 microamps: uc Operating on PCS External supply
- Use 13V as baseline voltage
- 80 mA: Power connected to PCS, slider switch OFF, uc Inactive (passive circuit drain)
- 24 mA: Vampire drain with PCS in SLEEP
- 230 mA: PCS on External Supply + SBC powered up normally
- 630 mA: Prev + WiFi Bridge + WiFi Amp (variable; this is mean-high estimate excluding spikes)
- 680 mA: To as high as 720 mA: As prev with a spike value once every 10--20 sec or so
- 500 mA: Prev - SBC disconnected
- 690 mA: PCS + SBC + WiFi + Amp + GPS mean-high estimate excluding spikes (low end is about 670)
- 740 mA: As prev with a spike value once every 10--20 sec or so
- 94 mA: PCS board: full-up power state all supplies OFF
Inferences per component
Using 13V supply voltage...
- SunSaver6 + Voltage Regulator 12V (vampires) = 24mA / 312 mA
- PCS board: 94-24 (vampire) = 70 mA / 910 mW
- SBC board: 230 - 94 = 136 mA / 1770 mW
- Amp: 120 mA / 1560 mW
- Bridge: 280 mA / 3640 mW (peak 4550, or call it 5 watts)
- GPS: 60 mA / 780 mW (no antenna connected)
- GPS: <<<>>> (antenna connected)
Please refer to the Gen 3.2 GPS page.
Facing the PCS end of the connector with red Pin 1 line at left, k is key on top of connector: Pre-dio where H = 3.3V and G is ground: k H H H H H H H H G H H H G G H H After running dio with 1010 0000 = 0xa0 we get: k H H H H H G H G G H H H G T H H with T = 2.8V Now the complement 0101 1111 = 0x5f we get: k G G G G G H G H <-- here are our 8 bits G H H H G T H H with T = 2.8V
High is a Zero and Ground is a One, these are in MSB-LSB right-to-left order. The wiring appears to be correct.
From the TS 7260 Manual:
DIO1 Port, note correspondence bits 0:7 to the above empirical (swap top and bottom rows):
GND ADC0 ADC4 DIO_8 SPI… SPI… SPI… 3.3V 2 4 6 8 10 12 14 16 1 3 5 7 9 11 13 15 DIO_0 DIO_1 DIO_2 DIO_3 DIO_4 DIO_5 DIO_6 DIO_7 LCD Port: GND Bias LCD_WR LCD_0 LCD_2 LCD_4 LCD_6 2 4 6 8 10 12 14 1 3 5 7 9 11 13 5V LCD_RS LCD_EN LCD_1 LCD_3 LCD_5 LCD_7 A/GP work properly although wired in backwards to JP9: A/GP-1 = JP9:4 A/GP-2 = JP9:3 A/GP-3 = JP9:2 A/GP-4 = JP9:1
Battery Monitor Circuit
- JP9:12 = Bat_Monitor = 2.91V into uC pin 24 = 24 RA5
- Vref = RA3 is 4.1V
- This ref voltage is now stable thanks to Bill Ashby adding two caps across U17 OUT / Gd.
- Approximate conversion equation: Voltage = 0.067 * ADC-value
Prototype and Build Boards
- P1, P2, and P3 need the MiniLynx switching voltage regulator fix
- Build board 1 has a RF-induced problem
- Build board 11 has a fault on the 5V Bridge supply (probably U11)
- The functionality diagrams simplify the schematics but (should) provide enough detail for system work.
- The firmware has a long documentation section at the top of the program.
This image is currently in-revision, to be restored asap:
ICD programming text
I am using Power Device from ICD setting and disconnecting the internal and external power from the PCS board during programming. Usually this results in one fail and then a success in two tries to program the uC.
Programming the PCS boards may rely on subtle effects. The primary of these is a good grounding connection: Ensure that the PCS board is resting on four mounting posts inside the Gen 3 enclosure. Attempts to program an isolated board sitting on the benchtop have failed.
MPLAB IDE Output window text during programming (2 tries) looks like this:
MPLAB ICD 2 Ready Programming Target... ...Validating configuration fields ...Erasing Part ...Programming Program Memory (0x0 - 0x53F) Verifying... ...Program Memory ICD0161: Verify failed (MemType = Program, Address = 0x3E, Expected Val = 0xECD4, Val Read = 0x8C90) ICD0275: Programming failed. MPLAB ICD 2 Ready Programming Target... ...Validating configuration fields ...Erasing Part ...Programming Program Memory (0x0 - 0x53F) Verifying... ...Program Memory ...Verify Succeeded Programming Configuration Bits .. Config Memory Verifying configuration memory... ...Verify Succeeded ...Programming succeeded 17-Oct-2007, 09:16:08 MPLAB ICD 2 Ready
edip switching works properly in the firmware. The value of the edip switch can be read on the SBC using ucInterface 1 5 which queries PORTE. For more on this query see this edip settings text. For the firmware please see Gen 3.2 PCS Firmware. Here is a quote:
; Op modes ; 000 All On Forever WDT Disabled Echo B7:4 to C7:4 ; 001 All peripherals on, then infinite loop timing test ; 010 WDT enabled intentional reset test ; 011 Testing rapid drop dead ; 100 Operational NO Drop Dead WDT Enabled ; 101 Operational 4-Hr Drop Dead WDT Enabled ; 110 Operational 1-Day Drop Dead WDT Enabled ; 111 Operational 7-Day Drop Dead WDT Enabled