This message is to inform all concerned parties that ACS FDCs 81 and 83 tripped on the night of DOY 115, sending the spacecraft to Sun Acquisition Mode, and activating the RTSs that place the observatory in low power mode (i.e. instrument power-offs/load shedding). Investigation revealed that the cause of the anomaly was due to consecutive/overlapping moon then sun interference for a duration that was greater than 1200 2 hz cycles (10 minutes), which is the FDC trip threshold value. These FDCs are currently disabled until a resolution can be defined...most likely a broadening of this 10 minute threshold. Several events were added and/or extended in order to perform the Low Power recovery steps, and by Saturday morning, the spacecraft was back to its normal configuration (except for the 2 disabled RTSs) with all of the instruments powered on again and collecting science. The TRMM Weekly report will contain more detailed information on the events, including an overview timeline similar to the one which was distributed after the previous TRMM Low Power scenario. Thank you for your patience and cooperation. DAY 115 SUN_ACQ / LOW POWER RECOVERY TIMELINE On 98-115 at 03:14, FDC Test #83 Tripped its second threshold and placed TRMM into a stable Sun_Acquisition Control mode in between real-time events. The first Threshold message was documented in real-time, but a reset command was not sent before the LOS due to required investigation and insufficient event time remaining. In conjunction, the low power RTS proceeded to perform its load-shed duties and powered off the instruments. After learning how the Observatory got to Sun_Acq, the FOT determined that the reason the threshold tripped was because consecutive sun then moon interference on the same sensor head exceeded the 10 minute duration of the FDC threshold. This was determined by referencing the ESID planning aids delivered by FDF and generated by the GSOC utility. Once the cause was found, the FOT determined that it was safe to return to Normal Mission mode and begin the process of powering on the instruments again (the FDCs 81 and 83 were disabled first). 1. FDC 83 1st Threshold at 01:42 (10 min) - R/T Event Message Attempted to send command to reset FDC 83 (/ACRSTFDCLMT COMPID=83), but did not get command to s/c in time before LOS. 2. Next Event (04:18 - 04:41) NO AOS. Initiated XMTRBLIND procedure, but it appears wrong polarity prevented Acquisition before LOS. Procedure will be modified. 3. Initiated successful XMTRBLIND and STOCFG for added event (05:22:30 - 05:32:30). TRMM verified to be in Sun Acq Mode at 05:26. 4. Verified H&S During next event (05:58 - 06:18), but no commanding or Event dumps performed due to command rate misconfiguration. 5. FOT Verified reason for entering Sun Acq using Interference Planning Aids. Decision was made at that time to transition to Nominal and then Instrument Power On, once FDC 81 and 83 Disabled. 6. Added Event (06:38 - 07:01).Verified Commanding and performed STORATECHG to 1/4 K. 7. Performed PB Event Dump at 06:45. This Dump revealed the Sun Acq sequence of events: FDC 83 1st Threshold at 01:42 (10 min) - Event Message FDC 81 1st Threshold at 02:18 (10 min) - Event Message FDC 83 2nd Threshold at 03:14 ACS Action 7 taken for Test 83 - Goto Sun Acqusition Mode TSM 16 changed Threshold at 03:14 (Sun Acq monitor) Initiates RTS #5 ACS from 4 to 1 (Normal to Sun Acq) at 03:14 RTS #5 (Calls RTS #3 - Loadshed) at 03:14 RTS #3 Starts RTS #2, Aborts ATS, Feathers the HGA, Turns OFF Transponder 1, Turns on Transponder 2 after configuring for Omnis. S/C ATS Aborted at 03:14 RTS #2 (Loadshed) Started at 03:14; Commands SPRU to PPT, Starts RTS#15, Powers ON TMI Survival Htrs. RTS #15 (Non-Ess relays Open) Started at 03:15:30 SUN_ACQ RECOVERY STEPS 1. Procedure ACGOTOEARTH started at 06:49. Earth Acq achieved at 06:51. Two quadrant ESA interference prevented Yaw Acq until 07:09 (out-of-view). 2. Procedure SSCSTARTATS started at 06:52. ATS Executing at 06:53. 3. Event Dump Revealed Yaw Acq achieved at 07:09. Normal mode achieved at 07:20. 4. XA2XMPW OFF command sent at 07:39 to turn off transmitter 2. 5. Procedure AFDC_CONFIG started at 07:41 to turn off FDC Tests 81 and 83 (because FDC #81 never reached 2nd Threshold, and didn’t want to return to Sun Acq). LOAD-SHED RECOVERY STEPS 1. LOW_POWER_RECOVERY procedure started at 08:15, complete at 08:20. 2. WPR_RFPS_RELAY procedure started at 08:17 (called by LOW_POWER procedure). Procedure complete at 08:20. 3. TSM 29 DISABLED at 08:21 (it had not been added to LOW_POWER procedure yet). 4. MPWRON procedure started at 09:06:43 MAPWR = ON at 09:08 MPWRON procedure ended at 09:08:40 MSPINUP procedure started at 09:08:58 MSPINUPCMDST = ON at 09:11:42 MSPINUP procedure ended at 09:11:43 MRCVRSON procedure started at 09:12:05 MRMCMDST = ON at 09:12:21 MRCVRSON procedure ended at 09:12:22 5. RAHPT ON command sent at 09:12:50 RA_HPT = ON at 09:13:05 6. PRSTARTON procedure started at 09:13:16 PR in HEALTH_CHK at 09:14:28 7. LPWRON procedure started at 09:18:15 LAPWR = ON at 09:18:49 LTHRESH procedure (called by LPWRON) started at 09:19:34. LTHRESH completed at 09:20:25 LPWRON completed at 09:20:26 8. XMTRBLIND procedure started for added 25 minute event (for PR TX/RX). Were not able to acquire for the event. Later PB showed the HGA track command got through, but the XPDR ON command never got through. 9. Extended 10:54 event by 15 minutes for PR. Procedure SSCRTSCFG run to disable LOS RTS # 110 for extended event. 10. PRSTARTON Procedure started at 10:56:16 and begun where previously left off. PR = STAND-BY at 10:56:48 PTXCODE103 started at 10:57:22 PRXCODE103 started at 11:06:29 PR in OBSERVATION at 11:19:42 PRSTARTON procedure ended at 11:19:43 11. PRXATTN 9db command sent at 11:20:03 12. VAPWRON command sent at 12:32:02 VAPWR = ON at 12:32:17 VOPSHTRS procedure started at 12:33:20, 8.5 W heater ON at 12:33:57 /VINSTGHTR OFF command sent at 12:35 VRADCDROP procedure started at 12:36, VRADCDOOR = OUTGAS at 12:36:55, VMTRNHTRAPWR = ON at 12:39 13. LBKGNDMDON procedure started at 12:43:26, L8BKGSNDMD = ON at 12:44:11 LBKGNDMDON procedure ended at 12:44:12 14. CPWRON procedure started at 14:24:41 CAPWR = ON at 14:25:21 Patch1 sent at 14:26:17 Patch2 sent at 14:27:03 Patch3 sent at 14:27:24 Patch4 sent at 14:27:42 Patch5 sent at 14:28:10 Patch6 sent at 14:28:49 CINSTMODE = STDBY at 14:31:42 CINSTMODE = CT at 14:32:21 CPWRON procedure ended at 14:32:23 15. /VOUTSTGHTR ON command sent at 14:32:52 16. TSM 11 Disabled at 14:51:32, RTS 3 & 5 Enabled at 14:55:23, TSM 19 & 20 Reset at 14:57 17. ACS Table 106 (FDC Configuration) dumped at 15:02 18. /VINSTGHTR ON command sent at 20:45:30 /VINSTGHTR OFF command sent on 116-11:21 /VOUTSTGHTR OFF command sent on 116-11:21:45 /VRADCDOOR OPEN command sent on 116-11:22:28 /VINFOFLGRST command sent on 116-13:36:44 /VMTRNHTRA OFF command sent on 116-15:07:15 /V8WOHTR OFF command sent on 116-15:15:50 /V15WOHTR ON command sent on 116-15:16:30 /VSOLCDOOR BUMPOPEN command sent on 116-15:22:17 /VSHSHTR OPEN on 116-16:47:38 /VSERVOMODE LOW on 116-16:48:37 /VLWIRGAIN ATTENUATE on 116-16:48:59 /VOUTGAS DISABLE on 116-19:31 RELEVANT REAL-TIME CONTACT EVENTS START STOP 115-01:38:00 115-01:58:00 (R/T FDC Event Message) 115-04:18:00 115-04:41:00 (No AOS) 115-05:22:30 115-05:32:30 (Added event (1/1K)) 115-06:38:00 115-07:01:30 (Added event (1/1K)) 115-07:34:00 115-07:54:00 115-08:12:00 115-08:24:50 115-09:05:00 115-09:25:00 115-10:00:00 115-10:25:00 (Added event No AOS - reason TBD) 115-10:51:00 115-11:26:00 (Extended event by 15 min for PR) 115-12:29:00 115-12:49:00 115-14:21:30 115-14:36:00 (Last Instrument back on) 115-14:45:00 115-15:05:00 (Final configurations complete except VIRS) SUMMARY The FOT successfully determined the cause of the Anomaly and took the appropriate steps to return the Observatory to science operations as soon as was practical. The spacecraft is currently in its nominal state, with all instruments operating efficiently. Because of the load-shed condition, it was now practical to perform VIRS outgassing and return VIRS temperatures to a more desireable value to maximize science data quality. Three actions remain for followup from an FOT perspective: determine what values to widen the FDC limits to for 81 and 83 before re-enabling; improve and practice procedures for blind acquisitions in all scenarios and data rates; Research cause for failure to command on one event, and failure to acquire during another added event.