Random Reporting User's Guide For The GOES DCS
Random Reporting User’s Guide for the GOES DCS July 15, 2021 Prepared for the National Oceanic and Atmospheric Administration
EXECUTIVE SUMMARY NOAA’s Data Collection System, or DCS, onboard GOES satellites is capable of monitoring and reporting significant environmental events in real-time through the use of the random reporting DCP message type. Successfully implementing this type of DCP message requires coordination and cooperation between NOAA, the DCS equipment vendor, and the DCS user community. This user’s guide was first published in 1980 when the technology involved was emerging, and best practices had yet to be established. Now, 40 years later, with extensive experience operating DCS platforms transmitting random reporting messages, NOAA has revised the user’s guide to provide recommendations to DCS vendors and users alike that outline the best practices for implementing e platforms configured to transmit DCS random reporting messages. This user’s guide begins by presenting the background and general information about the DCS random reporting message type. Random reporting performance is then discussed with the help of both the analysis of, and metadata from, actual random reporting message transactions, and the information is presented in an easy-to-understand format. In depth analysis, including the theory and math behind the random reporting message type is presented in appendices for those who are interested. The recommendations made in this user’s guide include details on how to configure a DCP to transmit random reporting messages with a high probability of success, expeditious delivery of messages, and how to best use this valuable shared resource. ii
TABLE OF CONTENTS EXECUTIVE SUMMARY .ii LIST OF FIGURES .iv LIST OF TABLES .iv Chapter 1: Introduction . 1 1.1. Overview . 1 1.2. Random Reporting . 1 1.3. Why a User’s Guide? . 2 Chapter 2: Recommendation Summary . 4 Chapter 3: Operation Variations. 6 Chapter 4: Random Channel Performance . 7 4.1. Discussion . 7 4.2. Predicted Performance . 10 4.3. Measured Channel Performance . 13 Chapter 5: DCS Data Dissemination. 19 Appendix 1: Statistical Analysis Assumptions . 20 Appendix 2: Performance Analysis . 21 Appendix 3: Maximum Number of DCPs Per Channel . 31 Appendix 4: Random Delay Interval Assessment . 33 Appendix 5: Performance Analysis Algorithm . 38 References . 39 iii
LIST OF FIGURES Figure 1: Random Reporting Transaction Timeline . 7 Figure 2: Predicted Random Channel Probability of Success . 11 Figure 3: Predicted Random Channel Throughput . 12 Figure 4: Measured Random Channel Probability of Success . 155 Figure 5: Measured Random Channel Throughput . 166 Figure A1: ALOHA System Normalized Throughput . 25 Figure A2: ALOHA System Probability of Success . 25 Figure A3: DCS Random Report Probability of Success . 29 Figure A4: DCS Random Report Normalized Throughput. 29 Figure A5: Matlab Script for DCS Random Channel Performance - Part 1 . 38 Figure A6: Matlab Script for DCS Random Channel Performance - Part 2 . 39 Figure A7: Matlab Script for DCS Random Channel Performance - Part 3 . 40 LIST OF TABLES Table 1: DCS Random Reporting Channels and Current Assignments . 3 Table 2: DCS Random Reporting Transaction . 7 Table 3: Optimum Number of Repeated Messages. 13 Table 4: Number of DCP Message Sources per Channel at Ps’ 95% . 13 Table 5: Measured DCS Random Channel Transmission Parameters . 17 Table 6: Calculated DCS Random Channel Performance Parameters . 18 Table A1: Normalized Channel Loading, 𝑮𝑮′, Assuming 𝑷𝑷𝑷𝑷′ 𝟗𝟗𝟗𝟗%. 30 Table A2: Normalized Channel Loading, 𝑮𝑮′, and throughput 𝑺𝑺′ . 30 Table A3: Normalized Channel Loading, 𝑮𝑮′, and Transmission Rate, 𝝀𝝀, Messages per Second . 32 Table A5: Number of DCP Message Sources per Channel Assuming Maximum Throughput . 33 Table A6: DCS Random Channel Assignments and Transmission Rates . 36 Table A7: Individual DCP Transmission Rates . 37 Table A8: Interval Delay Simulator Results . 37 iv
Chapter 1: Introduction 1.1. Overview The National Oceanic and Atmospheric Administration (NOAA) Geostationary Operational Environmental Satellite (GOES) is operated by the National Environmental Satellite, Data and Information Service (NESDIS) division of NOAA and includes a payload that relays scientific data and telemetry data transmitted from terrestrial-based environmental monitoring stations in the western hemisphere. The payload is called the Data Collection System (DCS) and the stations are referred to as Data Collection Platforms (DCPs). The specifications that dictate how DCPs are to be operated are maintained by NOAA, and the DCS vendors and user community, in a set of standards called the GOES DCS Certification Standards (CS). The current version of the standard is Certification Standard Version 2, or CS2, and was ratified in June of 2009 (NOAA 2009). The messages transmitted from DCPs can be one of three types. The most common message type is the self-timed message. These are messages that are sent on a specific channel, at a specific time, and on a repeating schedule. Transmissions can use either 300 bits-per-second (bps) or 1200 bps data rates. Typical DCP configurations transmit self-timed messages once per hour in a 5-15 second time window. The second commonly used type of DCP message is the random reporting message that is the focus of this user’s guide. These messages occur randomly in response to a triggering event experienced by the DCP, and are most often used to respond to environmental events and relay the associated scientific data prior to the next scheduled self-timed message. They are also sent without an acknowledgement of having been received. It is this form of data message that is the subject of this user’s guide. The third DCP message type is the two-way message. This is a new form of message that is currently under development by the NESDIS DCS team. Once implemented, this message type will create a two-way link with remote DCPs for management purposes. The capability of having DCPs not only transmit messages with environmental data, but also receive command messages, is intended to enhance the transmissions the of self-timed and random reporting message types by permitting the control of DCPs via remote communications. It will be possible, for instance, to remotely re-program, re-configure or shut-off a DCP that is transmitting incorrectly and interfering with other DCPs. 1.2. Random Reporting The principal advantage of supporting the random reporting DCS message type is the opportunity to report events more quickly, and with a temporary higher frequency, than selftimed reporting permits. The principal challenges random reporting are that the report time for the message is not deterministic, and that DCPs must share the channels assigned for random reporting in a peer-to-peer user profile where no DCP is given preference. As a result, the Page 1
probability of success when receiving a random report message is limited by the existing random reporting activity on the channel. The DCS system consists of 400 kHz of spectrum with approximately 330 kilohertz (kHz) of contiguous spectrum that is divided up into discrete channels for either 300 bps or 1200 bps operation. The 300 bps channels are 750 Hertz (Hz) wide, while the 1200 bps channels are 2250 Hz wide. Random reporting can use either data rate, however, only 300 bps channels are currently assigned for random reporting, and no 1200 bps channels are assigned for random use. Channels currently configured for random reporting are listed in the left column of Table 1. The CS2 specifications require that all random reporting messages be transmitted in a maximum time of 3 seconds for 300 bps operation, and 1.5 seconds for 1200 bps operation. Besides the scientific data transmitted in a random reporting message, the CS2 standard requires all transmissions from DCPs to include formatting and synchronization information to facilitate reception. This message overhead establishes a minimum duration for a random reporting message of approximately 0.8 seconds for 300 bps operation, and approximately 0.3 seconds for 1200 bps operation. Again, it should be noted that 1200 bps random reporting is not currently configured or allocated for any DCS user. Most DCS users implement self-timed reporting as their primary method for transmitting the platform’s data. Users who do chose to implement random reporting usually do so with it assigned as the secondary form of DCS messaging. Table 1 indicates the number of DCS users on each channel with either primary or secondary assignments for random reporting, as assigned in November 2020. The table clearly shows random reporting is most often used as a secondary form of DCS messaging. 1.3. Why a User’s Guide? Random reporting is a powerful tool in the DCS system but its success depends on DCS vendors and users implementing random reporting with best practices to ensure that channels assigned for random reporting function as a shared resource that is equally available to all assigned users. These best practices were not codified in the original certification standards because they had not been established. While that may change at some point in the future, for now, operating DCS random reporting successfully remains a coordinated effort for NOAA, the DCS vendors, and the DCS users. This user’s guide explains those best practices through recommendations, summarized here in the introduction, and presented in detail in Chapter 2, Recommendation Summary. The origin of these best practices is provided by in-depth technical appendices for those curious stakeholders who wish to investigate the math and theory supporting the recommendations. The additional chapters include a discussion on the variations in how random reporting is implemented in DCPs by vendors in Chapter 3, detailed discussion of the performance of the current random reporting scheme in Chapter 4, and finally a discussion on how random reporting data and self-timed data is disseminated by NOAA in Chapter 5. Page 2
Table 1: Random Reporting Channel Assignments – October 2020 Random Channel Number 104 114 115 118 119 120 121 1 123 124 125 126 127 128 129 130 131 132 133 134 135 Number of DCPs Assigned P Primary Assignment S Secondary Assignment 15P 25S 40 0P 159S 159 0P 1338S 1338 0P 1429S 1429 0P 1565S 1565 0P 202S 202 0P 1207S 1207 0P 585S 585 0P 1086S 1086 0P 2681S 2681 20P 1402S 1422 0P 1628S 1628 0P 1159S 1159 0P 1428S 1428 0P 2899S 2899 1P 2121S 2122 2P 1380S 1382 0P 1162S 1162 50P 6S 56 0P 1131S 1131 At the time this report was written, channel 121 was experiencing interference that corrupted its recorded data. This channel was ignored for the analysis in this report. 1 Page 3
Chapter 2: Recommendation Summary The successful operation of the DCS random reporting channels requires cooperation from the DCS users. The random reporting configurations that users implement in their DCPs must conform, as closely as possible, to the optimum settings listed below. If all DCS users follow these recommendations, then NOAA can continue to provide high probabilities of success that all DCS random reporting messages will be delivered. Recommendation 1: Follow the NOAA DCS CS2 requirement for transmit duration. Section 2 of the CS2 standard requires that all random reporting messages are transmitted in no more than 3 seconds for 300 baud operation and 1.5 seconds for 1200 baud operation. Recommendation 2: After a triggering event occurs, wait a random interval of time before initiating the first random message. The interval can be implemented with a Poisson distribution utilizing a message transmission rate of no faster than once per hour, or it can be implemented with a fixed plus random interval of no shorter than 5 minutes, plus a random interval of /- 1 minute determined by a uniform distribution. The combined fixed plus random interval would therefore range from 4 to 6 minutes in length. For users with near-real-time reporting requirements, it is appropriate to consider reducing the initial delay between the triggering event and the first random message transmission, e.g., to a 2 ½ minute fixed plus a 30 second random interval. In addition, for users that reduce their self-timed message interval significantly, e.g., to 15 minutes, still desire to maintain the use of the random reporting channel, it is appropriate to reduce the random reporting delay interval between redundant messages, e.g., 2-3 minutes instead of 4-6 minutes. Recommendation 3: Send no more than three copies of the original message and separate them in time at fixed plus random intervals from the initial message, and from each other. The interval can be implemented with a Poisson distribution utilizing a message transmission rate of no faster than once per hour, or it can be implemented with a fixed plus random interval of no shorter than 5 minutes plus a random interval of /- 1 minute determined by a uniform distribution. The combined fixed plus random interval would therefore range from 4 to 6 minutes in length. Recommendation 4: Use only 300 baud for random channel operation. The use of 1200 baud random reporting is permitted by CS2, however it is not recommended to do so and, to date, no users have been assigned to operate on a random reporting channel at 1200 baud. All users implementing random reporting on their DCPs currently use 300 baud. When compared on a transaction basis, 300 baud random reporting uses less of the available DCS channel resources than 1200 baud operation. Consider that 300 baud random reporting messages can be as long as 3 seconds but 1200 baud random reporting messages must be no more than 1.5 seconds long. This would suggest that for every random message sent at 300 baud, it would be possible to send twice that number of messages if we use 1200 baud. Unfortunately, 1200 baud random channel operation requires three times the channel bandwidth that 300 baud operation Page 4
requires. So while random messages are half the duration and quadruple the data rate with 1200 baud, they require 3 times the bandwidth that 300 baud requires; i.e. in place of every 1200 bps channel, three 300 bps channels can be assigned. This yields a net disadvantage of platform utilization, and only a 45% improvement effective data rate for 1200 baud random channel operation as compared to 300 baud operation. Taking into account the overhead portion of a DCS message, the maximum number of data bytes in a 1.5 second 1200 bps is 172 as compared to 79 bytes for a 3 second 300 bps message. Therefore 3 seconds of 1200 bps transmissions (i.e. two 1.5 second messages) can deliver 344 bytes on a single channel, but three 3 second 300 bps messages on different channels (i.e. from different platforms) can deliver 237 bytes. Recommendation 5: If a DCS user reduces self-timed message intervals significantly for a particular DCP, for example, down to 5 minutes, consider terminating the use of random reporting. Many DCS users benefit from random reporting as a tool to provide data from DCPs during the 1-hour interval between standard self-timed messages. If the self-timed interval is reduced, some users have indicated they will no longer require random reporting. Since DCS channels are a valuable satellite resource, reducing number of DCPs requiring random reporting would allow NOAA to reassign some random channels for self-timed use and expand the number of available assignments for self-timed transmissions. Page 5
Chapter 3: Operation Variations There are several variations in the DCP random channel configuration parameters that vendors make available in their DCP products for DCS users. Early in 2021, vendor surveys were conducted by NOAA for this report and the results are summarized here. All of the vendors who responded indicated that they implement repeated transmissions of the original random reporting message. Three vendors permit the number of messages to be configured. The ranges varied from 0 to 99 copies with 3 copies being the default for two of the three vendor’s products. One vendor’s DCP transmits three repeated copies, while another vendor transmits only one copy per random reporting sequence. All responding vendors also reported that they can support configurable interval durations between messages and repeated copies of the message, and can range from 0 minutes to 24 hours. The shortest default setting for the interval reported by vendors is 2 minutes, with the next shortest default interval being 5 minutes. This survey question did not specifically ask about any random component to the interval, so it is not known how vendors generate randomness for the interval All vendors indicated they implement a delay after the triggered event occurs before transmitting the original random reporting message. All vendors reported that the interval between transmissions includes a random component. Page 6
Chapter 4: Random Channel Performance 4.1. Discussion When a triggering event initiates a random report through DCS, a specific sequence of steps is initiated by the DCP that will send the report. The sequence that is typically used is based on the research conducted for the original user’s guide for random reporting that was written in 1980 (NOAA 1980). While this sequence is not required by the current DCS standards it is the most widely used sequence. The sequential steps are listed in Table . They are also presented in timeline form in Figure 1 to help explain how the steps and programmed delays interact to create a complete random reporting transaction. Note that the transmission times have a duration that is much shorter than the interstitial program delays that occur between transmissions. There is currently no requirement to limit the number of transmissions in a transaction so the sequence can continue until terminated by the DCP. Table 2: DCS Random Reporting Transaction Step 0 1 2 3 4 5 6 7 8 Description Triggering event Triggering event program delay Message Transmission Transmission Program Delay Message Transmission Transmission Program Delay Message Transmission Additional Transmission Program Delays Additional Message Transmissions Execution Time (seconds) 0 sec 240-360 sec 0.75 – 3 sec 240-360 sec 0.75 – 3 sec 240-360 sec 0.75 – 3 sec 240-360 sec 0.75 – 3 sec Figure 1: Random Reporting Transaction Timeline Page 7
The first step in a random reporting transaction is a delay following the triggering event that is intended to ensure that any other DCPs experiencing the same event will not transmit their initial message at the same time. The delay includes a 5-minute fixed delay, followed by an additional random delay component of /- 1 minute. This creates a 2-minute window, centered on a delay of 5 minutes, during which the initial message will be transmitted. The transmission is step two in the transaction. Once the initial message is transmitted, redundant messages are transmitted sequentially, each after an additional delay has expired. This redundancy is an important key to the success of the random reporting channel. Even though there was a singular triggering event, a DCP can send any number of redundant messages in response to a singular, triggering event. In this guide, we designate the number of redundant messages transmitted for the singular event by the variable ‘r’. The additional delays for redundant messages are computed separately, using the same metric as was used for the initial delay. The additional redundant pairs of delays and transmissions make up steps 3 through 8 in the random reporting transaction sequence example outlined in Table 1. If, for instance, 4 redundant messages are transmitted, then the entire random reporting transaction can take as few as 963 seconds (just over 16 minutes) to complete or as many as 1452 seconds (just over 24 minutes). The actual duration depends mostly on the random delay components of each sequential program delay. It is important to note that more often than not, the message will be delivered successfully by the first transmission. The redundant transmissions are included in the sequential steps to increase the probability of successfully receiving a random reporting message. If a DCS random reporting channel had only one active DCP assigned to it, then when a triggering event occurred, the initial message and all redundant message transmissions would be received reliably with minimal errors caused only by random noise. However, as discussed in the first chapter, each random reporting channel has multiple DCPs assigned to it that can initiate a random reporting transaction at any time. This means that the transmissions from other DCPs using the channel may overlap in time and interfere with each other, preventing both messages from being received. By requiring program delays with random components between message attempts, the DCPs will be less likely to interfere again during each of their next repeat transmissions. The sequential steps used to complete a DCS random reporting transaction are designed to ensure that the probability of successfully delivering at least one of the transmitted messages associated with a triggering event is maintained at or above 95%. This is accomplished through several important characteristics of the random reporting system: The rate at which triggering events occur is expected to be infrequent. Rates on the order of once per day or once every few hours, ON AVERAGE, are expected. DCS Page 8
random reporting users should not transmit random messages at a rate that could be supported by self-timed messages. Random reporting message duration must be kept short to minimize the chance of collision between messages from independent DCPs. In fact, the current CS2 specification requires that random reporting messages transmit for no more than 3 seconds. The number of random reporting DCPs assigned to a channel must be maintained below a maximum count that will support the expected total rate of triggering events for that channel, while ensuring the desired 95% probability of successfully delivering a message. There are several parameters that are needed to understand the performance of a random reporting channel. The first parameter is the rate at which triggering events occur, and is also known as the original message transmission rate. The rate is expressed as the number of events that occur during some convenient block of time, for example one second or one hour. When a triggering event occurs in reporting scheme, the complete message transaction that results will usually include the transmission of multiple messages comprised of the initial original message and redundant versions2 of the message. This means the total message transmission rate for a random reporting DCP is higher than its original message transmission rate. There is an important restriction on the original message transmission rate that must be discussed. This restriction is necessary for the communication system to work and for the mathematical analysis to be valid. For a DCS random channel to function, the channel message rate must never be allowed to exceed the capacity of the channel. Consider the example that all of the DCPs transmit original messages of 3 second duration (with no redundancy) and that they are coordinated so that there is no idle time on the channel. In that case, the highest possible message rate for the entire channel is 1 message every 3 seconds. If, in this example, there are 100 DCPs on the channel, all equally sharing the capacity of the channel, and still coordinated, then the highest possible message rate for any individual DCP is 1 message every 300 seconds. Should one or more DCPs violate this restriction the message rate would increase beyond the capacity of the channel. This will result in collisions on the channel and invalidate an important assumption used to analyze the performance of the channel. In the actual DCS system, there are multiple DCPs all capable of sending a multi-message, random reporting transaction. As a result, the total number of messages that will be transmitted onto the channel is more than the total event-based rate of transmission. All of the messages that are sent by all of the DCPs on a particular channel place a load on that channel. We define channel loading as the sum of all attempted message transmission time durations in Many DCPs will actually take the opportunity to update the sensor data contained in the repeated messages. This ensures that the data communicated in the random report will be the most up to date data possible, even if one of the repeated messages is the one received successfully. 2 Page 9
our convenient block of time. Mathematically it is defined as the product of the total message transmission rate multiplied by the average message duration. Keep in mind that the messages used to compute the channel loading may or may not overlap causing interference when they do. Since we are interested in knowing how successful we are utilizing each random reporting channel, we can extend the analysis to determine the throughput of the channel. Consider the example of any channel supporting a message transmission rate of one message every 2 seconds. If the messages are 2 seconds long then the channel will be fully utilized if it can deliver all of those 2 second messages by sending them one after the other with no idle time in between. We define the throughput for a DCP random message channel as the product of the triggered event rate, or message transmission rate and the message duration time. This quantity can never be greater than one and is usually quite small (often less than 0.15 or 15% throughput). In the case of the DCS random reporting channels, we are using some of the channel resources to send repeated copies of the original message. This means that the DCS random channel throughput will never reach its maximum possible value of one. When all of the random transmissions in the sequence fail to be delivered, then the transaction is concluded without delivery of the random reporting message. If this happens, on average, no more than five times for every 100 triggered events, we have achieved a probability of successful message delivery for the random reporting channel of 95% or better. All of these parameters: probability of success, transmission rates, and channel loading are shown to be interrelated, and the technical details of these relationships are presented in the appendices. 4.2. Predicted Performance The performance of
The DCS system consists of 400 kHz of spectrum with approximately 330 kilohert z (kHz) of contiguous spectrum that is divided up into discrete channels for either 300 bps or 1200 bps operation. The 300 bps channels are 750 Hertz (Hz) wide, while the 1200 bps channels are 2250 Hz wide.
Start by finding out how Python generates random numbers. Type ?random to find out about scipy's random number generators. Try typing 'random.random()' a few times. Try calling it with an integer argument. Use 'hist' (really pylab.hist) to make a histogram of 1000 numbers generated by random.random. Is th
Start by finding out how Python generates random numbers. Type ?random to find out about scipy's random number generators. Try typing 'random.random()' a few times. Try calling it with an integer argument. Use 'hist' (really pylab.hist) to make a histogram of 1000 numbers generated by random.random. Is the distribution Gaussian, uniform, or .
vibration. Today, random vibration is thought of as the random motion of a structure excited by a random input. The mathematical theory of random vibration is essential to the realistic modeling of structural dynamic systems. This article summarizes the work of some key contributors to the theory of random vibration from
producing random digits is, of course, in a state of sin.” [J. von Neumann, 1951] Sinful pleasures. “If the numbers are not random, they are at least higgledy-piggledy.” [G. Marsaglia, 1984] Does it look random enough to you? “Random numbers should not be generated with a method chosen at random.
ONE-DIMENSIONAL RANDOM WALKS 1. SIMPLE RANDOM WALK Definition 1. A random walk on the integers Z with step distribution F and initial state x 2Z is a sequenceSn of random variables whose increments are independent, identically distributed random variables i with common distribution F, that is, (1) Sn
Random interface growth Stochastic PDEs Big data and random matrices Traffic flow Random tilings in random environment Optimal paths / random walks KPZ fixed point should be the universal limit under 3:2:1 scaling. This is mainly conjectural and only proved for integrable models. KPZ fixed point Tuesday talk 1 Page 14
Probability Distribution. Mean of a Discrete Random Variable. Standard Deviation of a Discrete Random Variable. Binomial Random Variable. Binomial Probability Formula. Tables of the Binomial Distribution. Mean and Standard Deviation of a Binomial Random Variable. Poisson Random Variable. Poisson Probability Formula. Hypergeome tric Random Variable.
User Guide. This guide provides an overview of @ Work Reporting, the self-service reporting platform for clients of American Express Global Commercial Payments. This guide covers navigation of the tool and the setup process for Standard and Customized reports. For information about the Standard and Customized formats, as well as the specific report templates we offer, please refer to the @ Work Reporting Guide, which can be found under Reporting Help on the @ Work Reporting home page.
