Data Processing Concepts

Discussion Document:
Pump Controller units, Comments on Data Processing.

Data Collection
There are three distinct areas in data collection systems of this type. The first area is the device that collects raw data as events take place in the field. The second is the mechanism by which the data is collected from a number of sites in the field and stored. The final area relates to the way in which the collected data is collated into a meaningful result. Here is an expansion on these three areas as they relate to our system.

1. The pump controllers have a single chip microprocessor which manages the operation of the pump and associated controls. This is a low-level device that can only perform a limited number of mathematical functions. It can however count a wide range of events and store them in non-volatile memory . These events may be hours and minutes elapsed, the number of cycles accumulated and suchlike. In this case it is storing all events into a sliding log file that have taken place in the last zero to four hour block. At four hour intervals, all the events counted will slide into the next adjacent four hour block. After 24 hours the oldest four hour block is erased and only the running totals are retained. In this way, the controller is storing a picture of events in detail over the last 24 hours, plus a total of all events since the plant was first activated.
   
   When a Download of the memory content is requested via the infrared link, a snapshot of all this data, is sent in a brief one-second burst. Note that the serial number for each pump controller is unique and is embedded into the software of each chip. This unique I.D. is included in each data burst. A full description of the raw data is detailed in the appendix at the end of this document.
   
   It is not practical to do too much interpretation at this point as the microprocessor has limited resources.
   
   Also, the potential for this interpretation to be changed over time to suit individual customers is reasonably high. As the software is indelibly embedded into the microprocessor at the time of manufacture, a software upgrade in the pump controller would entail the complete replacement of the microprocessor.
   
   This spawns other problems, as any system with too many different software releases present in the field becomes quite unwieldy to administer.
   
   Another consideration established at the design stage was whether or not the pump controller should be fitted with a real-time-clock. Such a device behaves like a clock and calendar and has the capacity to time and date stamp every event that takes place in the pump controller. While this feature may sound good in principle, such systems create many problems. The installer must set the date and time correctly. The unit must keep counting hours accurately – even with the power off. Daylight savings changes have to be catered for and the data to be stored consumes 4 to 5 times more memory, which increases download times, cost of the product etc. The consequence of this is that the SM96 will count a wide range of events, but it does not record actual dates and times. This is not important as it may first seem, as the laptop PC downloading this information will have a real time clock and most events can then have the times calculated retrospectively and be attached to each record downloaded.
   
2. The data collection system needed the capacity to quickly visit multiple sites and extract the log data within. This is the principal function of the SM96TERM program that is available for the pump controller.
   
   As the data is downloaded via the non-contact infrared link, the present time/date within the PC is added to the data collected. The events that took place in the last 24 hours now have a time and date attached to them. (provided of course the user has set their PC with the right time) Data integrity checks are carried out during this download process. Additional notes on these checks are detailed at the end of this document.
   
   Other items of data can be calculated in the PC software as the data is collected. For example, If the total hours of operation are known, the ‘operate time’ for each pump cycle is known and the ‘litres-per-minute rating’ of the pump is known, then flow rates for each four hour block may be calculated and added to the raw data recorded.
   
   The most convenient method of storing this data is to start with a fresh blank database at the beginning of a session and record all the data for one download as separate fields in a single database record. This means that if a field operative visits fifty homes in a single day, he returns to the office with a single Microsoft Access database file that contains fifty records within. This file is saved with an incremental filename so that if it is placed within a master hard drive at a central point, it will not overwrite any other files present. What then happens to the database files is largely up to the customer.
   
   It is unwise to put too much analysis capacity into the data collection software as it is principally a terminal program. Each customer is likely to want to do different things with the data collected and very quickly this would result in too many different versions of the terminal program being used in the field.
   
   One of the main uses of the terminal program is its use as a simple diagnostic tool for problem sites. Should a customer be complaining about tank dry or tank flood alarms happening intermittently, then a quick download of the pump controller data at the site may reveal the cause of the problem. All too often with pump installations, a problem does occur, but by the time maintenance staff can attend, the problem has resolved itself, leaving the staff to merely shrug off the report and go home. The terminal program at least gives them more to work with. Because the downloading is non-contact and the potentially hazardous voltages remain behind the perspex safety barrier, there is no problem with data collection being performed by non-technical staff.
   
3. The third area of data handling is perhaps the most interesting. This is the area where records from multiple sites may be collated for trend analysis and graphing. As yet no clear trend has surfaced on exactly what the customers want to know. The records collected by the terminal program are held in a common Microsoft Access format. This data can be imported into Access or Excel for extensive manual manipulation by local I.T. staff.
   
   It is quite feasible to produce a dedicated program that will automate the collating and graphing of the files obtained from the field, but until a clear specification of required output is defined, production of a compilation program is largely futile. Some customers may wish to plot pump wear or alarm frequency, while others may want to concentrate on mean and peak effluent flow rates. Choices must also be made on the size of the data sample to analyse. Is it appropriate to collect data from all buildings in an estate, or just a percentage and extrapolate the rest.
   
   If flow rate data were of primary importance in an estate of say 150 homes, a good picture could be obtained by choosing twenty of those homes and collect data at the same time of day on each of seven days. That would allow a flow-rate plot to be derived over forty two, four-hour blocks. Such assessments would be very useful in planning present and future processing capacity at the treatment works.
   
   This action could be automated further by using direct downloads of the sample area. An interface under development is a cell-phone module that attaches to the standard SM96 control unit which would permit systematic downloads or automatic notification of alarm conditions. The expense of such an addition may only be justified for fixed trial periods or for dealing with customers with specific complaints.
   
   A useful item of data recorded is the response to alarm conditions by the customer activating the Alarm Mute button on the underside of the controller. When a tank dry or flood alarm occurs the beeper on the controller will sound in synchronisation with the alarm lamp. When the customer hears this alarm the usually press the Alarm Mute button to stop the buzzer. This will only halt the alarm for six hours, but in the extreme, this is long enough to obtain a nights sleep. By examining the Alarm Mute data along with Alarm Data it is possible to determine how ‘aware’ the customer was of the problem by checking their response time and frequency to alarm conditions.
   
   Municipalities could be creative with the use of this data. Statistics could be gathered on the number of homes that are empty for extended periods, or the mean number of people present in each home. Indications should be present on the number of homes using indoor spa’s as these would show up as abnormal flows or level-high alarms when emptied.
   
   A better picture on the working life cycle of the pump units could be obtained by tracking hours of operation over set periods. This data could potentially locate problems in homes before the owner is aware of them.
   
   The extent to which the collected data is to be analysed will vary widely with the resources and computer skills within a municipal department. These notes should assist in deciding how far down that path they wish to go.

Data Integrity
As previously mentioned, when information is extracted from the SM96 Pump Controller some data integrity checks are carried out. Firstly it is important to note that the download process has no impact on the data stored or on the operation of the controller. The information being sent through the link is merely a copy of that which is held in memory.

As the various values in the different fields of information are transmitted, the controller adds up the grand total of all the data values sent. One of the last fields of information sent is a copy of this grand total. This is called a checksum. In the SM96 TERM program the software also adds up a total of data values sent. It then compares this value with the checksum value received and they should match. If they do (as usually happens) then a GOOD DOWNLOAD message is displayed. If an insect were to fly through the path of the infrared link midway through a download, distorting some data, then the totals would not match in the terminal program and a BAD DOWNLOAD message would appear in red, prompting the user to try another download attempt.

The identity of the record downloaded is important. Each pump controller manufactured has a unique 5 digit serial number embedded within its software. This number is also written in five squares highlighted for this purpose on the circuit board backplane of each controller. As a system is installed, this identity should be recorded against a customer name or address. If needed, some fields of information are present in the download screen of the SM96 TERM program which will allow an address to be manually recorded against a download event. It would be possible to merge this information into the file against a ‘serial number’ to ‘customer address’ data file back at the office to help correlate a downloaded record with a physical location.

This document has been provided to give a better insight into how this pump control system may be used. It should be considered by the customer that collecting field data, just because it is possible, rapidly becomes a chore and a waste of resources. However, municipalities which from time to time choose to examine the performance of their distributed pumping system, will find this data resource to be a valuable empirical tool.



Data Contained Within Each SM96 Download
 (on single pump controller systems)