Spectral Analysis

For data that is cyclical, spectral-analysis techniques sometimes do a better job of filtering noise out of sample data than does simple curve fitting or moving averages. For example, consider the monthly sales data illustrated below:

It might be difficult to notice the seasonal fluctuations at first. But fit a trend to the data that is calculated using the Lowpass filter and the seasonal fluctuations stand out.

The spectral-analysis filters (Lowpass filter, Spectral noise filter, and Frequency spectrum) all filter your data using a Fourier transform technique to allow the filter to work in the frequency domain rather than the time domain. This is much like what a spectrum analyzer does when it breaks sound waves into frequency components.

In short, the theory behind spectral concepts is that any sample of data points can be broken down into a sum of sine waves of different frequencies. If you use enough sine waves, one for every two data points, the sum will precisely match your original data. Instead of thinking about your data's value at specific points in time, you can now think about your data's spectral content over all time.

If the number of data points in your original sample is a power of two, (e.g., 16, 32, 64, 128, 256, n2, etc.), DecisionPro will calculate the frequency components using a Fast Fourier Transform (FFT) algorithm. For any other number of points, DecisionPro uses a Discrete Fourier Transform (DFT) algorithm. The DFT and FFT algorithms produce the same results; but if you have many points, more than 50, the FFT algorithm is significantly faster.

In the sales forecast example, the original sales data presented above is broken down into its frequency components to produce the following spectrum. You can create a spectrum for your data by choosing Frequency spectrum in the Analysis page of the Graph Format dialog box.

You can see that there is a large frequency component at a frequency of three cycles. This is three cycles over the period occupied by the total number of samples. In this case, there were 36 samples (three years of data). Therefore, a frequency of three is three cycles every 36 months, or one cycle every 12 months (36/3). There is another spike at a frequency of 12, which corresponds to a period of 3 months (36/12). The frequency spectrum clearly shows that there are both seasonal (12-month) and quarterly (3-month) cycles in the sales data.

The height of the spikes tell you how much each spectral component contributes to the original data. The spike at a frequency of three has a magnitude of about 2.3. This means that the seasonal component in sales accounts for a swing of plus or minus 4.6 (2*2.3). You must multiply by two because the spike heights actually represent power rather than extent. For sine waves, the power contained in the wave is half the maximum extent of the wave.

The Lowpass Filter tool allows you to specify a threshold above which all frequency components are zeroed out. In the example above, all components with a frequency greater than five were zeroed. The assumption here is that all of these components are the result of random noise. However, this also filtered out the quarterly cycles.

Another spectrum analysis tool, the Spectral noise filter, allows you to specify the magnitude of the weakest spectrum component that is considered to be a real signal as opposed to noise. All components that have a magnitude less than this threshold are zeroed. For example, specifying a threshold of 1.0 will filter out all components except those at frequencies 3 and 12. The result is shown as follows:

As you can see, there is very little true noise in the sales data. Most of the monthly fluctuations are really seasonal and quarterly cycles.

Finally, you can extend this trend another year into the future by choosing the Extend trend to option in the Analysis page of the Graph Format dialog box and specifying an extent of 48 months (36 + 12).