The first Statistical Process Control (SPC) algorithms were developed in the first half of the 20th century and are integral to many important telecommunications and manufacturing processes that have been developed since then. SPC algorithms are necessary for functioning telecommunications systems (such as the internet) and for high-throughput manufacturing of all kinds, from cars to computer chips.

The premise behind SPC algorithms is that all production systems have sources of variation which are inherent in the process. When a product is produced, SPC algorithms are used to determine if the variation in that product falls within the expected variation that exists in the production system or is outside of that expected variation and therefore indicative of a production problem or an outside factor.

SPC algorithms are complex and have many parameters that must be “tuned” in order to perform effectively. Using  different values for each of the parameters results in different designations of when a system is “in control” versus when it is “out of control”. At Smart Data Science Solutions, we develop SPC Systems. SPC Systems are collections of SPC algorithms that are augmented by other statistical techniques such as Machine Learning. Incorporating these other techniques allows us to appropriately tune SPC algorithms to detect variation in a particular outcome of interest.


SPC Systems are powerful enough to use in cases where the expected process variation is large – such as poultry. Poultry production relies on both mechanical and biological systems, and biological systems are very noisy (i.e., have lots of expected variation). SPC Systems cut through this noise to determine whether a production process of interest is “in control” or not.

SPC Systems can be used to track many different production processes, including pathogen presence and key performance indicators.

Pathogen presence, for example Salmonella prevalence, can be measured at multiple points throughout the production process. SPC Systems can determine if Salmonella prevalence in regional operations, as measured by boot sock samples (or other method) taken in a chicken or turkey house, is within the historical norms for that region. This can be a key indicator of what the processing plant can expect to face when those birds are processed. In processing plants, SPC Systems can determine if product samples fall within a normal (or a desired) range Salmonella prevalence, which can foreshadow USDA testing results and changes to Category status, or initiate a series of safeguards to mitigate or eliminate the downstream risks (both regulatory and public health) due to Salmonella.

Key performance indicators, such as breast yield, can also be monitored with SPC Systems. These Systems will indicate when a KPI is doing exceptionally well, with the goal of learning why the performance has improved and instituting process changes to maintain that performance or mirror it in other locations. They will also indicate when the KPI is doing poorly, which could indicate that a process has changed unintentionally, or that a process needs to change in order to maintain that particular KPI.

The SPC Systems that Smart Data Science Produces utilize machine learning algorithms, which continue to learn from data as it is accumulated. This learning allows our SPC Systems to change dynamically when your processes change. If a new intervention is introduced that drastically lowers the overall Salmonella prevalence in live operations, our SPC Systems will confirm that the change is statistically significant and will adapt over time to the new “baseline”. When this baseline changes again, perhaps by the introduction of outside-sourced birds that carry a high Salmonella burden, the System will recognize this shift, even though it might be no higher than the original prevalence before it was lowered, and alert the user. Production ecosystems evolve and change over time, and our SPC Systems will track and adapt to those changes, allowing us to both validate expected changes and alert you of unexpected changes.

The most recent innovation to our SPC System is prediction. Once we have enough data to accurately model the system being observed, we can use the power of machine learning to forecast future outcomes. This allows us to assign an Outcome Score that indicates the most likely future value for the process of interest. One example of this is to generate Risk Scores for flocks that are 2 or more weeks away from the processing date, which indicate the most likely pathogen results from the meat produced from those birds. This allows producers to respond in multiple ways: they could act to avert the undesired outcomes (changing diet or medications), they could adjust the processing parameters on days that those birds are processed (changing interventions or scheduling), or they could divert product all together (for example, using the flocks with the highest Risk Scores for cooked or ready-to-eat product as opposed to raw product).

As with the general SPC Systems, our Outcome Score system can be used across many performance indicators to direct and inform future decisions.

Below are two of the SPC algorithms that make up the SPC System. They are used in tandem, like this:

control system prevelance

The algorithm above is the short-term algorithm and it picks up medium to large changes that occur in the short term (a week, in these graphs).

The algorithm below is the long-term algorithm and it picks up small to medium changes that occur over longer periods of time. For example, if your prevalence is creeping up by 1% per week, the short term algorithm won’t ever show an “alert” but the bottom one will after enough time.

control system statistic

These are the results from the Predictive SPC. They show two different Outcome Scores. The one on the left is a “Risk Score” in which higher numbers indicate a higher likelihood of the final product having Salmonella. The one on the right is a “Yield Score” in which higher numbers indicate a higher predicted yield from those locations and the lower numbers indicate a lower predicted yield.

Predictive SPC Risk_Score
Predictive SPC Yeld_Score
Questions? Give us a call: 415-797-8717