Patents
Intellix A/S holds two patents, WO 99/40521 [1] and WO 99/67694 [2], protecting various aspects of the SOUL kernel. In February 1998, Intellix A/S submitted the first patent application concerning the Intellix core technology. In June 1998, Intellix A/S filed a second patent application covering other aspects of the SOUL kernel. These two patents comprise various aspects of the Intellix core technology, the SOUL kernel, and contain approximately 100 claims concerning the inventions.

The first patent application has already been accepted in a number of important countries and the patents are expected to be issued by the end of the first quarter of 2002. For these countries, Intellix A/S expects that the patent process for the second application will be concluded no later than by the end of 2002.

The SOUL Kernel
The SOUL kernel enables the software of Intellix A/S to map knowledge. Given a set of rules and examples from a decision process, the SOUL kernel generates a model that encompasses the knowledge for the decision process in question. Subsequently, a previously unprecedented situation can be presented to the model and the model will be able to generalise from the knowledge and make a decision for the given situation.

Intellix Designer and the Intellix Knowledge Server
The SOUL kernel is used for creating knowledge domains using the Intellix Designer as well as for executing knowledge domains using the Intellix Knowledge Server; see Figure 1 and Figure 2, respectively. Prior to deploying a knowledge domain, the quality of the SOUL models contained within the domain can be scrutinized using the "Document Analysis" tool provided by the Intellix Designer.

At design-time the main usage of the SOUL kernel is

• To build a mapping of the user's knowledge,
• To assist the user in the knowledge mapping process by querying the user using the "Active Knowledge Acquisition" functionality, and
• To optimise the knowledge mapping by identifying the information that influence the decision process and deactivating superfluous inputs using the "Document Optimise" functionality.

• To determine the optimal order of the questions to present to the end-user, and
• To make decisions based on the knowledge contained in the kernel.

The n-tuple classifier
The main neural network algorithm used in the SOUL kernel to obtain the functionality described above is based on in the so-called n-tuple classifier that dates back to the late 1950'ies [3]. Since the mid 1980'ies, the n-tuple classifier has been studied intensively at Risø National Laboratory from which Intellix A/S has its roots. Today there is a close collaboration between Intellix A/S and Risø.

One of the key features of the n-tuple classifier is a very fast training scheme. Whereas the training process of ordinary neural networks can be quite slow, the training of a SOUL model is so fast that a "train" command is not present at all in the Intellix Designer - the model is simply built when needed without the user noticing. The speed is obtained by taking a radically different approach to the modelling process compared to traditional neural networks: Instead of building a single "strong" model, a high number of "weak" sub-models are combined into an ensemble by a voting process. Each sub-model is characterised by a very simple design and is operating in a sub-space of the input. An ensemble is known to outperform even the strongest sub-model of the ensemble, and furthermore ensembles most often provide better performance than single models - even if the single model is "strong" and the ensemble is combined from "weak" models. A major benefit of the approach used within the SOUL kernel is that the kind of sub-models applied allows on-line learning and unlearning of examples. This is highly attractive as it provides a basis for quality measures indicating the performance of the model. These quality measures make it possible to optimise the overall model with little or no user-intervention. Similar quality measures are computationally expensive to obtain for other algorithms. The operation of a trained SOUL model is sketched in Figure 3.

Extensions of the n-tuple classifier
Research into the decision scheme of the n-tuple classifier has revealed that despite exhibiting a competitive performance the native method exhibits a number of deficiencies [4]. These deficiencies imply that the native n-tuple classifier might not provide optimal performances. From theoretical insights into the applied algorithms, researchers have proposed a number of architectural changes or extensions to the classifier that allow the system to cope with the identified deficiencies. These extensions allow the SOUL kernel to learn a decision scheme from the training examples that will be a better approximation to the optimum than what can be obtained using the native scheme. The ability of the n-tuple classifier to perform on-line learning and unlearning is a key element in performing this optimisation. The methods used to circumvent the deficiencies of the native decision scheme are covered by one of the Intellix patents.

The use of modified decision schemes adds further functionality to the n-tuple classifier as it allows the classifier to incorporate costs. Costs are used to specify the price associated with different misclassifications made by a model. By considering the costs during the training process, it is possible to train the model to avoid making expensive errors rather than "just" avoiding errors in general, along the lines of classical decision theory.

Patented Technologies in the SOUL Kernel
The first patent of Intellix A/S [1] covers the use of non-binary weights within the SOUL kernel. The usage of non-binary weights in the neural network of the SOUL kernel facilitates models to be generated that are more flexible, effectively broadening the applicability of the SOUL kernel. Furthermore, the technology covered by [1] allows the models generated by the SOUL kernel to be optimised with respect to non-trivial costs; the kernel can be applied to problems where the criticality of a possible error depends upon the decision made. This is e.g. the case in a credit application where the price of providing credit to a bad customer is usually higher than the price of rejecting a good customer.

A theoretical analysis of the n-tuple classifier [4] shows that the so-called majority-voting scheme used in the native n-tuple classifier is inadequate in some situations for making decisions. The technology covered by the second patent of Intellix A/S [2] primarily covers the modified decision scheme used within the SOUL kernel (see Figure 3), which is able to eliminate the shortcomings of traditional decision schemes in a relatively simple manner. It is worth noting that the technique of forming a decision by combining the outputs of a large number of sub-models is not unique to the SOUL kernel. Nevertheless, the unique capabilities of the SOUL kernel to perform on-line learning and unlearning of single examples are the key elements that enable the use of the superior decision scheme covered by [2].

Summary
As described in this paper, the SOUL kernel is founded on the so-called n-tuple classifier. The n-tuple classifier has a number of appealing features; particularly noteworthy is the on-line learning and unlearning abilities as these provide access to powerful quality measures that are hard to obtain for other machine learning algorithms. However, the native n-tuple classifier has some deficiencies, limiting the performance that can be obtained. Intellix A/S has investigated the deficiencies of the architecture and managed to devise extensions to the n-tuple classifier that eliminate these deficiencies. The extensions are covered by the two patents mentioned in this document and the result is a SOUL kernel that inherits the attractive properties of the native n-tuple classifier without having its limitations.

References