The neural network community has recently shown a considerably interest in combining multiple neural networks (NNs). Such a combination usually improves the performance over a single NN because different NNs can complement each other. To achieve improved performance, the individual NNs must be trained independently. In this paper, three NNs are trained using the Ranking Figure of Merit objective function (with different parameters) that we introduced last year. We introduce a new method of combining NNs, which we call pooled objective function. The objective function is calculated for each NN and averaged to arrive at the pooled objective function. The combined vote is the class with the best pooled objective function. It is shown that the frequently used scheme of averaging the outputs is equivalent to the pooled mean squared error.

Radial basis function (RBF) networks and their training have garnered much interest in the neural computing literature. Current training methods resemble Expectation-Maximization (EM) techniques and optimize network performance by alternately minimizing performance measures with respect to disjoint parameter sets. Since the number of RBF parameters and the training data sets may be large, efficient training methods are imperative for these networks' practical use. This paper presents two hybrid interior point (IP) training algorithms based upon solvers for linear programming (LP) problems. Presented are small-step and large-step path following algorithms which implement an alternating minimization. The small-step algorithm converges to (epsilon) -neighborhoods of locally optimal solutions with a theoretical O((root)nlog₂((root)n/(epsilon) )) iteration rate and O(n^3.5log₂((root)n/(epsilon) )) floating point operation (flop) cost. It exhibits an empirical cost of approximately O(n²log₂(1/(epsilon) )) flops where n is the dimension of an associated LP problem. The large-step algorithm exhibits a constant convergence rate and approximately O(n^1.5log₂(1/(epsilon) )) flop cost. IP/EM algorithm comparison show that the IP methods exhibit lower order computational properties, and that for some problems the large-step algorithm outperforms the EM algorithm. Also presented are results demonstrating the small-step algorithm's use for training on a system identification problem.

Several standard neural networks, including counterpropagation networks, predictive ART networks, and radial basis function networks, are based on a combination of clustering (unsupervised learning) and mapping (supervised learning). A comparison of the characteristics of these networks for pattern classification problems is presented.

With its potential for parallel computation and general applicability, the Hopfield neural network has been investigated and improved by many researchers in order to extend its usefulness to various combinatorial problems. In spite of its success in several applications within different energy function formulations, determination of the energy coefficients has been based primarily on trial and error methods since no practical and systematic way of finding good values has been available preciously, although some theoretical analyses have been presented. In this paper, we present a methodical procedure which adaptively determines the energy coefficients leading to a valid solution as the network evolves. This method directly utilizes the value of each competing term in the energy function to balance the coefficients at each stage of the computation of the network. The advantage of this method is that the system itself controls the amount of energy which each term contributes to the total energy. To demonstrate the effectiveness of this approach, the N-Queens problem (a well known example of a constraint satisfaction problem) is studied and verified. Also, an inexpensive method for computation of the new energy level of each term at each stage of iteration is described based on incremental updating.

Feed forward neural networks are often configured with multiple hidden layers. The relative simplicity of a single hidden layer may allow the use of a broader number of algorithms than could be used with multiple hidden layers. The hardware mapping may also be simplified with all networks implemented in a single hidden layer. An algorithm is presented which will collapse a network to a single hidden layer. The algorithm replaces links between hidden layers with new units whose links bypass hidden layers. These new units have link weights calculated so that the new unit approximates the replaced link's contribution to the network. Trials with a variety of configurations and data sets demonstrate the validity and effectiveness of the concept.

This work concentrates on a novel method for empirical estimation of generalization ability of neural networks. Given a set of training (and testing) data, one can choose a network architecture (number of layers, number of neurons in each layer etc.), an initialization method, and a learning algorithm to obtain a network. One measure of performance of the trained network is how closely its actual output approximates the desired output for an input that it has never seen before. Current methods provide a `number' that indicates the estimation of generalization ability of the network. However, this number provides no further information to understand the contributing factors when generalization stability is not very good. The proposed method uses a number of parameters to define generalization ability. A set of values of these parameters provides an estimate of generalization ability. In addition, a value of each parameter indicate the contribution of such factors as network architecture, initialization method, and training data set etc. Furthermore, a method has been developed to verify the validity of estimated values of the parameters.

Theoretical and computational results have demonstrated that several types of neural networks have the universal approximation property, i.e., the ability to represent any continuous function to an arbitrary degree of accuracy, given enough hidden units. However, practical considerations, such as the relative advantages of different networks for function approximation using a small to moderate number of hidden units, are not as well understood. This paper presents preliminary results of investigations into the comparison of networks using sigmoidal activation functions and networks using radial basis functions. In particular, we consider the ability of several such networks to learn mappings from the unit square to the real interval [0,1].

When the given mapping in the supervised learning of a one-layered perceptron (OLP) satisfies the positive-linear-independency (or PLI) condition, the connection matrix of this OLP can be solved in a noniterative manner. On the other hand, if the given mapping violates this PLI condition, then no connection matrix exists for this OLP no matter what learning rule we use. This latter mapping is called an illegal mapping for the OLP. For an illegal mapping, it is found that a parallelly-cascaded, two-layered perceptron (PCTLP) consisting of hard- limited neurons will generally fulfill the learning duty and provide robust recognitions. The design of this PCTLP system is derived from the PLI condition. This PCTLP system is generally a much faster learning system than the conventional 3-layered perceptron system which is cascaded in series. The reason that it is much faster is not only that it is cascaded in parallel, but also that the learning is done noniteratively in a few algebraic steps.

This paper describes an application of Principal Component Analysis to the speeding-up the learning of a Multi-Layer Perceptron (MLP). A training algorithm, called the Incremental Input Dimensionality (IID) method, is presented that is constituted by some training steps, in each of which the dimension of the principal component subspace is increased. For each training step, some training epochs (presentations of the training set), using the Back- Propagation algorithm, are performed in order to reduce the mean square error on the test set. In this way, the last training step is performed with a subspace corresponding to the assigned reconstruction error. The performance of the MLP using IID, in the case of handwritten digit classification, are reported. For our data-base a choice of a reconstruction error rate of 5% in the IID algorithm implies a maximum dimension of the principal components subspace equal to 37. In the experiments reported in this paper, Back-Propagation using IID has turned out to be faster than standard Back-Propagation with a speed-up of about 73%. Moreover, as the IID method concerns only data representation, it can be combined with other speed-up techniques for MLP learning, and can be used by other classifiers.

In this paper, we discuss some practical implications for implementing adaptable network algorithms applied to non-stationary time series problems. Two real world data sets, containing electricity load demands and foreign exchange market prices, are used to test several different methods, ranging from linear models with fixed parameters, to non-linear models which adapt both parameters and model order on-line. Training with the extended Kalman filter, we demonstrate that the dynamic model-order increment procedure of the resource allocating RBF network (RAN) is highly sensitive to the parameters of the novelty criterion. We investigate the use of system noise for increasing the plasticity of the Kalman filter training algorithm, and discuss the consequences for on-line model order selection. The results of our experiments show that there are advantages to be gained in tracking real world non-stationary data through the use of more complex adaptive models.

The use of intelligent techniques in financial market has recently become one of the most exciting and promising application areas. In this paper, a user friendly decision support system which utilizes neural networks and GAs has been developed for dealing stocks intelligently. The proposed system has been tested for a number of listed enterprises in Tokyo Stock Market. The simulation results having been obtained suggest the usefulness of the proposed system.

Neural networks were used to predict the anomalies of the time series of monthly rainfall of the Northeastern Region of Brazil. The forecasts made using a feedforward network with backpropagation algorithm from the original data were not satisfactory. We have therefore tried to combine two advanced methods, Wavelet Transform and Neural networks. Three more types of neural networks were used. The selected neural networks include the Time Delay Neural Networks (TDNN), Radial Basis Functions network and Neural Network Adaptive Wavelet. All networks were implemented in neural network simulator SNNS. The Neural Network Adaptive Wavelet was implemented by changing the standard sigmoidal nonlinearities to wavelet nonlinearities in the neurons. We compare the results obtained with unfiltered and filtered data. Using data obtained by filtering the wavelet transform coefficients significantly improved the results for all networks. The combination of TDNN with wavelet filtered data gave the best results.

Financial analysis is based on two opposing views: market efficiency theory and technical or fundamental analysis. There are three forms of market efficiency: weak form, semi-strong form and strong form. The weak form of market efficiency precludes the trends and patterns that technical analysis attempts to exploit. This work investigates whether it is possible to detect or predict patterns underlying Eurodollar futures trading. Multilayer perceptrons and radial basis functions were chosen to predict three time series based on different financial models. The findings challenge the existing efficient market theory in the case of Eurodollar futures trading.

The switching operation of nonlinear electrical loads contributes greatly to harmonic distortion in power systems. Forecasting harmonic distortion levels enables power system operators to respond with the proper corrective action, thereby enhancing system manageability. Neural networks have proved to be viable alternatives in several modeling and prediction applications, including systems in which the dynamics are chaotic. In this paper, power system harmonics are predicted using Radial Basis Function networks. This approach has certain advantages over other conventional schemes, including the potential to track the harmonics in the time-varying power system environment.

In this paper we compare the performance of nonlinear neural network techniques to those of linear filtering techniques in the prediction of time series. Specifically, we compare the results of using the nonlinear systems, known as multilayer perceptron and radial basis function neural networks, with the results obtained using the conventional linear Wiener filter, Kalman filter and Widrow-Hoff adaptive filter in predicting future values of stationary and non- stationary time series. Our results indicate the performance of each type of system is heavily dependent upon the form of the time series being predicted and the size of the system used. In particular, the linear filters perform adequately for linear or near linear processes while the nonlinear systems perform better for nonlinear processes. Since the linear systems take much less time to be developed, they should be tried prior to using the nonlinear systems when the linearity properties of the time series process are unknown.

The mobile robot can essentially have only the relative position data for the real world. However, there are many cases that the robot has to know where it is located. In those cases, the useful method is to detect landmarks in the real world and adjust its position using detected landmarks. In this point of view, it is essential to develop a mobile robot that can accomplish the path plan successfully using natural or artificial landmarks. However, artificial landmarks are often difficult to construct and natural landmarks are very complicated to detect. In this paper, the method of acquiring landmarks by using the sensor data from the mobile robot necessary for planning the path is described. The landmark we discuss here is the natural one and is composed of the compression of sensor data from the robot. The sensor data is compressed and memorized by using five layered neural network that is called a sand glass model. The input and output data that neural network should learn is the sensor data of the robot that are exactly the same. Using the intermediate output data of the network, a compressed data is obtained, which expresses a landmark data. If the sensor data is ambiguous or enormous, it is easy to detect the landmark because the data is compressed and classified by the neural network. Using the backward three layers, the compressed landmark data is expanded to original data at some level. The studied neural network categorizes the detected sensor data to the known landmark.

Many real-world problems in aerospace sensing and controls can be approached by adding to traditional analytic techniques more information found in sparse, noisy and subjective data. Such data frequently plays a part in the modeling methodology. but formalizing its role can lead to better control and documentation ofthe system, extending its future. The combination of computational intelligence (CI) tools with the traditional system can provide techniques for appropriate generalization (or lack of it) from sparse data. Noise to be modeled or interpreted may not obviously follow a familiar distribution such as uniform, exponential. Gaussian. Raleigh, Poisson or Weibull. Many subjective decisions are made in implementing challenging data as one of the familiar distributions or in determining an appropriate empirical distribution. These problems can be successfully addressed by careful combination of artificial neural systems, fuzzy or soft systems and evolutionary systems. Recommended methodology is illustrated by examples from missile system guidance and control simulations. Expert interpretation of problem scenarios is recorded and timed to provide direction for project extensions and enhanced data visualization. Commercial applications ofthese methodologies to aerospace industry decision support systems and to biomedical control applications is discussed and illustrated. Keywords: computational intelligence, missile systems, guidance and control. artificial neural networks

In this paper, we develop an optimal controller for the separately excited dc-motor. The motivating application for this new controller is electric vehicle propulsion systems, although industrial and manufacturing applications as well as consumer products can also benefit from this approach. To achieve these optimal electric motor controllers, we propose a hybrid genetic algorithm/neural network design approach. In this case, the global optimization properties of the genetic algorithm are combined with the learning and generalization abilities of neural networks to produce a smooth controller which globally minimizes some specified cost or criterion function. Simulation results indicate that such optimal controllers can significantly improve motor efficiency. In particular, for the 4000 lb hybrid-electric vehicle constructed at Wayne State University, the optimal controller produced by our hybrid genetic algorithm/neural network approach can improve the efficiency of the motor by as much as 28.7% over conventional controllers.

The dynamics of complex neural networks modeling the self-organization process in cortical maps must include the aspects of long and short-term memory. The behavior of the network is such characterized by an equation of neural activity as a fast phenomenon and an equation of synaptic modification as a slow part of the neural system. We present new methods of analyzing the dynamics of a competitive neural system with different time scales: the K- monotone system theory developed by Kamke in 1932 as a global analysis technique and the theory of singular perturbations as a local analysis method. We also show the consequences of the stability analysis on the neural net parameters.

Most of applications on neural adaptive process control are developed on back-propagation or CMAC algorithms. We present here a new approach based on a derivative of Radial Basis Function Network: The Restricted Coulomb Energy (RCE) for a parallel implementation of adaptive process control. The RCE network has been implemented on a single board based on the Zero Instruction Set Computer (ZISC-036) neural processor of IBM. The network learning consists on identification of a real second order process (DC motor with position sensor). We expose the learning and generalization phases of network, then we give simulation and experimental results.

There exist numerous examples of grasping tasks in which stable grasping in the presence of uncertainties is needed. Applications such as underwater rescue and salvage and space exploration call for grasping of poorly specific objects in unstructured environments. A fully enveloping grasp, also called a power grasp, appears well suited for stable grasping in such unstructured environments. This is because a power grasp is characterized by inherent stability, increased weight handling capability, and resilience to modeling errors. However, research into power grasps has lagged behind research of precision or fingertip grasps for a variety of reasons, as discussed briefly in this paper. The primary objective of a power grasp system is to provide a stable grasp capable of withstanding external perturbations without crushing or otherwise damaging the object grasped. The system should also satisfy additional objectives, including allowances for inaccurate object specifications and other system inaccuracies such as compliant effects in the joints due to large focus, actuator modeling errors, and force sensor noise and modeling errors. This paper reports on progress towards meeting these objectives using an intelligent control architecture consisting of both adaptive fuzzy control techniques and artificial neural networks, along with traditional kinematic and impedance control algorithms.

A hybrid neural control system incorporated with feedforward and feedback dynamics is advocated for flexible multibody structures. The proposed neural controller is designed to achieve trajectory slewing of structural member as well as vibration suppression for precision pointing capability. The feedforward path corresponds to the steady-state output of the dynamics while the feedback path stabilizes the transient-state of the motion. In the spirit of model reference adaptive control, we utilize adaptive time-delay radial basis function networks as a building block to allow the neural network to function as an indirect closed-loop controller. The horizon-of-one predictive controllers cooperatively regulates the dynamics of the nonlinear structure to follow the prespecified reference signals asymptotically. The proposed control strategy is validated in the experimental facility, called the Planar Articulating Controls Experiment which consists of a two-link flexible planar structure constrained to move over a granite table. This paper addresses the theoretical foundation of the architecture and demonstrates its applicability via a realistic structural test bed.

A neural network is used to extract the flight model of guided, short to medium range, tripod and shoulder-fired missile systems which is then integrated into a training simulator. The simulator uses injected video to replace the optical sight and is fitted with a multi-axis positioning system which senses the gunner's movement. The movement creates an image shift and affects the input data to the missile control algorithm. Accurate flight dynamics are a key to efficient training, particularly in the case of closed loop guided systems. However, flight model data is not always available, either because it is proprietary, or because it is too complex to embed in a real time simulator. A solution is to reverse engineer the flight model by analyzing the missile's response when submitted to typical input conditions. Training data can be extracted from either recorded video or from a combination of weapon and missile positioning data. The video camera can be located either on the weapon or attached to a through-sight adapter. No knowledge of the missile flight transfer function is used in the process. The data is fed to a three-layer back-propagation type neural network. The network is configured within a standard spreadsheet application and is optimized with the built-in solver functions. The structure of the network, the selected inputs and outputs, as well as training data, output data after training, and output data when embedded in the simulator are presented.

This paper presents a neurocomputing approach for solving the algebraic matrix Riccati equation. This approach is able to utilize a good initial condition to reduce the computation time in comparison to standard methods for solving the Riccati equation. The repeated solutions of closely related Riccati equations appears in homotopy algorithms to solve certain problems in fixed-architecture control. Hence, the new approach has the potential to significantly speed-up these algorithms. It also has potential applications in adaptive control. The structured neural network architecture is trained using error backpropagation based on a steepest-descent learning rule. An example is given which illustrates the advantage of utilizing a good initial condition (i.e., initial setting of the neural network synaptic weight matrix) in the structured neural network.

Derivative-based feature saliency techniques were used to define the best of 25 Laws texture features for the classification of 101 malignant mass and benign mass regions. Statistical and derivative-based saliency techniques were used to select the best size, shape, contrast, and Laws texture features for the mass model. Nine features were chosen to define the model, of which four have been used by other researchers. Using this model, the regions were classified using a multilayer perceptron neural network architecture trained with an imbalanced training set weight update algorithm to obtain an overall classification accuracy of 100 percent for the segmented malignant masses with a false-positive rates of 1.8/image. The system has shown a sensitivity of 92 percent for locating malignant ROIs. The database contained 284 images (12 bit, 100 micrometers ).

A combination of image processing with neural network sorting was conducted to demonstrate feasibility of automated cervical smear screening. Nuclei were isolated to generate a series of data points relating to the density and size of individual nuclei. This was followed by segmentation to isolate entire cells for subsequent generation of data points to bound the size of the cytoplasm. Data points were taken on as many as ten cells per image frame and included correlation against a series of filters providing size and density readings on nuclei. Additional point data was taken on nuclei images to refine size information and on whole cells to bound the size of the cytoplasm, twenty data points per assessed cell were generated. These data point sets, designated as neural tensors, comprise the inputs for training and use of a unique neural network to sort the images and identify those indicating evidence of disease. The neural network, named the Fast Analog Associative Memory, accumulates data and establishes lookup tables for comparison against images to be assessed. Six networks were trained to differentiate normal cells from those evidencing various levels abnormality that may lead to cancer. A blind test was conducted on 77 images to evaluate system performance. The image set included 31 positives (diseased) and 46 negatives (normal). Our system correctly identified all 31 positives and 41 of the negatives with 5 false positives. We believe this technology can lead to more efficient automated screening of cervical smears.

The problem of facial recognition from grey-scale video images is approached using a two- stage neural network implemented in software. The first net finds the eyes of a person and the second neural network uses an images of the area around the eyes to identify the person. In a second approach the first network is implemented in hardware using the IBM ZISC036 RBF- chip to increase processing speed. Other implementations in hardware are also discussed, and includes preprocessing using wavelet (packet) transforms.

The motive of our work is to achieve aspect angle and motion independent robust classification of relevant objects in inverse synthetic aperture radar imagery. It is required that the classification decision should incorporate an estimate of confidence in order to reject weak decisions due to critical aspect angles or unknown objects. The proposed architecture employs a cascaded combination of an unsupervised and a supervised trained neural network. The unsupervised trained Self-Organizing Feature Map is used for object segmentation by clustering a 2D feature space and the supervised multi-layer perceptron (MLP) classifier performs the object recognition based on extracted features from the segmented object. Various features characterizing the geometrical appearance and the scatterer distribution of the objects are investigated and a combination of features, which maximize the MLP classification rate, is selected. For comparison purposes a nearest neighbor classification approach is also considered. A grouping mechanism which groups together similar views of the object, reduces the degrees of freedom of the classification process by using its own adapted classifier for each group. On simulated noisy images a recognition rates over 90% for 10 different object classes has been achieved.

Although the properties of the human visual system have been studied extensively, the knowledge of its operations are usually considered only for comparison, and models based on its function are rarely utilized. Complexity and speed of computation are often the most quoted reasons for choosing not to implement such models. Nonetheless, if we are to achieve the flexibility and power of the biological visual system, researchers would be wise to continue to explore practical, yet comprehensive models based on human vision. This paper examines the first stage of the primate visual system, the retina, and how simple models of its neurons, along with their properties and interactions, can mimic what is presently believed to be some of the initial forms of visual image processing. A static model based on the previously explored ideas of shunting dynamics will be presented along with the introduction of image (photoreceptor) blurring, driven by feedback from the shunting network. Simulations are used to demonstrate the model.

This research applies statistical and artificial neural network analysis to data obtained from measurement of organic compounds in the breath of Fisher-344 rats. The Research Triangle Institute (RTI) developed a breath collection system for use with rates in order to collect and measure volatile organic compounds (VOCs) exhaled. The RTI study tested the hypothesis that VOCs, including endogenous compounds, in breath can serve as markers to exposure to various chemicals such as drugs, pesticides, or carcinogens. A comparative analysis of chromatograms showed that the administration of carbon tetrachloride dramatically altered the VOCs measured in breath; both the compounds detected and their concentrations were greatly impacted. This research demonstrated that neural network analysis and classification discriminates between exposure to carbon tetrachloride versus no exposure. It also identified the chemical compounds in rat breath that best discriminate between carbon tetrachloride exposure and either a vehicle control or no dose. For the data set analyzed, 100 percent classification accuracy was achieved in classifying cases of exposure versus no exposure. The top three marker compounds were identified. The results obtained show that neural networks can be effectively used to analyze complex chromatographic data.

Statistical discriminative analysis and neural networks were used to prove that crop/weed/soil discrimination by optical reflectance was feasible. The wavelengths selected as inputs on those neural networks were ten nanometers width, reducing the total collected radiation for the sensor. Spectral data collected from several farms having different weed populations were introduced to discriminant analysis. The best discriminant wavelengths were used to build a wavelength histogram which selected the three best spectral broadbands for broccoli/weed/soil discrimination. The broadbands were analyzed using a new single broadband discriminator index named the discriminative integration index, DII, and the DII values obtained were used to train a neural network. This paper introduces the index concept, its results and its use for minimizing artificial lightning requirements with broadband spectral measurements for broccoli/weed/soil discrimination.

The use of artificial neural networks to the channel assignment problem for cellular code- division multiple access (CDMA) telecommunications systems is considered. CDMA takes advantage of voice activity and spatial isolation because its capacity is only interference limited, unlike time-division multiple access (TDMA) and frequency-division multiple access (FDMA) where capacities are bandwidth limited. Any reduction in interference in CDMA translates linearly into increased capacity. FDMA and TDMA use a frequency reuse pattern as a method to increase capacity, while CDMA reuses the same frequency for all cells and gains a reuse efficiency by means of orthogonal codes. The latter method can improve system capacity by factors of four to six over digital TDMA or FDMA. Cellular carriers are planning to provide multiple communication services using CDMA in the next generation cellular system infrastructure. The approach of this study is the use of neural network methods for automatic and local network control, based on traffic behavior in specific cell cites and demand history. The goal is to address certain problems associated with the management of mobile and personal communication services in a cellular radio communications environment. In planning a cellular radio network, the operator assigns channels to the radio cells so that the probability of the processed carrier-to-interference ratio, CII, exceeding a predefined value is sufficiently low. The RF propagation, determined from the topography and infrastructure in the operating area, is used in conjunction with the densities of expected communications traffic to formulate interference constraints. These constraints state which radio cells may use the same code (channel) or adjacent channels at a time. The traffic loading and the number of service grades can also be used to calculate the number of required channels (codes) for each cell. The general assignment problem is the task of assigning the required number of channels to each cell such that these constraints are satisfied. This study applies and extends the Hopfield-Tank neural network models to the channel assignment problem for both uniform and non-homogeneous cellular CDMA network topologies. These models are shown to be applicable to future networks that provide multiple types of service, dynamic demand, and mobile base stations. The derived algorithms minimize energy functions representing interference constraints and traffic demand based on local data at the cell sites. The primary objectives of the approach are to increase the forward and reverse link capacities and to distribute selected management tasks at the Mobile Telecommunications Switching Office to the cell sites. The structure of the resulting neural network algorithms have the advantage of inherent parallelism and the potential for extension to a wide range of interference criteria. Two cases are considered. In the first case, traffic demands are uniform over the radio cells, while the radio cells are assumed to be a fixed hexagonal pattern. The second case corresponds to an urban cellular radio environment, where the location of the radio cells are not homogeneous and the spatial distribution of traffic demand are non-uniform.

In this paper, an improved Integrated Adaptive Fuzzy Clustering algorithm is used for true color compression, and we can get distinct pictures on VGA display. We also propose another method for YUV representation which acquires better performance.

This paper gives a preliminary study on the possibility of using RBF network to recognize 3D objects by direct projection matching with no insertion of feature extraction procedure. Two problems are studied; one is the maximum distance (in terms of rotating angles) between consecutive 2D projections of the 3D object, beyond which correct discrimination will be out of the question. The other is the way to generate the attractor locations automatically, competitive learning algorithm of auto-clustering is proposed. Experimental results of both parts are presented.

The feature space trajectory neural net is reviewed. Its advantages over other classifiers are noted; it allows use of smaller training sets, large numbers of hidden layer neurons, low on- line computational loads, higher-order decision surfaces, the ability to reject false class input (clutter) data, etc. New test results on its 3D distortion-invariant classification performance are provided using a larger object and clutter database, input object contrast differences, a new preprocessing algorithm, and a new feature space. We note the problems with other neural net classifiers that our architecture and algorithm overcomes, the use of different distance thresholds and confidence measures to improve performance, advantages of using adjunct features, and numerous new test results.

Active vision refers to a purposeful change in the camera setup to aid the processing of visual information. An important issue in using active vision is the need to represent the 3D environment in a manner that is invariant to changing camera configurations. Conventional methods require precise knowledge of various camera parameters in order to build this representation. However, these parameters are prone to calibration errors. This motivates us to explore a neural network based approach using Vector Associative Map to learn the invariant representation of 3D point targets for active vision. An efficient learning scheme is developed that is suitable for robotic implementation. The representation thus learned is also independent of the intrinsic parameters of the imaging system, making it immune to systematic calibration errors. To evaluate the effectiveness of this scheme, computer simulations were first performed using a detailed model of the University of Illinois Active Vision System (UIAVS). This is followed by an experimental verification on the actual UIAVS. Several robotic applications are then explored that utilize the invariance property of the learned representation. These applications include motion detection, active vision based robot control, robot motion planning, and saccade sequence planning.

In this paper we introduce a technique for incorporating adaptive, rotationally invariant (RI), feature extraction into the initial layer parameters of a multilayer perceptron for classifying real IR imagery. Feature extraction parameters are not usually estimated directly due to their high dimensionality but it is possible to reduce the dimensionality by constraining these parameters to a feature subspace where the parameters are restricted to a continuous RI generating functional form (e.g. a circularly symmetric radial polynomial transform.) The lower dimensional function parameters and the classification parameters can then be estimated simultaneously to minimize an overall classification error criterion. This can be considered as an extension of previous work by other authors where non-RI filter parameters, such as Gabor filter directional selectivity, were successfully tuned for feature extraction.

The analysis of stochastic time series has shown to be a problem of major importance in the last several years, especially in the field of speech and handwriting recognition. Two approaches in particular have been proposed to achieve this goal, one based on hidden Markov modeling and the other using recurrent neural networks (RNNs). An internal class representation can easily be found using an hidden Markov model (HMM) trained by an expectation maximation algorithm. This can be regarded as a major advantage of the HMM approach. Since likelihood is being maximized (ML criterion), the classifier shows less discriminant power. In contrast, an RNN can be trained including counter examples. This improves the influence of features which discriminate competing classes. A gradient descent algorithm is applied to minimize the mean square error (MMSE criterion). The initial lack of internal representation of a class leads to serious convergence problems as the sequences lengthen. A new hybrid approach to the training of RNNs is investigated, combining the advantages of the previous methods. Various samples of handwritten letters are used to adapt the RNN, in order to examine the convergence and the discriminating ability of the new algorithm.

This paper demonstrates the application of new pattern recognition techniques that can be used to characterize space objects. The feature space trajectory neural network (FST NN) was first presented by Leonard Neiberg and David P. Casasent in 1994 as a target identification tool. Kenneth H. Fielding and Dennis W. Ruck recently applied the hidden Markov model (HMM) classifier to a 3D moving light display identification problem and a target recognition problem, using time history information to improve classification results. This paper shows how the FST NN and HMM can be used to automate optical detection of space object anomalies. Time sequenced images produced by a simulation program are used for testing these anomaly detection algorithms. Two data sets are tested. The second data set is tested with various levels of shot noise. The first data set is more difficult to classify, with the best FST NN test achieving a 100% anomaly detection rate with 5% false alarm rate. FST NN and HMM tests on the second data set achieved 100% anomaly detection with no false alarms. With various levels of shot noise added, the FST NN achieves a 100% anomaly detection rate with 4% false alarm rate. A variety of Fourier features are tested with energy normalized low frequency coefficients producing the best consistent results across data sets and noise levels. A new FST NN test is presented that measures how well the order of a test sequence matches other sequences in the database. The original FST NN is based strictly on feature space distance, but when the order of the sequence is important, the new test is useful.

One of the most important factors in reducing the effect of cancerous diseases is the early diagnosis, which requires a good and a robust method. With the advancement of computer technologies and digital image processing, the development of a computer-based system has become feasible. In this paper, we introduce a new approach for the detection of cancerous cells. This approach is based on the one-class problem approach, through which the classification system need only be trained with patterns of cancerous cells. This reduces the burden of the training task by about 50%. Based on this approach, a computer-based classification system is developed, based on the Fuzzy ARTMAP neural networks. Experimental results were performed using a set of 542 patterns taken from a sample of breast cancer. Results of the experiment show 98% correct identification of cancerous cells and 95% correct identification of non-cancerous cells.

In recent years, neural networks and genetic algorithms (GAs) have been studied quite extensively by many researchers. They have been applied to various actual problems. In this paper, neural networks and GAs are applied to the computer gaming system which is the modified version of the COMMONS GAME. It is shown that neural network technology is quite helpful for making a copy of each player's strategy in game playing. It is also shown that GAs can be utilized for making the game playing exciting. Further, applications of neural networks and GAs to the familiar computer simulation games `Nobunaga's Ambition' and `Sangokushi' are briefly introduced in the final part of this paper.

Automation of machine fault diagnosis is approached using an expert network which captures human expertise in symbolic form and is refined using historical performance data. A development environment for expert networks which draws from knowledge implicit in historical data to build and refine the expert network dynamically is presented. The testbed for the design of this development environment is fault diagnosis for gas chromatographs used in detecting contaminants in soil samples. The expert knowledge capture procedure for this testbed problem and its implementation in the G2 commercial expert system package were presented at AeroSense '95. The development environment for the fault diagnosis system includes several data-assisted methods which complement the expert knowledge embedded in the expert network. The first module presented, NetMaker, automatically constructs the network in G2 from an ASCH knowledge table file. NetMedic, the second module, is a data- assisted method which is used to confirm, refine, and augment expert knowledge in order to make the knowledge table more accurate. These tools form the foundation of the expert network development environment. The basis of the expert networks developed for machine fault diagnosis is the knowledge table, a matrix of signature symptoms and machine faults related by linguistic qualifiers. The knowledge table undergoes frequent revision due to refinements from the experts, data-enhanced knowledge from NetMedic, and improved symptom extraction algorithms. NetMaker satisfies the need to easily revise the knowledge tables and incorporate them seamlessly into the G2 expert network environment. NetMedic is used to improve machine fault diagnosis by suggesting alterations to the physical architecture of the knowledge table and the associated expert network, including several non-trainable parameters. This utility discovers relationships in the sample data using statistics from historical data. The experts may then incorporate new relationships in the expert knowledge as well as confirm existing knowledge. This approach preserves the ability to retrieve the expert knowledge from the modified network.

The World Wide Web has brought the information from a distributed environment into a global information universe. As users keep on surfing the Web and adding their bookmarks, it is increasingly difficult for them to find their desired information even though there are many search tools available. In this paper, a smart engineering system called Cyber Agent is proposed to help users search and organize the information. It contains two major subsystems, namely, WebTracer and WebOrganizer. They adapt their behavior dynamically according to the environment and the special preferences of each individual. WebTracer is the wavefront of the Cyber Agent while WebOrganizer is the brain of the Cyber Agent.

An important difference between biological vision systems and their electronic counterparts is the large number of feedback signals controlling each aspect of the image collection process. For every forward path of information in the brain, from sensor to comprehension, there appears to be several neural bundles which send information back to the sensor to modify the way the information is collected. In this paper we will examine the role of such feedback signals and suggest algorithms for intelligent processing of images directly on the focal plane, using feedback. We consider first what form these signals might take and how they can be used to implement functions common to conventional image processing with the objective of moving the computation out of the digital domain and place much of its on the focal plane, or analog processing close to the focal plane. While this work falls under the general heading of artificial neural networks, it goes beyond the static processing of signals suggested by the McCulloch and Pitts model of the neuron and the Laplacian image processing suggested by Carver Mead by including the dynamics of temporal encoding in the analysis process.

Recent studies of the visual cortices of cats and monkeys has led to the development of a new class of artificial neuron models. Eckhorn and his co-workers have developed one such neuron model. They have demonstrated that the recurrent networks of Eckhorn's neurons are capable of duplicating some of the neuro-physiological phenomena observed in cat's visual cortex. We have modified Eckhorn's neuron model in a way that the resulting neuron, referred to as the pulsed coupled neuron, becomes more suitable for image processing applications than his original model. It has been shown that a single layered laterally connected pulse coupled neural network (PCNN) is capable of smoothing, segmenting digital images. This paper describes an iterative segmentation scheme that utilizes smoothing, segmentation and feature extraction capabilities of PCNN. The knowledge driven iterative segmentation technique is powerful, flexible and has potential in real-time image processing systems.

This paper will demonstrate that the PCNN has the capability of performing logic more similar to a human approach to problem solving than a traditional computer algorithm. An example of the spaghetti strand problem is used to support this contention. The PCNN used here is a neural network with only local connections and it also is continuous in the input space. Optical implementation of such a design is feasible and presented here.

Frequency-resolved optical gating (FROG) is a technique that allows the determination of the intensity and phase of ultrashort laser pulses. In FROG, a spectrogram of the pulse, the so- called FROG trace, is produced, from which the intensity and phase is then retrieved using an iterative algorithm. This algorithm performs well for all types of pulses, but it sometimes requires more than a minute to converge, and more rapid retrieval is important for many applications. It is therefore desirable to have a non-iterative computational method capable of inverting the function that relates the pulse intensity and phase to its FROG trace. In previous work, we showed that a neural network can retrieve simple pulses rapidly and directly. This original approach involved feature extraction by computing the lowest-order integral moments of the FROG trace, making it particularly sensitive to the presence of additive noise. Using parallel-processing hardware, we are now able to use FROG traces of limited size (32 X 32 pixel) without any feature extraction as input for a neural net. In addition, FROG traces of 64 X 64 pixel size, typical for experimental data, can be used in conjunction with a more noise-insensitive feature extraction method.

A compact VLSI neural processor based on the Optimization Cellular Neural Network (OCNN) has been under development to provide a wide range of support for an intelligent remote sensing microspacecraft which requires both high bandwidth communication and high- performance computing for on-board data analysis, thematic data reduction, synergy of multiple types of sensors, and other advanced smart-sensor functions. The OCNN is developed with emphasis on its capability to find global optimal solutions by using a hardware annealing method. The hardware annealing function is embedded in the network. It is a parallel version of fast mean-field annealing in analog networks, and is highly efficient in finding globally optimal solutions for cellular neural networks. The OCNN is designed to perform programmable functions for fine-grained processing with annealing control to enhance the output quality. The OCNN architecture is a programmable multi-dimensional array of neurons which are locally connected with their local neurons. Major design features of the OCNN neural processor includes massively parallel neural processing, hardware annealing capability, winner-take-all mechanism, digitally programmable synaptic weights, and multisensor parallel interface. A compact current-mode VLSI design feasibility of the OCNN neural processor is demonstrated by a prototype 5 X 5-neuroprocessor array chip in a 2-micrometers CMOS technology. The OCNN operation theory, architecture, design and implementation, prototype chip, and system applications have been investigated in detail and presented in this paper.

Automatic recognition of star fields viewed by an imaging camera has numerous applications ranging from spacecraft navigation to pointing spaceborne instruments. The usual approach is to develop an efficient algorithm for matching stars in the imager's field of view with star data recorded in an on-board catalog. The matching process requires finding a subset of the stars in the catalog that have positions and magnitudes corresponding to those of the stars in the field of view. This paper presents an Adaptive Resonance Theory 2 (ART 2) approach to the problem of star field recognition. An ART 2 neural network is used to find a subset of stars in the catalog that provides a good match to stars in the imager's field of view. A method is presented which makes training the network unnecessary because the connection weights between the neurons are prescribed.

In this paper we study the effective behavior of a single-layer perceptron that is forced to learn a noisy mapping (e.g. associations of patterns with classes). The effect of different kinds of noise on the output of the network is discussed as a function of the noise intensity. It is argued that noise induces a random-like component in the overall behavior of the perceptron which we describe in terms of independent biased random flights in the space of the weights. These random processes (one for each class) are ruled by probability distributions specified by the weights themselves. Our model is applied to the real world application of the prediction of protein secondary structures. Several observations made in this task domain are rationalized in terms of the present model that, among others, provides a link between the seeming existence of an upper bound for the prediction efficiency and the amount of noise in the mapping.

A feedforward neural network using a smart pixel spatial light modulator was realized experimentally. The smart pixel device incorporates an integrated lenslet array and on-board electronics to perform the subtraction function in the neural network.

Recently, the Pulse-Coupled Neural Network (PCNN) was introduced. This network provides a general purpose processor that has the ability to extract edges, segments, and texture from images. These extractions are quite useful for image recognition engines. This paper will introduce a more efficient means of producing results similar to the original PCNN.

Analogue implementation of Artificial Neural Networks (ANN) especially as CMOS integrated circuits show several attractive features. During the last decade, numerous works show that small size analogue ANN operate correctly. However, today the efforts are focused on real industrial size application of ANN that will require large networks. On the other hand, all of the presented implementations of ANN have been supposed to be working in ideal conditions but real applications will be subject to some global perturbations. Especially in the case of the analogue and mixed digital/analogue implementation, the behavior analysis of the neural network with perturbation conditions is thus inevitable. Unfortunately, very few papers analyze the behavior of analogue neural network with global perturbations. We have investigated modeling and experimental validation of the behavior of analogue ANN in the case of a global perturbation of the network. We have analyzed the behavior of a CMOS analogue implementation of synchronous Boltzmann Machine model when the neural circuit is subject to perturbations. The perturbations we have considered concern the supply voltage of the neural circuit and ambient temperature in which the circuit operates. In this paper we present the analysis of the behavior of the analogue implementation of synchronous Boltzmann Machine with electrical and thermal perturbations. Simulation and experimental results have been exposed.

A oscillatory model of a Hodgkin and Huxley's neuron was proposed. It was shown that the unfolding degeneracy of this system in the vicinity of homoclinic trajectory go to a cascade of period-doubling bifurcations with the subsequent transition in deterministic chaos. A method of finding of homoclinic points in a system of ODE, based on the consideration of a system of partial differential equations of lower dimension in WKB-approximation is proposed.

In this paper, we consider the detection of a random vector in the presence of additive noise. First, we point out the relationship between linear optimal quadratic detector and the principal components of the random noisy signal. Then, we implement the classical quadratic detector for random signal using an adaptive unsupervised neural network. The basic element of the neural detector is the Principal Components Analysis (PCA) network proposed by Oja and Sanger. We show that the PCA network can be viewed as bandpass filter banks for noisy signals, thus it reduces the power of noise greatly. Therefore, it can improve the performance of the classical detector. The advantages of using the neural detector instead of the classical one are: (1) on-line adaptive algorithm can be used to train the neural detector; (2) noise can be reduced greatly with properly selected number of output units of the neural network; (3) parallel processing to simultaneously compute several eigenvalues of the input signal. Stimulated by this merit, it has been pointed out that neural networks can be used for most signal processing problems based on subspace decomposition.

It is well known that Principal Components Analysis (PCA) is optimal in the sense of Mean Square Error (MSE). However, the estimation based on MSE is sensitive to noise or outliers, therefore, it is not a robust estimator. In order to get a robust estimation, absolute error criterion (L₁ norm) could be used, but it is not differentiable at the origin point; and minimax criterion (L_(infinity ) norm) could be also applied, but only for batch learning. In this paper, a cost function is proposed for robust estimation of the principal components of random variables. This cost function is rooted in the M-estimators in robust statistics. It has the form (phi) (t) equals 1-exp[-(Beta) t²]/1+exp[-(Beta) t²]. It is easy to verify that this function is an even, nonnegative, and differentiable at any value t. Hence, it overcomes the drawback of the discontinuity of the M-estimators. We derived an on- line adaptation rule for both the weights and the slope (Beta) of the cost function in a linear network. With the adjustable (Beta) , the relative position between near-linearity and the saturation can be adapted based on the given random data. Simulation results showed that the representation error of PCA is much more smoothed using the new cost function than that of the original PCA with MSE.

One of the most remarkable breakthroughs in the field of neural networks may be the invention of the back propagation (BP) method by Rumelhart et al. The back propagation method has so far contributed a lot in constructing various useful intelligent systems. However, the original BP method involves several limitations. One of the most important is that it sometimes falls into a local minimum without being able to find a global minimum of the total error function. In order to overcome limitations of the original BP method, various modified BP methods have been proposed so far. The BP method with momentum may be one of the most popular modified algorithms. It has been reported that the BP method with momentum has been applied quite successfully to various actual problems. However, despite its effectiveness, this method involves the following serious problem: `Its learning performance depends heavily upon the selection of the value of the momentum factor'. Unfortunately, it seems that there has not so far been proposed an intelligent algorithm for determining an appropriate value of the momentum factor. In this paper, we suggest that hierarchical structure stochastic automata are quite helpful for finding an appropriate value of the momentum factor of the BP method with momentum. Several computer simulation results confirm our suggestion.

Edge detection can be treated as a problem in optimization--a problem that can be solved with a Hopfield neural network. This paper describes a neural-network based model capable of producing high-quality edge imagery.

Neural networks for function approximation are the basis of many applications. Such networks often use a sigmoidal activation function (e.g. tanh) or a radial basis function (e.g. gaussian). Networks have also been developed using wavelets. In this paper, we present a neural network approximation of functions of a single variable, using sinc functions for the activation functions on the hidden units. Performance of the sinc network is compared with that of a tanh network with the same number of hidden units. The sinc network generally learns the desired input-output mapping in significantly fewer epochs, and achieves a much lower total error on the testing points. The original sinc network is based on theoretical results for function representation using the Whittaker cardinal function (an infinite series expansion in terms of sinc functions). Enhancements to the original network include improved transformation of the problem domain onto the network input domain. Further work is in progress to study the use of sinc networks for mappings in higher dimension.

This paper will present two types of neural networks applied to IC lithography: the feedforward network with backpropagation and the abductive information modeling (polynomial) network. The inputs are various desired photoresist mask patterns while the corresponding outputs are the parameters that are used to generate these patterns. Results are shown for the 1D case with three exposures made while varying different combinations of two of the three parameters, amplitude, period and phase. Tradeoffs between the two types of networks when applied to this application are discussed.

Real-world applications of computational intelligence can enhance the fault detection and identification capabilities of a missile guidance and control system. A simulation of a bank-to- turn missile demonstrates that actuator failure may cause the missile to roll and miss the target. Failure of one fin actuator can be detected using a filter and depicting the filter output as fuzzy numbers. The properties and limitations of artificial neural networks fed by these fuzzy numbers are explored. A suite of networks is constructed to (1) detect a fault and (2) determine which fin (if any) failed. Both the zero order moment term and the fin rate term show changes during actuator failure. Simulations address the following questions: (1) How bad does the actuator failure have to be for detection to occur, (2) How bad does the actuator failure have to be for fault detection and isolation to occur, (3) are both zero order moment and fine rate terms needed. A suite of target trajectories are simulated, and properties and limitations of the approach reported. In some cases, detection of the failed actuator occurs within 0.1 second, and isolation of the failure occurs 0.1 after that. Suggestions for further research are offered.

We present in this paper an adaptive linear neural network architecture called PLSNET. This network is based on partial least-squares (PLS) regression. The architecture is a modular network with stages that are associated with the desired number of PLS factors that are to be retained. PLSNET actually consists of two separate but coupled architectures, PLSNET-C for PLS calibration, and PLSNET-P for prediction (or estimation). We show that PLSNET-C can be trained by supervised learning with three standard Hebbian learning rules that extracts the PLS weight loading vectors, the regression coefficients, and the loading vectors for the univariate output component case (single target values). The PLS information that is extracted by PLSNET-C after training, i.e., three sets of synaptic weights, is used by the PLSNET-P as fixed weights (through the coupling) in its architecture. PLSNET-C can then yield predictions of the output variable given test measurements as its input. Two examples are presented, the first illustrates the typical improved predictive capability of PLSNET compared to classical least-squares, and the second shows how PLSNET can be used for parametric system identification.

Accurately predicting the number of defects in program modules is a major problem in the quality control of large software development efforts. With good estimates early in the software development cycle, software engineers can take actions to avoid or prepare for emerging quality problems. Some source code measures are closely related to the distribution of defects across program modules. Using these relationships, software engineers develop models that provide early defect content estimates. Work with neural network based models has demonstrated their utility for this purpose. In this paper, we expand upon early neural network results for predicting the number of defects in program modules.

A method for performing piecewise smooth restorations on images corrupted with high levels of noise has been developed. Based on a Markov Random Field (MRF) model, the method uses a neural network sigmoid nonlinearity between pixels in the image to produce a restoration with sharp boundaries while providing noise reduction. The model equations are solved using the Gradient Descent Gain Annealing (GDGA) method (an efficient deterministic search algorithm) that typically requires less than 200 iterations for image restoration when implemented as a digital computer simulation. A novel feature of the GDGA method is that it automatically develops an annealing schedule by adaptively selecting the scale step size during iteration. The algorithm is able to restore images that have up to 71% of their pixels corrupted with non-Gaussian sensor noise. Results from simulations indicate that the MRF-based restoration remains useful at signal-to-noise ratios 5 to 6 dB lower than with the more commonly used median-filtering technique. These results are among the first such quantitative results in the literature.

We briefly review the use of the wavelet and wavelet packet transforms. We describe their application to signal processing as devices for feature extraction and reduction of data for neural networks, in particular their implementation in hardware for signal identification.

Society relies on telecommunications to such an extent that telecommunications software must have high reliability. Neural network models can be used to identify fault-prone modules for extra attention early in development, and thus reduce the risk of unexpected problems with those modules. This paper is an experience report on a model of a large telecommunications system with almost 7,000 changed modules, consisting of over 7 million lines of code in procedures. We developed a neural network model to identify fault-prone modules based on nine design product metrics. Misclassification of not fault-prone modules would incur only modest cost in terms of extra attention to those modules. Misclassification of fault-prone modules would risk unexpected problems late in the development or even after release. Changed modules were randomly divided into a fit data set and a validate data set. We simulated utilization of the fitted model with the validate data set, successfully demonstrating generalization.

The well test is one of the primary diagnostic and predictive tools used in the analysis of oil and gas wells. In these tests, a pressure recording device is placed in the well and the pressure response is recorded over time under controlled flow conditions. The interpreted results are indicators of the well's ability to flow and the damage done to the formation surrounding the wellbore during drilling and completion. The results are used for many purposes, including reservoir modeling (simulation) and economic forecasting. The first step in the analysis is the identification of the Well-Test Interpretation (WTI) model, which determines the appropriate solution method. Mis-identification of the WTI model occurs due to noise and non-ideal reservoir conditions. Previous studies have shown that a feed-forward neural network using the backpropagation algorithm can be used to identify the WTI model. One of the drawbacks to this approach is, however, training time, which can run into days of CPU time on personal computers. In this paper a similar neural network is applied using both a personal computer and a neurocomputer. Input data processing, network design, and performance are discussed and compared. The results show that the neurocomputer greatly eases the burden of training and allows the network to outperform a similar network running on a personal computer.

This work, investigated the use of Neural Network technology to simulate faults and to generate input/output patterns used to diagnose electronic circuits via pattern classification. There are several types of circuits (i.e., digital, analog, hybrid (digit-analog), RF, and microwave). This study focused on digital circuits while maintaining the posture of considering other types in the future with similar solutions. The main focus was to investigate a methodology to model complex digital components using system identification neural network architectures. Using those components in software, a digital circuit was assembled. Faults indicating stuck at 1 or 0 was propagated through the circuit (one at a time). Input and output sequences were combined for each situations modeled and those sequences were classified to the known modeled behavior using the Adaptive Resonance Theory neural network algorithm. In addition, a data reduction methodology was established to generate input patterns required for the recognition scheme.

HISPARS is devoted to the assessment of the value of neural networks for military pattern recognition applications including the development of a software framework and of three demonstrators. Different military scenario offer the possibility of evaluation of ANN compared to classical processing on real-world data as well synthetic data.

An attempts has been made to establish a discrete-time neuron model with a radial basis function. The neuron is utilized to build RBF-networks with locally distributed dynamics to identify input/output models of dynamic nonlinear processes. The adaptation algorithm which ascertains the optimal network parameters is provided. Further, an enhanced parameter estimation algorithm is derived, the so-called compound estimation procedure, which combines elaborated least squares techniques to highly decrease the training times. The proposed neural model is applied to identify black-box models of a turbocharging process within a Diesel engine. Benefits and drawbacks of the proposed neural structure are worked out.

In this paper we discuss the rigorous approach to the statistical mechanics of mean field desordered Ising model. The description of the thermodynamic behavior for the desordered systems is a formidable problem, and the mean field spin glass represents, for its complex behavior, a kind of paradigm for the study of complexity. The most advanced results in this direction have been obtained by means of the non—rigorous methods of theoretical physics, and are associated to the important discovery of spontaneous replica simmetry breaking and the Parisi ansatz. On the othe side, in order to improve the comprehension of complex systems, it is necessary to build suitable rigorous methods. We explain one of these techniques, the marginal martingale approach, and compare the "theoretical physics" results with the rigorous ones. In particular, we give a rigorous proof for the existence of a functional order parameter for the Sherringhton—Kirkpatrick model, and show how, by means of an suitable assumption, the ultrametric structure of the pure state space can be found. The tools developed for this mathematical study can be used to study in a rigorous way the neural network thermodynamics. In the next two paragraphs we compare the mean field spin glass model to the ferromagnetic and the neural network models and discuss the concept of functional order parameter for the desordered systems. Then, in the fourth paragraph the marginal martingale is defined and we give a representation theorem for it. Finally, by means of the cavity ansatz we find rigorously the Parisi expression of the ultrametric topology of the pure phases space. This work has been partially supported by MURST (Italian Ministry of University and Scientific and Technological Research) and INFN (Italian National Institute for Nuclear Physics). keywords spin glass, marginal martingale, neural network, ultrametric topology

In this paper, starting from a general discussion on neural network dynamics from the standpoint of statistical mechanics, we discuss three different strategies to deal with the problem of pattern recognition in neural nets. Particularly we emphasized the role of matching the intrinsic correlations within the input patterns, to solve the problem of the optimal pattern recognition. In this context, the first two strategies, we applied to different problems and we discuss in this paper, consist essentially in adding either white noise or colored noise (deterministic chaos) on the input pattern pre-processing, to make easier for a classical backpropagation algorithm the class separation, respectively because the input patterns are too correlated among themselves or, on the contrary, are too noisy. The third more radical strategy, we applied to very hard pattern recognition problems in HEP experiments, consists in an automatic (dynamic) redefinition of the same net topology on the inner correlations of the inputs.

A survey of hardware implementations of neural-network triggers in high-energy physics experiments is presented.

Neural chips now are used in the trigger devices for HEPE. Three years ago we talked the problem of using also fuzzy chip microprocessors because a fuzzy system in principle can work as a neural system and is more flexible. We made them a comparison between the two approaches and the conclusions were: fuzzy chips running at a speed suitable for trigger devices were not available on the market, therefore one should have to design his own VLSI chip while, for the neural solution, one can use commercial chips or design a dedicated VLSI chip; the fuzzy solution requires an expert to develop the fuzzy system, that is the rules, while the neural solution requires a training phase; the fuzzy solution is more flexible because you known its knowledge basis and you can improve on-line the related performances by changing the rules. To day this situation is improved because there are SW tools, called Rule Generators, able to develop a fuzzy system by means of Neural Network or Genetic Algorithms. This paper starts with a comparison between Neural Networks and Fuzzy Logic with the aim to summarize the advantages of using both the HEPE trigger devices, then are described the chips already constructed or designed: a first 1 micrometers VLSI fuzzy chip with four 7 bits input and one output running at 50 Mega Fuzzy Inference per Second therefore its processing rate depends upon the fuzzy system to process; a second one, which will be sent to the foundry next march with four 7 bit inputs running at a rate of 300 ns whichever is the fuzzy system; a third one, now in design phase, with 8 - 16 inputs running at 100 - 50 MFIPS with a rule selector to further reduce the total processing speed.

The study of the human decision-making process has led the investigators to formulate formal paradigms in order to gain knowledge by examples. Among of these, neural networks are high performance paradigms to build up example-based classifiers. In cosmic ray space experiments, a large volume of information for each cosmic ray trigger are produced. Some of the data was visual in nature, other portions contained energy deposition and timing information. The data sets are amenable to conventional analysis techniques but there is no assurance that conventional techniques make full use of subtle correlations and relations amongst the detector responses. In this paper a data analysis, based on neural network classifiers, is presented in order to study cosmic ray propagation and production in the Galaxy, by means of the electron and positron signal, detected in a balloon borne experiment.

Artificial Neural Networks (Feed-Forward Neural Networks in particular) have been proposed as a tool for event classification in High Energy Physics. After the first experimentation and an impressive success, they are nowadays a standard technique, for which an engineering is hoped.

Artificial Neural Networks (ANN) are a powerful tool widely used in High Energy Physics to solve track finding and particle identification problems. A entirely new class of application is related to the problem of recovering the information lost during data taking or signal transmission. Good performance can be reached by ANN when the events are described by quite regular patterns. Such a method was used for the DELPHI luminosity monitor to recover calorimeter dead channels. A comparison with more traditional techniques is also given.

A FFNN has been used to identify five different 1-prong (tau) decays among the (tau) ⁺(tau) ^- events collected by the DELPHI experiment at LEP. The method operates with better efficiency-purity than the classical methods for the previously studied channels and allows the possibility of analyzing a new one. It has been applied to 1994 data to obtain the branching fractions of these channels. A first estimation of the systematic errors has been performed.

This paper uses a high level vision model to describe the information passing and linking within the primate visual system. Information linking schemes, such as state dependent modulation and temporal synchronization, are presented as methods the vision system uses to combine information using expectation to fill in missing information and remove unneeded information. The possibility of using linking methods derived from physiologically based theoretical models to combine current image processing techniques for pattern recognition purposes is investigated. These image processing techniques are transforms such as (but not limited to) wavelet filters, hit or miss filters, morphological filters, and difference of gausian filters. These particular filters are chosen because they simulate functions that are performed in the primate visual system. To implement the physiologically motivated linking methods, the Pulse Coupled Neural Network (PCNN) is chosen as a basic building block for the vision model which performs linking at the neuronal pulse level. Last, an image fusion network which incorporates information linking based on the PCNN is described, and initial results are presented.