NetMaker

Category Intelligent Software>Neural Network Systems/Tools

Abstract NetMaker (neural network simulator and designer) is freeware package for building neural networks.

It was developed primarily to support particle interaction classification in high energy and neutrino physics experiments at CERN (European Organization for Nuclear Research).

NetMaker was also the manufacturer's test-bed for experiments with various algorithms (some well known and some of the manufacturer's own ideas).

Now NetMaker has turned into quite a flexible neural network (NN) designer with many possible applications - the main idea is to have networks, pre- / post-processing and data sets split into uniformly working, task independent blocks, which can form bigger systems.

NetMaker motivation(s) --

There are many neural network (NN) packages around, some obviously bigger, with plenty of network architectures, so what was the reason(s) to make another one?

1) Most of the NN packages let you create and play with a single network at a time; NetMaker is rather job-oriented: you can have multiple networks, data sets, connect them, reconnect, apply preprocessing, or Not.

The manufacturer will continue to develop NetMaker in this direction, so you will Not see unreadable images of spaghetti of hundreds of neuron connections, but more of a flexible data flow instead.

2) Still, few packages offer automatic adjustment of the network size. NetMaker development concentrates on such architecture.

Currently, there are two (2) network models implemented: Cascade-Correlation (growing only), and MLP with the manufacturer's growing/pruning algorithm and OBS weights elimination.

3) There are various error (cost) functions that the network can minimize, Not only the standard Mean-Squared Error (MSE).

It's true, that in many cases you won't see the difference, but there are applications where this appropriate function is crucial.

NetMaker neural engine features --

1) Neural network types -

• a) MLP - feed-forward Multi-Layer Perceptron;
• b) RMLP - recurrent Multi-Layer Perceptron (back-propagation through time with teacher forcing);
• c) Cascade-Correlation.

2) Training algorithms -

• a) Standard steepest-descend with momentum term (off-line and on-line training);
• b) Conjugate gradients (+2 modifications);
• c) Quick-prop;
• d) Levenberg-Marquardt (developing, works for epoch size less than 400 events);
• e) Automated training stop;
• f) Dynamic network size adjustment: 1) Smart insertion of new hidden units; 2) Pruning of twin, dead and constant hidden units;
• g) Optimal Brain Surgeon (OBS) for weights elimination.

Note: All structure modifications are safe to the network state (No error increase should be observed).

3) Activation functions -

• a) Standard sigmoid (logistic);
• b) Zero-centered sigmoid;
• c) Hyperbolic tangent;
• d) Elliott function (+unipolar version);
• e) Arcus tangent (scaled to unipolar and bipolar);
• f) Linear;
• g) User defined.

4) Error functions -

• a) Standard MSE;
• b) Pow4 and integrated hyperbolic arcus tangent for improved sensitivity on network error distribution tails;
• c) Integrated hyperbolic tangent for training data polluted with outliers and gross errors;
• d) Various asymmetric functions for different costs of sig->bkg and bkg->sig mis-identification.

5) Preprocessing -

• a) Normalization (scaling) to Zero-mean and unitary standard deviation;
• b) QSVD / ICA transformations - elimination of redundant data dimensions, better data representation;
• c) Fast Fourier Transform (FFT) filtering.

6) Standard classification algorithms -

• a) kNN (k-Nearest Neighbors);
• b) Support Vector Machine (SVM) classification and regression based on the LIBSVM library;
• [LIBSVM is an integrated software product for Support Vector Classification, (C-SVC, nu-SVC), Regression (epsilon-SVR, nu-SVR) and distribution estimation (one-class SVM). It supports multi-class classification]
• c) Probability density estimation.

System Requirements

Contact manufacturer.

Manufacturer

• Robert Sulej
• Warsaw University of Technology
• Institute of Radioelectronics
• 00-665 Warsaw, Poland
• E-mail: Robert.Sulej@cern.ch

Manufacturer Web Site NetMaker

Price Contact manufacturer.

G6G Abstract Number 20598

G6G Manufacturer Number 104200