AI News, Charla Magistral "UE artificial intelligence

Artificial neural network

Artificial neural networks (ANN) or connectionist systems are computing systems that are inspired by, but not identical to, biological neural networks that constitute animal brains.

For example, in image recognition, they might learn to identify images that contain cats by analyzing example images that have been manually labeled as 'cat' or 'no cat' and using the results to identify cats in other images.

An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain.

In ANN implementations, the 'signal' at a connection is a real number, and the output of each neuron is computed by some non-linear function of the sum of its inputs.

Signals travel from the first layer (the input layer), to the last layer (the output layer), possibly after traversing the layers multiple times.

ANNs have been used on a variety of tasks, including computer vision, speech recognition, machine translation, social network filtering, playing board and video games and medical diagnosis.

In 1970, Linnainmaa published the general method for automatic differentiation (AD) of discrete connected networks of nested differentiable functions.[15][16]

Werbos's (1975) backpropagation algorithm enabled practical training of multi-layer networks.In 1982, he applied Linnainmaa's AD method to neural networks in the way that became widely used.[10][18]

who discovered that basic perceptrons were incapable of processing the exclusive-or circuit and that computers lacked sufficient power to process useful neural networks.

Schmidhuber adopted a multi-level hierarchy of networks (1992) pre-trained one level at a time by unsupervised learning and fine-tuned by backpropagation.[23]

(2006) proposed learning a high-level representation using successive layers of binary or real-valued latent variables with a restricted Boltzmann machine[24]

Unsupervised pre-training and increased computing power from GPUs and distributed computing allowed the use of larger networks, particularly in image and visual recognition problems, which became known as 'deep learning'.[citation needed]

Between 2009 and 2012, ANNs began winning prizes in ANN contests, approaching human level performance on various tasks, initially in pattern recognition and machine learning.[28][29]

won three competitions in connected handwriting recognition in 2009 without any prior knowledge about the three languages to be learned.[32][31]

ANNs began as an attempt to exploit the architecture of the human brain to perform tasks that conventional algorithms had had little success.

They soon reoriented towards improving empirical results, mostly abandoning attempts to remain true to their biological precursors.

ANNs retained the biological concept of artificial neurons, which receive input, combine the input with their internal state (activation) and an optional threshold using an activation function, and produce output using an output function.

They can be pooling, where a group of neurons in one layer connect to a single neuron in the next layer, thereby reducing the number of neurons in that layer.[39]

The outputs are actually numbers, so when the error is low, the difference between the output (almost certainly a cat) and the correct answer (cat) is small.

A high learning rate shortens the training time, but with lower ultimate accuracy, while a lower learning rate takes longer, but with the potential for greater accuracy.

While it is possible to define a cost function ad hoc, frequently the choice is determined by the functions desirable properties (such as convexity) or because it arises from the model (e.g., in a probabilistic model the model's posterior probability can be used as an inverse cost).

commonly used cost is the mean-squared error, which tries to minimize the average squared error between the network's output and the desired output.

This can be thought of as learning with a 'teacher', in the form of a function that provides continuous feedback on the quality of solutions obtained thus far.

The cost function is dependent on the task (the model domain) and any a priori assumptions (the implicit properties of the model, its parameters and the observed variables).

)

2

whereas in statistical modeling, it could be related to the posterior probability of the model given the data (note that in both of those examples those quantities would be maximized rather than minimized).

In applications such as playing video games, an actor takes a string of actions, receiving a generally unpredictable response from the environment after each one.

In reinforcement learning, the aim is to weight the network (devise a policy) to perform actions that minimize long-term (expected cumulative) cost.

at each point in time the agent performs an action and the environment generates an observation and an instantaneous cost, according to some (usually unknown) rules.

At any juncture, the agent decides whether to explore new actions to uncover their costs or to exploit prior learning to proceed more quickly.

s

1

,

.

.

.

s

n

a

1

a

m

t

t

t

t

t

+

1

t

t

because of ANNs ability to mitigate losses of accuracy even when reducing the discretization grid density for numerically approximating the solution of control problems.

However, batch learning typically yields a faster, more stable descent to a local minimum, since each update is performed in the direction of the batch's average error.

In order to avoid oscillation inside the network such as alternating connection weights, and to improve the rate of convergence, refinements of this algorithm use an adaptive learning rate that increases or decreases as appropriate.[61]

Similar to a ball rolling down a mountain, whose current speed is determined by the current slope of the mountain and by its own inertia, inertia can be added:

depend both on the current gradient of the error function (slope of the mountain, 1st summand), as well as on the weight change from the previous point in time (inertia, 2nd summand).

Some of the main breakthroughs include: convolutional neural networks that have proven particularly successful in processing visual and other two-dimensional data;[62][63]

competitive networks such as generative adversarial networks in which multiple networks (of varying structure) compete with each other, on tasks such as winning a game[70]

The basic search algorithm is to propose a candidate model, evaluate it against a dataset and use the results as feedback to teach the NAS network.[72]

Design issues include deciding the number, type and connectedness of network layers, as well as the size of each and the connection type (full, pooling, ...).

Hyperparameters must also be defined as part of the design (they are not learned), governing matters such as how many neurons are in each layer, learning rate, step, stride, depth, receptive field and padding (for CNNs), etc.[74]

object recognition and more), sequence recognition (gesture, speech, handwritten and printed text recognition), medical diagnosis, finance[80]

the dynamics of neural circuitry arise from interactions between individual neurons and how behavior can arise from abstract neural modules that represent complete subsystems.

Studies considered long-and short-term plasticity of neural systems and their relation to learning and memory from the individual neuron to the system level.

specific recurrent architecture with rational-valued weights (as opposed to full precision real number-valued weights) has the power of a universal Turing machine,[105]

but also in statistical learning theory, where the goal is to minimize over two quantities: the 'empirical risk' and the 'structural risk', which roughly corresponds to the error over the training set and the predicted error in unseen data due to overfitting.

Supervised neural networks that use a mean squared error (MSE) cost function can use formal statistical methods to determine the confidence of the trained model.

By assigning a softmax activation function, a generalization of the logistic function, on the output layer of the neural network (or a softmax component in a component-based network) for categorical target variables, the outputs can be interpreted as posterior probabilities.

Potential solutions include randomly shuffling training examples, by using a numerical optimization algorithm that does not take too large steps when changing the network connections following an example, grouping examples in so-called mini-batches and/or introducing a recursive least squares algorithm for CMAC.[59]

Sensor neurons fire action potentials more frequently with sensor activation and muscle cells pull more strongly when their associated motor neurons receive action potentials more frequently.[108]

Other than the case of relaying information from a sensor neuron to a motor neuron, almost nothing of the principles of how information is handled by biological neural networks is known.

Alexander Dewdney commented that, as a result, artificial neural networks have a 'something-for-nothing quality, one that imparts a peculiar aura of laziness and a distinct lack of curiosity about just how good these computing systems are.

Neural networks, for instance, are in the dock not only because they have been hyped to high heaven, (what hasn't?) but also because you could create a successful net without understanding how it worked: the bunch of numbers that captures its behaviour would in all probability be 'an opaque, unreadable table...valueless as a scientific resource'.

In spite of his emphatic declaration that science is not technology, Dewdney seems here to pillory neural nets as bad science when most of those devising them are just trying to be good engineers.

argued that the brain self-wires largely according to signal statistics and therefore, a serial cascade cannot catch all major statistical dependencies.

While the brain has hardware tailored to the task of processing signals through a graph of neurons, simulating even a simplified neuron on von Neumann architecture may consume vast amounts of memory and storage.

Schmidhuber noted that the resurgence of neural networks in the twenty-first century is largely attributable to advances in hardware: from 1991 to 2015, computing power, especially as delivered by GPGPUs (on GPUs), has increased around a million-fold, making the standard backpropagation algorithm feasible for training networks that are several layers deeper than before.[115]

Furthermore, researchers involved in exploring learning algorithms for neural networks are gradually uncovering general principles that allow a learning machine to be successful.

Advocates of hybrid models (combining neural networks and symbolic approaches), claim that such a mixture can better capture the mechanisms of the human mind.[119][120]

The simplest, static types have one or more static components, including number of units, number of layers, unit weights and topology.

Project MUSE - Informatizing the Universities: Reflections on One Cuban Experience

Abstract:This article explores the current situation of Cuban universities as they face the challenges caused by the informatization of society.

Although Cuban universities began to join the digital world from a relatively early date for a country from the Global South, many obstacles, both objective and subjective, have slowed progress in this race.

The article discusses the experience at the University of Havana and asks, What are the primary obstacles that block informatization in Cuban society, and what can universities do to overcome them and make progress toward digital transformation?