Summary of Learning of Non-Linear Distributed Neuron Network Representations

Posted: August 26th, 2021

Student’s Name

Instructor’s Name

Course

Date

Summary of Learning of Non-Linear Distributed Neuron Network Representations

In the 1980s, there existed only two algorithm methods of learning the non-linearly distributed multilayers of neuron network representations. The back-propagation method came to substitute the Boltzmann as a machine learning technique because the former initializes the weights of the established feed-forward neural networks faster as compared to other methods. On the other hand, the Boltzmann machine learning technique has been carefully modified under varied generative development models, hence impacting its initialization of the neuron network representation effectively. Notably, the Boltzmann has been consistently fine-tuned after 25 years to encompass a vastly deep learning machine of the graphical models of the binary variables.

According to Hinton, the introduction of the graphical models signified the application of variation learning via Boltzmann machine as a method of directing the generative binary variables in the academic neural nets (4). The author states that the utilization of the sigmoid belief nets of Boltzmann machine impacts learning effectively, especially when there is a sampling of hidden states of the networks from their subsequent distribution. As much as sampling may seem difficult from a series of vastly and closely connected systems, Hinton suggests sampling of the hidden states from a simpler distribution because machine learning facilitates the optimization of a varied bound of the data log generated probability (Hinton 4). Therefore, the main idea of optimizing machine learning is to maximize the likelihood of producing the training log data, equating theright posterior of the neural networks to the type of simple distribution being approximated.

Moreover, Hinton writes about another form of Boltzmann learning that is associated with contrastive divergence as a precise technique of sampling the hidden units of a neuron network (7). Despite encompassing independent data, the contrastive divergence form of Boltzmann machine still poses a significant challenge in learning since the sampling from undirected models seems quite tricky. To resolve this problem, Hinton suggests the application of wrong statistics like activity products of the generation process a means of reconstructing data from the hidden units of highly restricted connectivity (Hinton 8). Apart from that, the learning of the Boltzmann machine offers a further formation of the deep models vis-à-vis the stacking of the highly restricted neural network connectivity. Therefore, there arises the Restricted Boltzmann Machine (RBM) that is useful in sampling dense and large hidden units from data, hence stacking another form of deep RBM (Hinton 11). The whole process of stacking another RBM is active through the initialization process of back-propagating the fine-tuned feed-forward neural nets. Therefore, the composed generative model resulting from the stacked RBMs is considered a hybrid, unlike a multilayer form of Boltzmann machine due to forming a deep belief net by replacing the bottom-up inputs with top-down ones.

As a way of impacting the machine learning of the distribution representation effectively, Hinton hints on the combination of both variationaleducation with the persistent Markov chains. Such analytical narrative resulted from the assumption that the Boltzmann machine methods are accurate in predicting both the data-dependent as well as data-independent statistics (Baum 5). Therefore, the combination of these machine learning techniques is more effective since the variational machine tool is better with data-independent statistics. In contrast, Markov persistent chains are better with dependently-statistical data (Baum 5). In brief, Hinton seems apprehensive of the future learning of neural network, simply because of the inherent model flaws. Consequently, the study of machine learning presumes that a real cortical neuron cannot efficiently predict the actual value of distributed networks.

Works Cited

Baum, Leonard E. “An Inequality and Associated Maximization Technique in Statistical Estimation for Probabilistic Functions of Markov Processes.”Inequalities, vol. 3, no. 1, 1972, pp.1-8.

Hinton, Geoffrey. “Where Do Features Come From?” Department of Computer Science, University of Toronto, 2013, pp. 1-33.