I realised recently that learning how to program neural networks is a major pain. To begin with, the underlying mathematics can get very hairy. Then to top it, code tutorials are not very helpful either. In particular, when I tried to look for the simplest Keras model, they were already talking about ReLUs and sigmoid activation etc. Funnily enough, while everybody started their explanations with perceptrons, nobody actually demonstrated this simplest neural network with Keras code.
A simple perceptron is capable of learning a simple linear fit. And it turns out it is trivial to do it with Keras. To my mind, that is the place one should start. So here’s the code.
from random import random import numpy as np from keras.models import Sequential from keras.layers import Dense X = [[random(), random(), random()] for _ in range(0, 100)] y = [1.3 * x + 0.5*x - 0.8*x for x in X] X = np.array(X) y = np.array(y) model = Sequential() layer = Dense(1, input_dim=3, kernel_initializer='zero') model.add(layer) model.compile(loss='mean_squared_error', optimizer='sgd') model.fit(X, y, epochs=1000, batch_size=20) print(layer.get_weights())