Day 2 Understanding Neural Network.ipynb

Kernel: Python 3 (ipykernel)

Neural Networks: Introduction

• Biological neurons and synapses have inspired the development of artificial neural networks (ANNs) in the field of machine learning. Let's explore this biological inspiration:

1. Biological Neurons:

Structure: Neurons are the basic building blocks of the nervous system in living organisms. They consist of a cell body, dendrites (input receptors), an axon (output transmitter), and synapses

• Function: Neurons communicate with each other through electrochemical signals. When a neuron receives a sufficient amount of input from its dendrites, it "fires" and sends an electrical signal down its axon.

• Activation Threshold: Neurons have an activation threshold; they only fire if the combined inputs reach a certain level.

2. Synapses:

• Connection Points: Synapses are the junctions between neurons. They allow information to pass from one neuron to the next.

• Strength of Connection: The strength of a synapse can be modified over time. If two neurons frequently communicate, the synapse between them strengthens, making future communication more efficient.

• Plasticity: Synaptic plasticity is the ability of synapses to change their strength. This is fundamental to learning and memory in biological systems.

3. Inspiration for Artificial Neural Networks (ANNs):

Mimicking Neurons:

• ANNs are composed of artificial neurons, which are mathematical models inspired by biological neurons. These neurons take inputs, apply weights, sum them up, and pass them through an activation function to produce an output.

Layers and Connectivity:

• ANNs have layers of neurons, similar to the layers of neurons in the brain. Information flows from the input layer through one or more hidden layers to the output layer.

Weights as Synaptic Strengths:

• The weights in an artificial neuron represent the strengths of the connections, similar to the synaptic strengths in biological neurons. Learning and Adaptation

• ANNs can learn from data through a process analogous to synaptic plasticity. During training, the weights are adjusted based on the errors in the predictions, allowing the network to adapt and improve its performance.

4. Advantages of Biological Inspiration:

• Parallel Processing: Like the brain, ANNs can process information in parallel, making them efficient for tasks like pattern recognition and image processing.

• Robustness and Fault Tolerance: ANNs can be robust to noisy data and can continue functioning even if some neurons or connections are damaged or missing.

5. Differences and Abstractions:

• While ANNs are inspired by biological systems, they are highly simplified abstractions. They don't capture the full complexity of biological neurons and synapses, but they provide a powerful computational framework.

• Overall, the biological inspiration of neurons and synapses has provided a valuable conceptual framework for developing artificial intelligence and machine learning algorithms. ANNs have demonstrated remarkable capabilities in tasks such as image recognition, natural language processing, and more, showcasing the effectiveness of this approach.

6. . Perceptrons:

A perceptron is the simplest form of a neural network. It's a mathematical model inspired by the way individual neurons work in the brain.

Characterstics of Perceptrons:

• Structure:A perceptron takes multiple binary inputs (0 or 1) and produces a single binary output. Each input is associated with a weight, which represents the importance of that input

• Activation Function: The weighted sum of inputs is passed through an activation function. In the original perceptron model, this function is a step function (e.g., if the weighted sum is greater than a threshold, the output is 1; otherwise, it's 0)

• Learning Rule: The perceptron learning rule is a way to adjust the weights based on the error in the output. It updates the weights to reduce the error in future predictions

• Single-Layer Model: A perceptron is a single-layer neural network. It can only learn linearly separable functions, meaning it can only classify data that can be separated by a straight line (or plane in higher dimensions).

Multi-layer Perceptrons (MLPs):

• Multi-layer Perceptrons, often referred to simply as neural networks, are a more powerful and versatile extension of perceptrons. They overcome the limitations of single-layer models and can learn complex, non-linear relationships. Here are the key characteristics of MLPs:

• Structure: An MLP consists of multiple layers of neurons, including an input layer, one or more hidden layers, and an output layer. Each neuron in a hidden layer and the output layer is connected to every neuron in the previous layer.

• Activation Functions: Unlike perceptrons, neurons in an MLP typically use non-linear activation functions (e.g., sigmoid, ReLU) which allow the network to learn non-linear relationships.

• Universal Function Approximator:

An MLP with a sufficient number of neurons in its hidden layers can approximate any continuous function. This property is known as the Universal Approximation Theorem.

• Learning Algorithms: Backpropagation is the primary learning algorithm used for training MLPs. It involves computing gradients of the loss function with respect to the weights and adjusting the weights using gradient descent or other optimization methods.

• Deep Learning: When an MLP has multiple hidden layers (more than one), it is often referred to as a deep neural network. Deep learning is a subset of machine learning that focuses on networks with many hidden layers. MLPs are the foundation of modern deep learning and have been highly successful in a wide range of applications, including image recognition, natural language processing, speech recognition, and more.

In summary, while perceptrons are simple, single-layer models with binary outputs, multi-layer perceptrons (MLPs) are more complex, multi-layer networks with non-linear activation functions that can learn complex relationships between inputs and outputs.

A neural network, also known as an artificial neural network (ANN), is a computational model inspired by the structure and functioning of the human brain. It's a fundamental concept in machine learning and deep learning.

• The neural network is simulated by a new environment.

• Then the free parameters of the neural network are changed as a result of this simulation.

• The neural network then responds in a new way to the environment because of the changes in its free parameters.

Structure of a Neural Network:

A neural network is composed of interconnected nodes, or "neurons," organized into layers.

• Neural networks extract identifying features from data, lacking pre-programmed understanding. Network components include neurons, connections, weights, biases, propagation functions, and a learning rule. Neurons receive inputs, governed by thresholds and activation functions. The three main types of layers are:

Input Layer:

• This layer contains nodes that represent the input features or variables of the problem. Each node corresponds to a feature, and the number of nodes in the input layer is equal to the number of features.

Hidden Layers:

• These layers are between the input and output layers. Each hidden layer consists of multiple nodes (neurons). Deep learning, a subset of neural networks, is characterized by the presence of multiple hidden layers.

Output Layer:

• This layer produces the final output of the neural network. The number of nodes in the output layer depends on the type of problem. For example, in binary classification, there might be one node (0 or 1), while in multi-class classification, there might be multiple nodes representing different classes.

Connections and Weights:

Every connection between nodes in adjacent layers is associated with a weight. These weights are the learnable parameters of the neural network and are adjusted during the training process.

Activation Functions:

Each node (except in the input layer) has an activation function that determines the output of the node based on the weighted sum of its inputs. Common activation functions include ReLU (Rectified Linear Unit), sigmoid, and tanh.

Training Process:

• Forward Propagation: The input data is fed forward through the network. The weights and biases are applied to the inputs, and the output of each node is computed.

• Loss Calculation: The output of the network is compared to the true target values, and a loss (or error) is calculated. This quantifies how far off the predictions are from the actual targets.

• Backpropagation: This is the heart of the training process. The gradients (derivatives of the loss with respect to the weights) are calculated by propagating the errors backward through the network. This allows for the adjustment of weights in a direction that minimizes the loss.

• Gradient Descent: The weights are updated using optimization algorithms like stochastic gradient descent (SGD). This process iteratively adjusts the weights to reduce the loss. Learning and Generalization: Through this iterative process of forward propagation, loss calculation, backpropagation, and weight adjustment, the neural network learns to make better predictions on the training data. The goal is for the network to generalize its learning to make accurate predictions on new, unseen data.

Applications:

Neural networks have found applications in a wide range of fields, including computer vision (e.g., image recognition), natural language processing (e.g., language translation), speech recognition, game playing (e.g., AlphaGo), autonomous vehicles, healthcare, and many more.

Deep Learning:

When a neural network has multiple hidden layers (more than one), it's often referred to as a deep neural network, and the process is known as deep learning. Deep learning has demonstrated remarkable capabilities in handling complex tasks and processing large amounts of data.

Python Libraries for Neural Network:

To work with neural networks in Python, you'll typically need a combination of the following libraries:

• NumPy: NumPy is a fundamental package for numerical computations in Python. It provides support for working with arrays and matrices, which are essential for handling the mathematical operations involved in neural networks.

Keras or TensorFlow (or both):

• TensorFlow: TensorFlow is a powerful open-source library for numerical computation and machine learning. It provides a comprehensive set of tools for building and training various types of neural networks.

• Keras: Keras is an open-source neural network library that acts as an interface for various backends, including TensorFlow. It provides a high-level API for building and training neural networks, making it easy to get started.

• PyTorch (Optional): PyTorch is another popular open-source deep learning library that provides dynamic computation graphs. It's known for its flexible and dynamic approach to building neural networks.

• Scikit-learn (Optional): While primarily a library for traditional machine learning, scikit-learn includes various tools for preprocessing data and evaluating the performance of machine learning models, which can be useful in conjunction with neural networks.

• Matplotlib or Seaborn (Optional): These libraries are used for data visualization, which can be helpful for understanding the performance of neural networks and visualizing results.

• Pandas (Optional): Pandas is a versatile library for data manipulation and analysis. While not directly related to neural networks, it's often used for tasks like data preprocessing.

• Jupyter Notebook (Optional): Jupyter notebooks are interactive environments that allow you to write and execute Python code in a document-style format. They are popular for experimenting with and visualizing neural network models.

• CUDA and cuDNN (Optional): If you're working with deep learning models on GPU hardware, installing CUDA (NVIDIA's parallel computing platform) and cuDNN (a GPU-accelerated library for deep neural networks) can significantly speed up computations.

Basic feedforward neural network for binary classification.

# Regression
X = np.random.rand(10000, 10)  # Features (2 input neurons)
y = 4*X+8

import numpy as np
np.random.seed(0)
X = np.random.rand(3, 2)
X

# Step 1: Import necessary libraries
import numpy as np
from keras.models import Sequential
from keras.layers import Dense

# Step 2: Generate some random data for demonstration
np.random.seed(0)
X = np.random.rand(100, 2)  # Features (2 input neurons)
y = (X[:, 0] + X[:, 1] > 1).astype(int)  # Binary target variable 

Create a Sequential model

## Step 3: Create a Sequential model
model = Sequential()

# Step 4: Add layers to the model
model.add(Dense(3, input_dim=2, activation='relu'))  # Hidden layer with 3 neurons
model.add(Dense(1, activation='sigmoid'))  # Output layer with 1 neuron (binary classification)

Accuracy = TP+TN/ (TP+TN+FP+FN)
MSE=
R2= 

# Step 5: Compile the model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Step 6: Train the model
model.fit(X, y, epochs=10, batch_size=11)

# Step 7: Evaluate the model
loss, accuracy = model.evaluate(X, y)
print(f'Loss: {loss:.4f}')
print(f'Accuracy: {accuracy*100:.2f}%')

Epoch 1/10
10/10 [==============================] - 1s 1ms/step - loss: 0.6688 - accuracy: 0.5200
Epoch 2/10
10/10 [==============================] - 0s 1ms/step - loss: 0.6678 - accuracy: 0.5200
Epoch 3/10
10/10 [==============================] - 0s 2ms/step - loss: 0.6672 - accuracy: 0.5200
Epoch 4/10
10/10 [==============================] - 0s 1ms/step - loss: 0.6667 - accuracy: 0.5200
Epoch 5/10
10/10 [==============================] - 0s 1ms/step - loss: 0.6660 - accuracy: 0.5200
Epoch 6/10
10/10 [==============================] - 0s 2ms/step - loss: 0.6656 - accuracy: 0.5200
Epoch 7/10
10/10 [==============================] - 0s 1ms/step - loss: 0.6650 - accuracy: 0.5200
Epoch 8/10
10/10 [==============================] - 0s 1ms/step - loss: 0.6644 - accuracy: 0.5200
Epoch 9/10
10/10 [==============================] - 0s 1ms/step - loss: 0.6638 - accuracy: 0.5200
Epoch 10/10
10/10 [==============================] - 0s 1ms/step - loss: 0.6633 - accuracy: 0.5200
4/4 [==============================] - 0s 1ms/step - loss: 0.6629 - accuracy: 0.5200
Loss: 0.6629
Accuracy: 52.00%

import numpy as np

# Define the sigmoid activation function and its derivative
def sigmoid(x):
    return 1 / (1 + np.exp(-x))

def sigmoid_derivative(x):
    return x * (1 - x)

# Define the neural network class
class NeuralNetwork:
    def __init__(self, input_size, hidden_size, output_size):
        # Initialize weights and biases
        self.weights_input_hidden = np.random.rand(input_size, hidden_size)
        self.weights_hidden_output = np.random.rand(hidden_size, output_size)
        self.bias_hidden = np.random.rand(1, hidden_size)
        self.bias_output = np.random.rand(1, output_size)

    def feedforward(self, inputs):
        # Calculate activations at hidden layer
        self.hidden_sum = np.dot(inputs, self.weights_input_hidden) + self.bias_hidden
        self.hidden_activation = sigmoid(self.hidden_sum)

        # Calculate activations at output layer
        self.output_sum = np.dot(self.hidden_activation, self.weights_hidden_output) + self.bias_output
        self.final_output = sigmoid(self.output_sum)

        return self.final_output

    def train(self, inputs, targets, learning_rate):
        # Calculate error at output layer
        output_error = targets - self.final_output
        d_output = output_error * sigmoid_derivative(self.final_output)

        # Calculate error at hidden layer
        hidden_error = np.dot(d_output, self.weights_hidden_output.T)
        d_hidden = hidden_error * sigmoid_derivative(self.hidden_activation)

        # Update weights and biases
        self.weights_hidden_output += np.dot(self.hidden_activation.T, d_output) * learning_rate
        self.bias_output += np.sum(d_output, axis=0, keepdims=True) * learning_rate

        self.weights_input_hidden += np.dot(inputs.T, d_hidden) * learning_rate
        self.bias_hidden += np.sum(d_hidden, axis=0, keepdims=True) * learning_rate

# Define training data
inputs = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
targets = np.array([[0], [1], [1], [0]])

# Define hyperparameters
input_size = 2
hidden_size = 4
output_size = 1
learning_rate = 0.1
epochs = 10000

# Create and train the neural network
nn = NeuralNetwork(input_size, hidden_size, output_size)

for epoch in range(epochs):
    output = nn.feedforward(inputs)
    nn.train(inputs, targets, learning_rate)

    if epoch % 1000 == 0:
        error = np.mean(np.square(targets - output))
        print(f"Epoch {epoch}, Error: {error}")

# Test the trained neural network
test_input = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
test_output = nn.feedforward(test_input)
print("\nFinal Output:")
print(test_output)

Activation Functions:

• Activation functions introduce non-linearity into the neural network. This allows the network to learn complex relationships between inputs and outputs.

Types of Activation functions:

1. Sigmoid Function:

• Range: (0, 1) - Pros: Smooth, interpretable as probabilities, historically popular.

  • Cons: Suffers from vanishing gradients problem (gradients become very small), not used much in hidden layers anymore.

    • Formula:

2. ReLU (Rectified Linear Unit)

• Range: [0, +∞)

• Pros: Fast to compute, helps with the vanishing gradients problem, widely used in hidden layers

• Cons: Can suffer from the dying ReLU problem (neurons can get stuck during training and stop learning).

  • Formula: F(x) = max(0,x)

  

3. Leaky ReLU

• Range: (-∞, +∞)

• Pros: Addresses the dying ReLU problem by allowing a small gradient for negative inputs.

• Cons: Slightly more computationally expensive than ReLU.

• Formula: f(x)=max(0.01x,x) (or a small, non-zero slope for negative values

3. Softmax (for multi-class classification)

• Range: (0, 1), and the sum of all outputs is 1.

• Purpose: Scales the outputs so that they can be interpreted as probabilities, commonly used in the output layer for multi-class classification.

• formula:

Choosing an Activation Function:

• Generally, ReLU or its variants are preferred in hidden layers due to their computational efficiency and better performance in practice.

• Sigmoid or softmax is commonly used in the output layer for binary or multi-class classification task

Loss Functions

• Loss functions quantify the error or discrepancy between the predicted output of the neural network and the actual target values during training.

Sparse Categorical Cross Entropy:

• Similar to categorical cross entropy, but when target values are integers (class labels) instead of one-hot encoded vectors

Choosing a Loss Function:

The choice depends on the nature of the problem (regression, binary classification, multi-class classification). Ensure that the chosen loss function aligns with the activation function used in the output layer. Remember, the selection of activation and loss functions can significantly impact the performance of your neural network, so it's important to experiment and choose the right combination for your specific task

Implement using Dataset for multiclasification

# Step 1: Import necessary libraries
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import to_categorical

A - 0                            A B C
B  - 1                            1 0 0
C - 2                             0 1 0
A
B

from keras.utils import to_categorical
# Step 2: Load and preprocess the dataset
iris = load_iris()
X = iris.data
y = iris.target
y_binary = np.where(y == 0, 0, 1)  # Convert to binary classification (class 0 vs. other)

# One-hot encode the binary labels
y_binary = to_categorical(y_binary)
# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y_binary, test_size=0.2, random_state=42)

feature_names = iris.feature_names
print("Feature Names:", feature_names)
import pandas as pd
Features=pd.DataFrame(X)
Features

Feature Names: ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']

# Step 3: Create a Sequential model
model = Sequential()

# Step 4: Add layers to the model
model.add(Dense(8, input_dim=4, activation='relu'))  # Hidden layer with 8 neurons
model.add(Dense(2, activation='softmax'))  # Output layer with 3 neurons (Multi-classification)

# Step 5: Compile the model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

# Step 6: Train the model
model.fit(X_train, y_train, epochs=100, batch_size=20)

Epoch 1/100
6/6 [==============================] - 1s 2ms/step - loss: 7.7697 - accuracy: 0.3333
Epoch 2/100
6/6 [==============================] - 0s 1ms/step - loss: 7.4103 - accuracy: 0.3333
Epoch 3/100
6/6 [==============================] - 0s 2ms/step - loss: 7.0391 - accuracy: 0.3333
Epoch 4/100
6/6 [==============================] - 0s 2ms/step - loss: 6.6772 - accuracy: 0.3333
Epoch 5/100
6/6 [==============================] - 0s 2ms/step - loss: 6.3281 - accuracy: 0.3333
Epoch 6/100
6/6 [==============================] - 0s 2ms/step - loss: 5.9695 - accuracy: 0.3333
Epoch 7/100
6/6 [==============================] - 0s 2ms/step - loss: 5.6136 - accuracy: 0.3333
Epoch 8/100
6/6 [==============================] - 0s 2ms/step - loss: 5.2697 - accuracy: 0.3333
Epoch 9/100
6/6 [==============================] - 0s 2ms/step - loss: 4.9183 - accuracy: 0.3333
Epoch 10/100
6/6 [==============================] - 0s 1ms/step - loss: 4.5950 - accuracy: 0.3333
Epoch 11/100
6/6 [==============================] - 0s 2ms/step - loss: 4.2501 - accuracy: 0.3333
Epoch 12/100
6/6 [==============================] - 0s 2ms/step - loss: 3.9156 - accuracy: 0.3333
Epoch 13/100
6/6 [==============================] - 0s 2ms/step - loss: 3.5905 - accuracy: 0.3333
Epoch 14/100
6/6 [==============================] - 0s 2ms/step - loss: 3.2502 - accuracy: 0.3333
Epoch 15/100
6/6 [==============================] - 0s 2ms/step - loss: 2.9400 - accuracy: 0.3333
Epoch 16/100
6/6 [==============================] - 0s 2ms/step - loss: 2.6192 - accuracy: 0.3333
Epoch 17/100
6/6 [==============================] - 0s 1ms/step - loss: 2.3193 - accuracy: 0.3333
Epoch 18/100
6/6 [==============================] - 0s 1ms/step - loss: 2.0159 - accuracy: 0.3333
Epoch 19/100
6/6 [==============================] - 0s 1ms/step - loss: 1.7270 - accuracy: 0.3333
Epoch 20/100
6/6 [==============================] - 0s 2ms/step - loss: 1.4549 - accuracy: 0.3333
Epoch 21/100
6/6 [==============================] - 0s 1ms/step - loss: 1.2097 - accuracy: 0.3333
Epoch 22/100
6/6 [==============================] - 0s 2ms/step - loss: 0.9957 - accuracy: 0.3417
Epoch 23/100
6/6 [==============================] - 0s 2ms/step - loss: 0.8047 - accuracy: 0.3833
Epoch 24/100
6/6 [==============================] - 0s 1ms/step - loss: 0.6490 - accuracy: 0.4833
Epoch 25/100
6/6 [==============================] - 0s 1ms/step - loss: 0.5293 - accuracy: 0.5750
Epoch 26/100
6/6 [==============================] - 0s 2ms/step - loss: 0.4359 - accuracy: 0.7333
Epoch 27/100
6/6 [==============================] - 0s 2ms/step - loss: 0.3660 - accuracy: 0.9250
Epoch 28/100
6/6 [==============================] - 0s 1ms/step - loss: 0.3115 - accuracy: 0.9833
Epoch 29/100
6/6 [==============================] - 0s 2ms/step - loss: 0.2697 - accuracy: 0.9917
Epoch 30/100
6/6 [==============================] - 0s 2ms/step - loss: 0.2390 - accuracy: 1.0000
Epoch 31/100
6/6 [==============================] - 0s 2ms/step - loss: 0.2141 - accuracy: 1.0000
Epoch 32/100
6/6 [==============================] - 0s 2ms/step - loss: 0.1944 - accuracy: 1.0000
Epoch 33/100
6/6 [==============================] - 0s 4ms/step - loss: 0.1790 - accuracy: 1.0000
Epoch 34/100
6/6 [==============================] - 0s 2ms/step - loss: 0.1658 - accuracy: 1.0000
Epoch 35/100
6/6 [==============================] - 0s 1ms/step - loss: 0.1540 - accuracy: 1.0000
Epoch 36/100
6/6 [==============================] - 0s 2ms/step - loss: 0.1444 - accuracy: 1.0000
Epoch 37/100
6/6 [==============================] - 0s 1ms/step - loss: 0.1355 - accuracy: 1.0000
Epoch 38/100
6/6 [==============================] - 0s 2ms/step - loss: 0.1285 - accuracy: 1.0000
Epoch 39/100
6/6 [==============================] - 0s 1ms/step - loss: 0.1217 - accuracy: 1.0000
Epoch 40/100
6/6 [==============================] - 0s 1ms/step - loss: 0.1156 - accuracy: 1.0000
Epoch 41/100
6/6 [==============================] - 0s 1ms/step - loss: 0.1105 - accuracy: 1.0000
Epoch 42/100
6/6 [==============================] - 0s 1ms/step - loss: 0.1052 - accuracy: 1.0000
Epoch 43/100
6/6 [==============================] - 0s 1ms/step - loss: 0.1008 - accuracy: 1.0000
Epoch 44/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0966 - accuracy: 1.0000
Epoch 45/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0927 - accuracy: 1.0000
Epoch 46/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0892 - accuracy: 1.0000
Epoch 47/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0857 - accuracy: 1.0000
Epoch 48/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0827 - accuracy: 1.0000
Epoch 49/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0798 - accuracy: 1.0000
Epoch 50/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0770 - accuracy: 1.0000
Epoch 51/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0743 - accuracy: 1.0000
Epoch 52/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0719 - accuracy: 1.0000
Epoch 53/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0696 - accuracy: 1.0000
Epoch 54/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0674 - accuracy: 1.0000
Epoch 55/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0653 - accuracy: 1.0000
Epoch 56/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0634 - accuracy: 1.0000
Epoch 57/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0615 - accuracy: 1.0000
Epoch 58/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0598 - accuracy: 1.0000
Epoch 59/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0581 - accuracy: 1.0000
Epoch 60/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0565 - accuracy: 1.0000
Epoch 61/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0550 - accuracy: 1.0000
Epoch 62/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0536 - accuracy: 1.0000
Epoch 63/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0522 - accuracy: 1.0000
Epoch 64/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0509 - accuracy: 1.0000
Epoch 65/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0497 - accuracy: 1.0000
Epoch 66/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0485 - accuracy: 1.0000
Epoch 67/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0473 - accuracy: 1.0000
Epoch 68/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0463 - accuracy: 1.0000
Epoch 69/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0452 - accuracy: 1.0000
Epoch 70/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0442 - accuracy: 1.0000
Epoch 71/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0432 - accuracy: 1.0000
Epoch 72/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0423 - accuracy: 1.0000
Epoch 73/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0414 - accuracy: 1.0000
Epoch 74/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0405 - accuracy: 1.0000
Epoch 75/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0397 - accuracy: 1.0000
Epoch 76/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0389 - accuracy: 1.0000
Epoch 77/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0381 - accuracy: 1.0000
Epoch 78/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0373 - accuracy: 1.0000
Epoch 79/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0366 - accuracy: 1.0000
Epoch 80/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0360 - accuracy: 1.0000
Epoch 81/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0353 - accuracy: 1.0000
Epoch 82/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0346 - accuracy: 1.0000
Epoch 83/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0340 - accuracy: 1.0000
Epoch 84/100
6/6 [==============================] - 0s 942us/step - loss: 0.0334 - accuracy: 1.0000
Epoch 85/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0328 - accuracy: 1.0000
Epoch 86/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0322 - accuracy: 1.0000
Epoch 87/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0316 - accuracy: 1.0000
Epoch 88/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0311 - accuracy: 1.0000
Epoch 89/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0305 - accuracy: 1.0000
Epoch 90/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0300 - accuracy: 1.0000
Epoch 91/100
6/6 [==============================] - 0s 946us/step - loss: 0.0295 - accuracy: 1.0000
Epoch 92/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0290 - accuracy: 1.0000
Epoch 93/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0286 - accuracy: 1.0000
Epoch 94/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0281 - accuracy: 1.0000
Epoch 95/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0276 - accuracy: 1.0000
Epoch 96/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0272 - accuracy: 1.0000
Epoch 97/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0268 - accuracy: 1.0000
Epoch 98/100
6/6 [==============================] - 0s 2ms/step - loss: 0.0263 - accuracy: 1.0000
Epoch 99/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0259 - accuracy: 1.0000
Epoch 100/100
6/6 [==============================] - 0s 1ms/step - loss: 0.0255 - accuracy: 1.0000

<keras.callbacks.History at 0x19242fe6a00>

# Step 7: Evaluate the model
loss, accuracy = model.evaluate(X_test, y_test)
print(f'Loss: {loss:.4f}')
print(f'Accuracy: {accuracy*100:.2f}%')

1/1 [==============================] - 0s 148ms/step - loss: 0.0221 - accuracy: 1.0000
Loss: 0.0221
Accuracy: 100.00%

# Print the feature names
feature_names = iris.feature_names
print("Feature Names:", feature_names)

Feature Names: ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']

# Get the learned weights from the first layer
weights = model.layers[0].get_weights()[0]

# Print the feature importance based on the learned weights
feature_importance = np.mean(np.abs(weights), axis=0)
print("Feature Importance:", feature_importance)
print("Accuracy:", accuracy)

Feature Importance: [0.36568964 0.6201108  0.38020194 0.4190791  0.3266273  0.20344871
 0.18020876 0.3281672 ]
Accuracy: 1.0

# Identify the index of the most important feature
most_important_feature_index = np.argmax(feature_importance)
print("Most Important Feature Index:", most_important_feature_index)

Most Important Feature Index: 1