Performance Evaluation of Swish-Based Activation Functions for Multi-Layer Networks
Artificial Neural Networks (ANNs) are computational modelling implements that are universally endured in many disciplines and utilized to model complicated real-world problems such as regression, classification, function approximation, identification, control, pattern recognition, and forecasting. The importance of the ANN originates from the information processing properties of biological systems like nonlinearity, fault tolerance, parallelism, and learning. Activation function (AF) is a crucial characteristic of an ANN since it gains nonlinearity to the networks. One of the most well-known AF used in the literature is swish AF which is generally used in deep networks. In this study, in addition to the swish AF we use three different AFs based on the swish AF which are mish, e-swish, and exponential swish (exp-swish) AFs for multi-layer perceptron. In order to compare the ANN models using different AFs for multi-layer networks, we use four different benchmark datasets from the University of California Irvine (UCI) Machine Learning Repository and get the result that exp-swish AF has the best performance.
Copyright (c) 2022 Artificial Intelligence Studies
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Artificial Intelligence Studies (AIS) publishes open access articles under a Creative Commons Attribution 4.0 International License (CC BY). This license permits user to freely share (copy, distribute and transmit) and adapt the contribution including for commercial purposes, as long as the author is properly attributed.
For all licenses mentioned above, authors can retain copyright and all publication rights without restriction.