Member-only story
EfficientNet was proposed in 2019 and since then, it gains popularity due to its training and inference speed, as well as model accuracy. EfficientNet has much lower number of parameters, but could achieved higher ImageNet Top-1 Accuracy, as compared to the other state-of-the-art at that time.
EfficientNet is widely used in the industry as it can reduce model development time while maintaining production grade accuracy. But what makes EfficientNet efficient? What are the differences between EfficientNet and the other architecture?
Deep learning model get its name because of the “deep” layer. In the early days of deep learning model, just blindly adding more layers in the neural network often improve the accuracy (until the model overfit). The scaling of the model is often arbitrary, just trial and error to get a good scaling method. This means that although the accuracy of the model is improved, the model development time is much tedious and the computational effort for training and inference is also increase arbitrary. EfficientNet was proposed to tackle this issue. The authors want to scale the model efficiently, without scaling some layers without improving the model performance.
There are 3 different scaling methods:
- Width Scaling — increasing the ‘channels’ in the layer.
