A Deep Learning Framework for Smart Agriculture: Real-
Time Weed Classification Using Convolutional Neural
Network
Sushilkumar S. Salve,
*
Sourav S. Chakraborty, Sanskar Gandhewar and Shrutika S. Girhe
Department of Electronics and Telecommunication, Sinhgad Institute of Technology, Lonavala, Maharashtra, 410401, India
*Email: sushil.472@gmail.com (S. S. Salve)
Abstract
Agricultural sector being the foundation of food supply and raw material production contributes significantly to the
GDP growth and value chain. Thus, the effective elimination of weed in modern day agriculture is essential as the
current world scenario demands for efficient and resourceful ways for crop cultivation and harvesting. The urgent
need for elimination of weeds arises due to their tendency to extricate all the essential minerals and moisture that
the crops require for their appropriate growth. The main objective of this study is to successfully acquire live video
feed as input, classification into categories of crop, weed and none. Finally, upon detection of weed the spraying
mechanism releases a pre-determined amount of herbicide upon the weed. A total of 5471 image samples were
captured to train the CNN model. The prototype mentioned in this paper uses Convolutional Neural Network (CNN)
technique for feature extraction, Fully Connected Layers or Dense Layers (FCLs) for classification using SoftMax as the
activation function respectively. The activation function also here is being used to remove all negative (less significant)
values. Also, a comprehensive comparison was made between the CNN and YOLOv4 technique and performance
parameters of both were evaluated. The CNN technique achieved an accuracy of 95.50% whereas YOLOv4 achieved
91.00%. Finally, the F1 Score was evaluated to be 96.25% and 91.96% respectively. Compared to existing models, our
prototype demonstrated higher accuracy and real-time adaptability in field conditions, proving suitable for
autonomous weed management systems. Unlike earlier systems that depended mostly on stored images or fixed
datasets, our approach stands out by using a live video feed to identify weeds in real-time. It’s built on a mobile
platform that can automatically spray herbicides, making precision farming possible without the need for constant
human supervision.
Keywords: Computer vision; Convolutional Neural Network; Deep learning; SoftMax; Max pooling; Weed detection; Image
pre-Processing.
1. Introduction
In crop fields, weeds are naturally occurring plants that compete with crops for vital resources like space, light,
moisture, and air, which may lower crop yield. Effective weed control is essential during cultivation because they
impede crop growth.
[1]
Farmers may experience lower yields and financial losses as a result of weeds competing for
resources with cash crops. The impact of weeds varies depending on the crop type and the farm’s geographical
location.
[2]
Weeds can reduce yield by up to 34% if they are not controlled, whereas animal pests and diseases cause
yield loss of 18% and 16%, respectively. Weed infestations can result in crop losses of roughly 23% to 44% in typical
crop fields.
[1]
Simultaneously, the agricultural sector is under pressure to achieve steadily rising yields as the demand
for more food production rises at the same time.
[3]
This emphasizes how precision farming and robotics are necessary
to increase yield while lowering dependency on conventional farming practices. Modern technology has made it
possible for autonomous machines to carry out agricultural tasks effectively. High-quality crops can be produced with
little human labor when robotics and intelligent machinery are integrated into agriculture.
[3,4]
A weed detection system uses machine learning algorithms to identify unwanted plants in an agricultural field. Farmers
can reduce their use of weed and herbicides, which can be harmful to the environment and public health. Plans for
targeted weed control can be created by utilizing the information on the types of the weeds that the detecting system
can supply.
[5]
A new technology that has the potential to completely transform agriculture is machine learning-based
weed detection. The system's purpose is to locate and identify weeds in a field so that farmers can take specific action
to get rid of them, gather live videos and photos of a field, apply machine learning techniques to the same, and then
determine the weeds. Numerous methods, such as object detection, feature extraction, segmentation, and
Classification, can be used to complete this process. We decided to use a live feed CNN technique to address this
problem it's more like analyzing the input dataset to find the weeds.
[5,6]
The weeds within rows might not be accurately removed by conventional machinery. Sunil G C et al. emphasized
while introducing their study on the thought that the herbicide which is sprayed uniformly across the field, treating
weeds and crops alike, at a set pace when compared to site-specific herbicide applications prove less feasible as blanket
herbicide applications may have a more negative impact on the ecosystem. As a result, applying an herbicide
selectively to areas of concern may improve precision while lowering input costs and environmental problems.
Umamaheswari S et al.
[7]
mentioned about the field of robotic farming and precision agriculture that needs
to advance in response to current problems with the lack of agricultural labour and resources, the emergence of new
crop diseases, and weeds. The issues of climate change and sustainable agriculture are intimately tied to the challenge
of effective weed classification and detection. According to various resources and findings the study suggests, existing
species may be exposed to new and hybrid weeds as a result of climate change. Because weeds can hinder the growth
of farm crops, it is crucial to create new technologies that aid in identifying them. Identifying weeds can also help
remove them, which lowers the need for pesticides and offers effective substitutes when the crops are harvested.
O.M. Olaniyi et al.
[8]
mentioned about the various ways of weed eliminations as people have become more civic and
knowledgeable about weeds, experts have been looking for ways to eradicate the infamous pest with the least amount
of harm to the plant. The three main strategies for controlling weeds are cultural, chemical, and automated approaches.
Bush fallowing, mulching, fire clearance, early flooding, hand weeding, shifting crops, and maintaining a clean reaper
are all components of the cultural approach of weed management. This approach has significant labour costs and
drawbacks. Applying herbicides is thought to be a significant alternative to hand weeding. However, excessive
herbicide use can result in harvest losses, harm to the environment, high production costs, and the development of
herbicide resistance. Without getting to the weeds, some of these pesticides even wind up on the soil and food crops.
Since spraying food crops is viewed as a risk to the safety of the food being consumed, a thorough weed control method
is necessary.
On the other hand, as specified by P. Kavitha Reddy et al.,
[5]
deep learning techniques particularly those that use neural
networks have become increasingly popular in recent years. These methods use big datasets of tagged images to train
and intricate neural network models. The neural network automatically collects pertinent information and classifies
the input photos using iterative learning procedures. The YOLO algorithm is a well-known implementation of the
convolutional neural network (CNN), which is the foundation of deep learning techniques in computer vision (CV).
In this paper, a low cost and robust weed detection, live video-based and elimination system with automated spraying
using Convolutional Neural Network (CNN) as the main computing algorithm, SoftMax and ReLU as activation
functions and classification of the same using Fully Connected Layers (FCLs) is given along with a detailed
comparison of YOLOv4 with the proposed method.
The major reason for why CNN was selected is due to its ability to focus on fine-grained feature learning, especially
useful in identifying small or overlapping weed patterns. YOLOv4 was chosen for comparison due to its real-time
detection speed. Other models like Faster R-CNN or ViT were not used due to higher computational demands
unsuitable for edge deployment on Raspberry Pi 4. The two activation functions SoftMax and ReLU were selected for
their simplicity, speed, and established use in CNN architectures. Alternatives like Swish or Leaky ReLU can improve
performance but require higher computational cost and tuning.
2. Materials and methods
This particular section describes the materials and design required for the successful development of the particular
proposed system. Here a detailed overview of the components, methodology utilized and many other specifications
are mentioned. The system prototype well integrates the combination of Internet of Things (IOT) with image
processing, feature extraction, deep learning algorithm and identification along with precision spraying unit.
2.1 System overview
The proposed system is implemented using Convolutional Neural Network to develop and cultivate a robust, multi-
scalable and versatile weed detection system that produces accurate results in real time using live video feed via a
webcam. The input dataset then goes through various processes and at the end determines the result based on three
particular parameters i.e., i) weed, ii) crop, iii) none. The various processes particularly include, Image Acquisition,
feature extraction, classification and training of the model.
A generalized block diagram is represented as Fig. 1 that gives an idea regarding the actual flow of the components
within the proposed system, and their particular task involved in the accurate execution. The proposed prototype
contains various components mounted on a robust wooden platform which are powered by a 12V DC adapter.
Fig. 1: Block diagram of proposed prototype.
The main microcontroller unit i.e., Raspberry Pi 4 Model B is powered by a 5V USB-C type charger. The output can
be observed on a desktop monitor via connection with an HDMI cable. Fig. 2 shows stage by stage deployment and
implementation of a particular CNN based weed detection system using Max Pooling, ReLU, Dropout, Fully
Connected Layers (FCLs) and SoftMax for multiple stages of detection and processing of the input dataset.
[9,10]
Fig. 2: Schematic of the proposed system overview.
2.2 Working principle
2.2.1 Hardware
The “A Deep Learning Framework for Smart Agriculture: Real-Time Weed Classification Using CNN” uses a robust
and sturdy navigable prototype that enables the system to be mounted of a hard bound wooden base with a four-wheel
chassis. The two forward wheels are attached with two 12V DC geared motors of 300 r.p.m each and the two rear
wheels are attached as dummy wheels for support. As they support heavier load i.e., in this case a wooden platform
12V DC geared motors are used. These motors are then connected to an L293D module. This L293D module is a motor
driver module which is widely used in embedded systems to control the direction of DC motors and stepper motors.
This module is capable of driving two DC motors independently in both forward and reverse direction. This adds
precision and control to the whole system and grants mobility across the field. Both the L293D module and the DC
motors are powered using a 12V DC power supply. The L293D is also interfaced with the Raspberry Pi 4 model B as
the master control unit.
A Bluetooth module i.e., HC-04 is also interfaced to the microcontroller for controlling the directions provided by the
motor driver module. This Bluetooth module supports V2.0+ EDR (Enhanced Data Rate) up to 3 Mbps modulation
along with 2.4 GHz radio transceiver and baseband. A python program is being compiled and executed by the
microcontroller that enables the user to connect with the Bluetooth module using the application “Serial Bluetooth
Terminal”, where user can give commands in the form of numbers for specifying movements in specific direction (i.e.,
1 = forward, 2 = reverse, 3 = left, 4 = right, 5 = terminate).
A single-channel relay module is also being used and interfaced with the microcontroller in order to control the fluid
pump inside the sprayer prototype. It is rated for switching up to 10A at 250V AC or 24V DC. This is also powered
using the 12V DC power supply which was used to power the motors and driver module. Now in this relay when the
input (IN) is driven LOW, at that time the relay coil energizes and it switches the normally closed (NC) contact point
to normally open (NO) contact. This action effectively contributes in turning a connected device (i.e., the fluid pump)
on or off at specified intervals upon weed detection.
The camera module used in the particularly developed system is the Xiaomi Mi HD USB 2.0 Web-Cam. It can capture
live video feeds up to a resolution of 1280720p HD and has a frame rate of 30 FPS. With up to a 90 wide angle field
of view it has no driver requirements, thus compatible with the Raspberry Pi 4 microcontroller. The OpenCV library
efficiently helps the prototype to capture and process the live feed of input dataset for image pre-processing.
The Raspberry Pi 4 model B microcontroller acts as the heart and brains of the system. This is basically a card sized
mini-computer that operates using its own software, performing tasks that an actual desktop can perform independently
including browsing, media playback and major IOT development. It has a 4 GB LPDDR4-3200 SDRAM and has a
microSD card slot that comprises of the actual controller software. It consists of four USB ports two with USB 3.0
capabilities and the other two with USB 2.0 support. Two micro-HDMI slots are provided for interfacing with external
display peripherals supporting resolutions up to 4K 60 FPS. The power supply is provided via a 5V DC USB-C type
connector, and has an ambient operating temperature within the range of 0C to 50C.
Fig. 3 gives a glimpse of the proposed system that is being cultivated and developed for the comprehensive study of
both the algorithms. The image gives a clear idea about all the particular components and their whereabouts in the