Ļć½¶Ö±²„scientists use $1M grant to āget a gripā on automated blackberry harvesting
Contact: Meg Henderson
STARKVILLE, Miss.āThe labor-intensive work of harvesting delicate blackberries by hand is a must, but the development of advanced technologies by Mississippi State scientists could help automate the tedious process.
Many agricultural crops are picked quickly by machines, and Ļć½¶Ö±²„Assistant Professor Xin Zhang, of the Department of Agricultural and Biological Engineering, is working with a university team to do the same for ripe blackberriesātaking this high-value specialty crop from special handling to robotic harvesting.
In the universityās Mississippi Agricultural and Forestry Experiment Station, Zhang and her team are developing a blackberry detection and localization system, the āeyesā and ābrainā of a robotic harvester system powered by an innovative, artificial intelligence-driven deep learning approach.
Zhang is co-principal investigator on a $1 million multi-institutional effort funded by the USDA National Institute of Food and Agriculture National Robotics Initiative 3.0 (NRI-3.0) program in collaboration with the National Science Foundation.
As the Ļć½¶Ö±²„team develops this critical component of the automated harvester, partners at Georgia Tech are working on a soft touch robotic arm and gripper and a bipedal mobile platform to work hand-in-glove with the MSU-trained perception system. The prototype gripper is equipped with sensors located at the endsālike tiny fingertipsāallowing it to grasp and pick the berry without squeezing and damaging it. University of Arkansas scientists are focused on post-harvesting fruit analysis.
The perception system designed by Zhang and her team is powered by YOLOv8 (You Only Look Once, version 8), a vision-based object detection model that identifies and locates objects of interestāin this case, ripe blackberriesāquickly and accurately. This kind of technology is powerful enough to support robots, surveillance systems and self-driving cars.
The team trained a series of YOLO models to not only identify each blackberry on a bush but also detect its level of ripeness, from ripe (black in color), to ripening (red) and unripe (green). The process utilized over 1,000 images from plant canopies in various Arkansas commercial orchards, preparing the system to simultaneously single out ripe berries for harvest and keep track of others in preparation for the next round of picking.
āOur main focus is detecting the ripe berries for the robotic harvester, but we added the other two categories to provide a total berry count,ā Zhang said. āThis way, the system not only identifies harvest-ready berries but also helps provide growers estimates of their total harvest.ā
During tests with multiple configurations and variants, the best-performing model was 94% accurate in identifying ripe berries, 91% for ripening berries and 88% for unripe berries. It also detected impressively high-resolution images in real time, clocking 21.5 milliseconds per image.
āThe perception system identifies the berry and sends out 3D coordinates, including distance, to the robotic arm, which uses that feedback to reach out and pick the berry,ā Zhang said. āIt is critical that our perception system communicates quickly and accurately with the arm and gripper system.ā
Aside from their contributions to the harvester, Zhang and her team are beginning to develop a mobile app based on their image detection system.
āThe app is a separate project, but it would give growers a quick and easy way to forecast their total harvest at the beginning of the harvest season so they can swiftly adjust their marketing strategy,ā she said.
To learn more about the Mississippi Agricultural and Forestry Experiment Station, visit .
Ļć½¶Ö±²„ is taking care of what matters. Learn more atĀ .