“We already have a lot of capabilities in managing our plants in a precise way,” Manoj Karkee, associate professor of biological systems engineering at Washington State University, said in a recent virtual presentation. The goal is to “manage our orchards and vineyards at the individual plant level.”

Reaching that goal means incorporating mechanization, artificial intelligence (AI) and cyber physical systems (CPS). Working together, AI and CPS integrate machine learning, analysis of historical and real-time data, machine vision, predictive learning, pattern recognition and robotics. This information is utilized to make farming decisions such as canopy management, pest and nutrient management and harvesting decisions. Combined with robotics, tasks such as harvesting, pruning, thinning and other plant management activities can then be automated for precise results.

Collectively, these approaches are known as Smart Farming, or Ag 4.0.

The goal is to increase efficiencies, do more with less labor or inputs and increase productivity and yields. While there are many precision tools being utilized in row crops, doing so in specialty crop production – particularly in tree fruit or small fruits – has been less successfully implemented, and remains in development.

“So far, there hasn’t been any remarkable commercial success in automating fruit or vegetable crops, including harvesting,” Karkee said. “Some of the limitations include speed, accuracy and robustness – or their lack of those features.”

There has been some concern about damage to the crop, the tree or to the machines, and this has been a factor in wide-scale implementation of mechanization in specialty crops. The initial cost of the emerging technology for use in fruit production has been a concern as well.

“Some of these high-tech technologies have relatively higher costs of production in the beginning,” Karkee said. Because of this, there hasn’t been enough adaptation to help bring those costs down due to economies of scale.

Harvesting Automation

Researchers at WSU, in collaboration with other universities, are working on numerous innovative approaches to developing integrated systems for tree fruit harvesting as well as other uses. Cherry and apple automated harvesting platforms are a focus of that research. Different robotic systems and catching mechanisms, including mass harvesting technologies, are being researched.

Robotic harvesting of apples involves a robotic hand which, in conjunction with imaging systems, can pick ripe fruit. Some issues with collision between the arm and the tree, or trellis wires, remain. Increasing the robot’s awareness of the environment surrounding the apple, through the use of cameras, has helped to optimize picking. Enhancing the hand’s ability to pick only the apples and not surrounding sticks (a limiting factor in previous experiments) will vastly improve efficiency. Field evaluations are ongoing.

“We have demonstrated the capabilities of robotic hands in picking this fruit,” Karkee said. “There are some situations where robotic picking makes a lot of sense,” while other situations might call for different technology.

Targeted “shake and catch” harvesting of apples works better on some varieties or growing systems. Systems are being developed where the tree can be simultaneously harvested in layers. Sections of branches can be targeted using less energy, which decreases the distance the apples need to travel to be collected and minimizes fruit damage to about 10% of the harvest.

“We are still working on it to improve it. I believe this kind of technology could also provide a viable commercial solution, in the near future, for some varieties,” Karkee said.

Automated cherry harvesting, developed in the 1990s, shakes entire trees so the ripe fruit falls and is collected. Recently, the efficiency of robotic cherry harvesting has been augmented via the use of different shaking patterns. Gentler shaking means less fruit damage. Using advanced imaging systems has also allowed the robots to find missed branches and grab them individually on a second pass. Handheld shakers, used by workers, are also being designed, which can improve worker harvesting efficiencies up to tenfold.

Canopy & Load Management

Vision systems with stereo cameras are making automated pruning a reality. These systems utilize machine learning to formulate pruning rules and identify targets. Laboratory experiments can mimic high density growing systems and use a robotic pruner, which can learn to identify the proper branches to prune out. This system will be getting field trials as soon as possible, but has been delayed due to the pandemic.

Systems to thin flowers in fruit trees – removing select flowers from each cluster – is also being field tested. Crop load estimating using handheld solutions can provide orchardists with the ability to count and size apples and grapes.

Raspberry growers can use automation too. Cane bundling and tying can be automated to hold raspberry canes together.

In vineyards, green shoot thinning for wine grapes has been in development over the past three years. Using models made in the dormant season, it is possible to accurately predict where the cordons are, even under the canopy, and prune them precisely. Even with heavy canopy cover, the automated system had good correlation to manual pruning.

“We can use this kind of image processing system to potentially do dormant season selective pruning in vineyards, and similar technologies could be used in training apple trees,” Karkee said.

Smart Irrigation systems take both historical and current sensor data on soil, weather, crop and other factors and analyze them to inform decisions on when, where and how much water or fertigation should be applied. AI interprets the data, which is relayed to automated systems, which then provide precise levels of irrigation.

The Precision Orchard Management Decision Support Tool is the newest automated tree fruit project at WSU. It uses color information, spectral imaging and soil sensing data, the size of the trees and other data to inform decision-making.

The future of automated systems for fruit depends upon lowering costs and increasing robustness. Speed is a key factor. Developing tools that can be used by many to increase production efficiencies is the hope for the near future. Karkee envisions a virtual, remote office where farm managers can analyze information and remotely manage orchard tasks, such as applying chemical sprays, and oversee multiple machines operating in the field, performing different tasks.

“It is not just a robotic system performing harvesting three months of the year, and sitting there idle for nine months,” Karkee emphasized. “Rather, we want to develop a machine where we can plug and play new tools to do harvesting, pruning, thinning, training and maybe even chemical application – and maybe even pollination – as we go into the future.”