Robots can do a lot. They build cars in factories. They sort goods in Amazon warehouses. Robotic dogs can, allegedly and a little creepily, make us safer by patrolling our streets. But there are some things robots still cannot do – things that sound quite basic in comparison. Like picking an apple from a tree.

“It’s a simple thing” for humans, says robotics researcher Joe Davidson. “You and I, we could close our eyes, reach into the tree. We could feel around, touch it, and say ‘hey, that’s an apple and the stem’s up here’. Pull, twist. We could do all that without even looking.”

Creating a robotic implement that can simply pick an apple and drop it into a bin without damaging it is a multimillion-dollar effort that has been decades in the making. Teams around the world have tried various approaches. Some have developed vacuum systems to suck fruit off trees. Davidson and his colleagues turned to the human hand for inspiration. They began their efforts by observing professional fruit pickers, and are now working to replicate their skilled movements with robotic fingers.

Their work could help to transform agriculture, turning fruit-picking – a backbreaking, time-consuming human task – into one that’s speedy and easier on farm workers.

These efforts have gained impetus recently as researchers point to the worsening conditions for farm workers amid the climate crisis, including extreme heat and wildfire smoke, and also a shortage of workers in the wake of the pandemic. The technology could lead to better working conditions and worker safety. But that outcome depends on how robots are deployed in fields, farm workers’ organizations say.

While robotic tools for agriculture have made big strides in recent years, those AI-based tools are mostly used for weeding, monitoring soil moisture and other field conditions, or for planting soybeans using remote-controlled tractors. “But when it actually comes to doing physical work like pruning trees or picking fruit, that’s still the realm of people today,” Davidson says.

Teaching robots to perform these tasks requires modernized versions of both the orchard and the apple.

Traditional orchards, with irregularly shaped trees and giant canopies, are too much of a challenge for algorithms to parse and process. Shifting sunbeams, fog and clouds add to computer vision’s challenges. Tangled, tall old trees are problematic even to human pickers, who end up spending much of their time hauling and positioning ladders, not picking fruit.

Now, many growers have transitioned to orchards where trees grow flat against trellises, their trunks and branches at right angles to create a “wall of fruit”, says Scott Jacky, owner of Red Roof Consulting, a group that helps optimize farm technologies. The thinner canopy also lets more sunlight in, encouraging fruits to form.

Since the 1990s, breeders have been working to develop apple varieties more resistant to sunburn – a side-effect of those sparser canopies – and less prone to bruising when dropped into bins. All these changes to the trees and the apples themselves make the job easier for robots (and for humans).

In orchards with trellised trees, human fruit pickers can cruise through rows of trees in pairs on slowly rolling platforms. One person crouches to reach low-hanging fruit, the other reaches for the higher branches. Professionals working this way take about two seconds to pick one apple.

The robot in Davidson’s lab, which is essentially a giant arm mounted on a rolling platform, takes about five seconds to make its moves. At the click of a key, the robotic arm reaches up for the fruit – actually a plastic apple made for testing purposes – with its three-fingered palm. Its fingers are covered in cushiony silicone “skin”, which conceals individual motors wired to tendons that drive its fingers. Thirty sensors under each fingertip track the pressure, speed, angle and other aspects of its grasp to help the robot complete its task.

Another keystroke and the fingers tighten, then twist, and the apple – successfully picked – rests in the robot’s palm.

The fruit-picking robot has picked an apple successfully about half of the 500 or so times it has tried so far. Still, the robotic arm has cracked some problems that posed hurdles to automation. For instance, it can avoid damaging both fruit and tree limbs in the harvesting process. Rapid improvements in computing make Davidson and others hopeful the robots will work on farms within the next five to 10 years.

The US government is placing significant bets on this technology. Last year alone, federal funding agencies granted $20m to support the AgAID institute, a new group that supports several researchers, including Davidson, in efforts to develop artificial intelligence-backed tools for agriculture.

Proponents of harvest automation say there will still be jobs for people, such as training and operating the robots. “There are going to be plenty of tasks where the robotic instruments and digital devices will necessarily have to work with humans,” said Ananth Kalyanaraman, professor at Washington State University and director of the AgAID institute. “That’s going to actually empower humans because it gives them new skillsets.”

For now, it’s unclear to many farm workers how the robots will affect their livelihood. “If they’re used properly, they can actually be a support system for workers and improve standards at work,” says Reyna Lopez, executive director of PCUN, a Latinx farm workers’ organization in Oregon.

But so far, Lopez and others say they have not been involved in conversations about the fruit-picking robots. “Historically, farm workers have not been placed at the center of any of these conversations,” they say. Across various industries, including agriculture, waves of automation have led to job losses and a devaluing of human work. Often in the wake of such shifts, “what happens to low-wage workers is that people lose their jobs,” Lopez says.

The emergence of robotic farm workers could even be an opportunity for humans to engage in different – and far less strenuous – work than pruning or harvesting, says Ines Hanrahan, executive director of the Washington Tree Fruit Research Commission. “There’s a lot of folks in rural communities who, even if they would like to, physically cannot do these jobs,” she says.

“When you take the physical aspect out, these tasks become more accessible to older workers or those less physically capable of lugging ladders and things. It enables more people to be drawn into this work.”