Digital Orchard

Towards an Autonomous Pruning Robot

The research program ‘Digital Orchard’ focuses on the development of sensor technologies and digital applications that enable smart precision agriculture for the fruit sector.

Managing fruit orchards in a smart and sustainable way is challenging because tree maintenance is labor-intensive and requires the right expertise. Finding suitable labor is becoming an increasing challenge for the fruit sector. Therefore, the importance of using smart autonomous field robots in commercial orchards is increasing worldwide.

Autonomous management of fruit orchards

Robotization of fruit cultivation mainly focuses on labor-intensive tasks such as fruit picking, tree pruning, or blossom thinning. It can be particularly challenging to work in an orchard due to the complexity of the environment, where the fruit is often hidden behind leaves and not immediately visible. Furthermore, the impact of decisions like pruning may only become apparent in the future, with growth, bloom, and fruit set.The latest orchards are designed and planted to be suitable for future robotization. OnePlanet Research Center follows these developments and investigates in the research program ‘Digital Orchard’, which sensors and image processing systems are needed on a robot platform in the outdoor environment, for continuous data and image collection in the orchard of the future.

LiDAR scanners and stereo cameras

OnePlanet Research Center, in collaboration with various partners, is developing sensor fusion systems that combine images from LiDAR scanners, stereo cameras, and hyperspectral cameras. In this way, OnePlanet captures images of fruit trees in three dimensions and can digitally reconstruct individual trees based on the 3D images. These 3D reconstructions form the basis for recognizing structures and deviations (fruit, leaf, blossom, branches, insects) and for teaching tasks (picking, pruning, thinning, spraying) to future robot systems.

Digital twin of the orchard

The data and images are combined with knowledge of plant models and form the basis for building a digital orchard (a digital twin). They can also be used for educational purposes in VR applications, in which the user can practice and learn tasks in a virtual 3D environment. Future robotization tasks that OnePlanet is working on include autonomously pruning fruit trees (apple and red currant bushes) and better recognizing diseases and insect pests in pears.

Application-driven research with robotics

In addition to technical feasibility, the future acceptance and implementation of robots or cobots in future cultivation systems, require careful choices regarding practical applicability, safety, and ultimate costs. By seeking the right collaborations, OnePlanet Research Center tries to ensure those practical conditions in its research and development activities.

The Mood & Eating Behavior project at OnePlanet Research Center investigates the link between mood and eating behavior. The long-term objective is to get insights in the link between mood and eating behavior to assist people with just-in-time-adaptive-interventions to enhance mood and improve their health.

Smart SnackBox

The smart SnackBox is a tool, developed at OnePlanet Research Center, to assess dietary consumption. The emphasis is on drinks and snacks that you consume between meals. This is also referred to as snacking behavior in everyday life.

In the Netherlands, we see that on average 30% of the daily caloric intake is coming from snacking. For overweight adults this can even be around 50%. Besides the obvious relationship between snacking behaviour and physical health, snacking has a particular strong connection to mental health as well. Where hunger is often the key factor in the quantity of consumption for meals, for snacking this is mood and mental well-being.

Studies caloric intake from snacking

There are quite a few laboratory studies that show that moods influence calorie intake through snacks. Moods like sadness or stress influence snack behaviour. With the smart SnackBox, OnePlanet was able to replicate these findings in a natural setting, at work or at home.

Besides the relationship with mood, the accuracy in time of the SnackBox also enables investigating the relationship between snacking and physiology in natural settings, using wearable data as additional input here.

Mapping these relationships helps in:

1. understanding decision making and impulse control related to snacking behavior on group level.
2. predicting future snack events on individual level.

The latter could also help the user by inducing just-in-time adaptive interventions and increasing self-awareness. This feedback-loop has proven to be effective in other lifestyle intervention programs on for example stress, and the smart SnackBox allows us to apply that framework to snacking behavior.

The world’s population is growing exponentially. Precision agriculture running on integrated photonics is helping to keep our food supply safe and sustainable in the long term.

OnePlanet Research Center Introduces OpenPlanet; An accessible digital environment for entrepreneurs and organizations in health, agriculture and food.

OpenPlanet is an accessible digital ecosystem for entrepreneurs and organizations in health, sustainable agriculture and food. It is a secure data & application environment in which data is collected, analyzed and shared. OpenPlanet aims to provide an answer to the current fragmentation of solutions and the lack of knowledge to effectively convert data into insights and digital services. OpenPlanet offers users the opportunity to share and enrich relevant data for end-user applications across application domains. it also provides an environment for developing advanced algorithms and models.

Imagine if there was one open, secure data & application environment for companies and organizations working in the fields of sustainable agriculture, food and health?

The platform consists of applications for data collection (sensors, apps, links with various data sources), data connectivity (sensor network, digital infrastructure), data storage and analysis (AI, digital twins), a marketplace for algorithms and applications for action perspective, support and feedback to end-users (apps, dashboards). For example, data from wearables can be shared live between doctor and patient, or data from sensors can be used to optimize real-time processes in agriculture.

OpenPlanet is a collaborative project of OnePlanet Research Center, Orikami, IVIDO, JoinData, and SIDN. OpenPlanet has received a REACT-EU contribution from East Netherlands. With this, Europe is investing in the provinces of Overijssel and Gelderland in a rapid economic recovery after COVID-19.

This is a contribution that is in line with the priorities of the province of Gelderland: “The build-up of the data ecosystem and associated activities are aimed at green, digital and resilient recovery of the economy through economic structure strengthening through targeted investments in facilities and valorization activities that aim to give SMEs the opportunity to develop new ideas and products. test, develop and scale up in an accessible way’. – Province of Gelderland

Know more about OpenPlanet?

Jacob Beeuwkes, Program manager OpenPlanet Program – OnePlanet Research Center 

Jacob.Beeuwkes @imec.nl

Together with key partners, OnePlanet Research Center is using the newest technology innovations to tackle problems in supply chains of perishable food products. Our goal is  to improve product quality and reduce greenhouse gas emissions and food waste with sensing and digital twin technology. The innovative sensor systems and the digital twins results in real-time status information and decision support. This opens possibilities for advanced quality-based control allowing for more efficient food chains with better matching supply and demand, more insight in the quality of the food products and less food waste. 

Imagine that.. we could use sensors and digital twins to improve the quality of perishable food products, while at the same time reduce greenhouse emissions and food waste!

To make a real impact within this domain, it is key to establish real-time access to supply chain conditions and product information in each stage of the supply- and production chain. OnePlanet Research Center is exploring the use of (i) advanced IoT sensor systems to monitor products in the supply chain,(ii) real-time data access and data integration to create the full picture of what is going on in the supply chain, and (iii) relevant models that allow for prediction of product characteristics at each moment of its lifetime.

With these elements in place, a digital twin of the product in the food supply chain can be created and continuously updated. The digital twin will allow for simulation of future behaviour in various scenarios, and thereby enables chain actors to make optimal decisions at each moment in time.

Two prototypes of a digital twin will be developed:

1. The digital twin of the fresh supply chain, in which the emphasis lies on the long transportation and quality development of tropical fruits, such as bananas or avocados, and on non-destructive quality sensing of glasshouse vegetables such as tomatoes.
2. And the digital twin of the meat production chain, in which the emphasis lies on individual quality monitoring of carcasses.

This work is done in a close collaboration between system and data integrators, sensor developers, use case partners and research institutes Wageningen Research and Imec as part of their collaboration in the OnePlanet Research Center initiative. OnePlanet is an innovation center for chip and digital technology in agri, food, health and environment. It is a collaboration between nano-technology R&D institute imec, Radboud University, Radboudumc and Wageningen University & Research.

This collaboration is a great example of how digital transformation can be applied in practice and have both economic and societal impact by reducing food waste, increasing product quality, reducing greenhouse gas emission, and increasing the profitability of the food chains.

Smart, sensor-based solutions will be a big step towards ultra-controlled greenhouse production that, in turn, will be a milestone in Indoor Autonomous Farming. The innovations developed will provide farmers and greenhouse management companies new opportunities for sustainable food production.

If we had the tools to fully automate crop cultivation in greenhouses, non-specialized workers would be able to perform basic greenhouse-management tasks, and enable professional practitioners and commercial greenhouse management companies to work more efficient, freeing them from repetitive and heavy labor. It would deliver farmers and plant scientists new insights to optimize plant health and production yields, and thus opportunities to make food production more sustainable. And it would allow food to be produced in urban locations, particularly where there is limited space, shortening the distance between producer and consumer.

Sensor-based, smart solutions

In Indoor Autonomous Farming, OnePlanet Research Center is developing a 3D sensor-based monitoring network and an integrated data platform that opens the way to autonomous greenhouse production and, in the longer term, fully-controlled vertical farming systems. The system might combine, for example, cameras measuring leaf temperature (an indicator of respiration) with RGB cameras that give insight into nutrient deficiencies and the early onset of plant diseases, and a whole range of other sensors throughout a greenhouse.

Conventionally, growth conditions are measured at only one location in the greenhouse. Sensors at different locations will provide farmers and greenhouse specialists additional insight into how a greenhouse functions – for example whether windows are opened and closed correctly, or whether the heating pipes are doing their job – and how plants respond to growth conditions in real time.

In this program, OnePlanet combines its expertise on high-tech sensors with a deep understanding of data-sciences, machine-learning, agriculture and food to address the complex issues that inevitably accompany radical innovation. How to combine and integrate the output of sensors located in different positions in a greenhouse, and how to link them, in real-time, with data on plant health? What kind of electronics will be durable and reliable in humid greenhouse conditions and will they need special protection? Novel sensors developed in another OnePlanet program, Emerging Sensing, will be validated in indoor farms, and linked to crop data in order to create models for improving crop health, yield and use of resources in indoor settings.

Towards an online platform

AGROS, the first research program in this innovation line, kicked-off in May 2020, funded by two Dutch Topsectors and with over 26 private partners. Cucumber plants are the first crop to be cultivated, using a smart sensor network prototype that includes measurement of greenhouse atmosphere and humidity, and thermal and hyperspectral imaging that provides pictures of individual plant health. First measurement outcomes using the sensor protoype show agreement with reference sensors. By 2026, the data collected will be integrated into an online platform that allows autonomous control of greenhouse climate and fertigation.

Digitizing agriculture to improve the yield of potatoes and lowering emissions and water usage. That’s the main goal of this research project. The research team is developing new methods for below and above ground sensing and multiple techniques, like radar, x-ray, hyperspectral imaging and impedance tomography.

The project was initiated by Wageningen University and Research and NARO, the Japanese National Agriculture and Food Research Organization.

Researcher Jan Willem de Wit explains the TTADDA project in the field:

To improve lifestyle recommendations, we need to learn what works particularly for one person and what doesn’t. Since everybody has different genetics, health status, activity levels and environments. Artificial intelligence (AI) can help us here. In the ICAI Lab researchers aim to increase the quality and quantity of health, dietary and behavioral data and to develop AI algorithms and models to improve personalized lifestyle feedback.

Learn more about the Lab team and the Lab partners

What if … we can give more accurate personalized advice on diet and healthy lifestyle with help from artificial intelligence?

Non-invasive methods to measure the condition of plants is one of the highly-anticipated outcomes of Emerging Sensing. These techniques will provide equipment builders, technology providers and the fruit and vegetable sector with new tools to increase yields, improve quality and reduce waste.

Suppose we could see right ‘through’ a plant: discerning the presence of diseases, the ripeness of fruit and many other properties that are currently hidden to the naked eye. This would enable farmers to optimize the health of their plants, making much better decisions about when to harvest and when to transport produce to market. It would also facilitate the development of advanced picking robots, reducing the need for the increasingly scarce, specialized personnel in this sector.

Innovative spectral imaging

In Emerging Sensing, OnePlanet Research Center is working on innovative acoustic and spectral imaging techniques that make visible what is hidden under the skin of the plant and its fruits; techniques that allow one to see ‘through’ fruits and ‘sense’ from a distance. While existing hyperspectral cameras can already detect invisible bruises under the skin of apples and pears, complementary sensing modalities are being investigated for the non-destructive and robust detection of moisture content, fruit ripeness and firmness. OnePlanet is also working on early pest detection tools, combining spectral imaging with systems that register the presence of insect DNA and pheromones in a data fusion approach.

In this program, OnePlanet combines its expertise on high-tech sensors and wearables with a deep understanding of data-sciences, machine-learning, crop cultivation and post-harvest quality, to answer challenging questions. What are, for example, the possibilities and limitations of existing imaging techniques, in the context of which types of fruits and vegetables? Are specific spectra more suited to food quality assessment?

Towards a demo system

In 2020 three research projects kicked off: the public-private partnership, Future Sensors, which focuses on sensors for tomatoes and their plants; the European program Haly-ID, dedicated to apples and pears; and the Public-Private Partnership ‘Weet wat er leeft’ (Know what’s going on) which focuses on automatic identification and quantification of insects in greenhouses. A measuring-system proof-of-concept will be ready by the end of 2021. Once validated, it will be extended with applications that integrate sensor data, and predict the quality and shelf life of fresh produce. A demo system is expected by 2024.

Less greenhouse gas emissions and improvement of animal welfare. That is what OnePlanet Research Center is researching in this process, in specially established “AgriFood Validation Labs”.

OnePlanet develops new digital concepts for sustainable livestock farming together with network partners and educational institutions. With special attention to animal-friendly and sustainable production of animal proteins and products.

OnePlanet Research Center is working on quantifying the impact of agricultural activity on the environment and nature, by means of fine-meshed sensor networks.

Whether it concerns nitrogen in the air or nitrate in the surface water, all attention is focused on the correct values. But how do you measure them? OnePlanet Research Center develops new technologies that enable accurate and long-lasting on-site measurements, suitable for different end users.

OnePlanet develops fine-mesh sensor networks to measure air quality. Including digital environment for data access. This technology provides insight into local emissions.

In order to be able to measure substances such as nitrate in surface water, OnePlanet is developing a handheld nitrate sensor with partners. This sensor enables farmers to carry out nitrate measurements themselves. For example, they can detect nitrate leaks into surface water.

Better quality of food products and less waste in the chain, that is what OnePlanet Research Center strives for with this innovation process. A ‘digital twin’ of food processing processes makes it possible.

Using new sensor and digital technologies, OnePlanet gains real-time insights into food processing processes and the supply chain. This can lead to higher quality food and a more sustainable food production.

A “digital twin”, a virtual copy of certain products or processes, can be especially interesting for vulnerable products. It provides valuable insights into the logistics process, on a continuous basis, making it easier to make adjustments. In this way, more food products of better quality reach the end user, while using fewer resources. So less waste and healthier food!