Using computer vision and artificial intelligence to monitor cargo inside large commercial trailers for trucking

Computer vision (CV) and AI technology have advanced to a point where it is possible to evaluate camera footage to gain valuable business insights reliably. For example, in the highly standardized logistics industry, monitoring the cargo is critical to reducing unused space, improving worker safety, and detecting movements that could damage the load and the vehicle.

Computer vision = “An interdisciplinary scientific  field that deals with how computers can gain high-level understanding from digital images or video.”

The road freight logistics industry, which has an estimated global market size of €1,810B and an expected growth rate of 6.5% per year, is a mature industry where differentiation is hard to achieve. Hence, companies are turning towards advanced technical solutions to get an edge over their competitors. Accordingly, numerous startups, which aim at finding appropriate solutions, have sprung up in recent years. Gestoos offers an AI platform that enables cameras and sensors to see, understand, and respond to movement and behavior in any environment.

In this context, ZF was introduced to Gestoos back in November 2019. As a leading vehicle parts maker, ZF provides innovative logistics solutions to respond to customer requests. Therefore, ZF’s initial interest was using computer vision and AI to understand the height profile of the cargo at any given point in time within large commercial trucking trailers (around 14 meters in length). The height profile was needed for the entire trailer area, front to back and side to side. This information, along with additional sensor information, would then be used for the operational and logistical optimization of cargo trailers.

Gestoos — Redefining interaction through intelligent vision

Gestoos offers software solutions that allow its customers to improve safety, operational efficiency, and consumer HMI experience. Specifically, the company provides a platform for modeling human gestures & activities and 3D object movements by combining computer vision with proprietary patented machine learning and AI.

The company’s USP for this project lies in its expertise in evaluating time-of-flight data and adapting its patented computer vision AI technology to meet the client’s specific application needs, leading to quality gains and positive business impact.

Time-of-flight = “A time-of-flight camera emits an infrared light signal and measures how long the signal takes to return to measure the depth per pixel.”

The Project

ZF and Gestoos agreed that the aspired solution should involve three elements:

1) computer vision/AI software, 2) 3D depth-sensing camera, and 3) business logic software, which would receive the inputs from the CV/AI software and camera hardware. After an initial feasibility study, the scope of the project was expanded to include:

· Volume detection: The entire trailer volume is measured to detect cargo presence and unoccupied space

· Object detection: Each piece of cargo is identified and tracked continuously throughout the transport

· Movement detection: Any movement of the cargo is detected, creating alerts when defined levels are exceeded

· Person detection: Unwanted passengers are detected when the truck in route and/or doors are closed

· Door status detection: Each door is monitored individually as to its status

To simplify future commercial installation, the software was installed and ran on the camera’s CPU, eliminating the need for an external computing unit. Both two-camera and one-camera options were explored in the feasibility study, deciding to use a one-camera system for both hardware cost reduction and retrofit simplification.

CPU = “The component of the computer that retrieves and executes instructions; It essentially is the brain of the computer.”

Outcome

The joint team effort resulted in developing a cargo monitoring solution whose benefits included increased operational efficiency and improved safety. A working prototype, which was enhanced over time to include all desired features, was first created and tested in January 2020. Several complications (mostly corner cases) arose in the process, but the team solved them through close collaboration.

Future Outlook

The next step for this project is to launch commercial pilots as soon as the final camera hardware (time-of-flight sensor) is selected to thoroughly test the solution in real-world environments in preparation for commercial launch.

Eager to learn more? Check out the whole project teaser to read up on the details: https://lnkd.in/dENjvSPx

Alina Biermann, Gandert Van Raemdonck, Dr. Joscha Maerkle-Huss, Simone Bahrs, Des O’Regan, Stefan Beller, Eva Walle, Germán León, Todd Stein, Julian Wesch, Hannah Boomgaarden, Nina Enders