HERB: Human-augmented Efficient Reinforcement learning for Bin-packing

Abstract

Packing objects efficiently is a fundamental problem in logistics, warehouse automation, and robotics. When dealing with highly diverse 3D objects (household or grocery items), closed-form solutions are infeasible, and heuristic or model-free Reinforcement Learning~(RL) methods tend to focus solely on geometric optimization, relying on exhaustive searches of the discretized solution space. This leads to long training times (for pure RL) and high latency (heuristics), limited transferability to robotic scenarios, and ultimately ignores object characteristics (fragility, deformability) and human preferences. We propose HERB, a human-augmented RL framework for packing irregular objects, the first to explore the potential of learning from human demonstrations to solve this complex task. It leverages human demonstrations of packing strategies, which inherently exhibit latent factors such as space optimization, stability, and object properties that are difficult to model explicitly. The human-expert data is combined with RL exploration to provide the placement of each object inside the container. Experimental results show that our method outperforms heuristic, purely RL-based, and imitation learning approaches in packing efficiency and latency. Qualitative results highlight that our packing strategy produces more stable, human-like arrangements, which we expect to be more appropriate and widely accepted. Finally, we demonstrate the real-world feasibility of our method on a robotic system.