PerMIS 2010 Special Sessions

SS1 Performance Metrics for Mixed Palletizing Operations
Organizers: Stephen Balakirsky (NIST) and Henrik Christensen (Georgia Tech.)

Stacking objects onto pallets is the most widely used method of bulk shipping, accounting for over 60% of the volume of goods shipped worldwide. This technique, known as mixed palletizing, is utilized by companies ranging from manufacturers to grocery and beverage distributors to the United States Post Office and Army. Due in part to the wide variation in size, shape and weight of objects to be palletized, successful stacking is more of an art than a science. In addition, due to the large client base for palletizing applications there is no one-size-fits-all metric for evaluating the performance of algorithms or quality of a finished pallet. Add to this the fact that pallet stacking is an example of the three-dimensional cutting stock problem, a variant of the combinatorial NP-hard knapsack problem. This makes it hard to find a common heuristic to satisfy requirements for major shippers and receivers. ?There are many commercially available palletizing solutions that offer software planners as well as complete robotic systems. Almost all such solutions use proprietary statistical heuristics to determine the best pallet and present the user with these statistics to determine the pallet pattern to use. Typically, only an expert user of a particular system can make sense of such statistics.

Once these pallets have arrived at their destination, you experience the opposite problem of unpacking the mixed pallet. This operation must be performed either by humans or automation that works upon a pallet that may have become destabilized during shipping. Once again, commercial solutions exist for this process, but no widely accepted performance metrics for these systems exists. In this workshop, we will feature papers and discussion from end users, manufacturers of palletizing solutions, and academia thus bringing together users these groups to discuss techniques for measuring the performance of mixed palletizing algorithms and the quality of the final pallets.

---

SS2 Unmanned and Autonomous System Test Technology
Organizer: Robert Heilman (OSD-TRMC)

Although recent advancements in the technology of Unmanned and Autonomous Systems (UAS) have improved the effectiveness, suitability, and survivability of the overall system; in large, these systems have failed to achieve a technology readiness level suitable for fielded systems deployed in their respective operational domains.  The insatiable appetite for UAS in theater today has magnified the need to expedite the acquisition of these systems.  In order the meet this demand, it is essential for the test and evaluation community to develop innovative technologies capable of predicting the envelope of performance, emulating realistic test conditions based on the these bounds, and assessing the performance based on accepted measure of effectiveness.  This session intends to explore emerging test technologies being developed to help formulate an understanding of the challenges associated with the test and evaluation of Unmanned and Autonomous Systems.

---

SS3 Evaluation of Human Detection and Tracking for Robot Safety, Collaboration and Interaction
Organizers: Tsai Hong (NIST) and Roger Eastman (Loyola) 

This special session is dedicated to sensor systems for human detection and tracking for advanced automation, with special emphasis on metrics for performance assessment. Accurate and reliable tracking of human position, trajectory, gesture and behavior will be key in many emerging technology, from human-robot collaboration in manufacturing, to automation-assisted surveillance and rescue in public safety, and safety systems in building construction. Presentations will cover applications from manufacturing automation, construction safety, public security and emergency response.

---

SS4 Evaluation of Sensors for Object Pose Estimation in Manufacturing Applications
Organizers: Roger Eastman (Loyola), Tsai Hong (NIST), and Hui-Min Huang (NIST) 

Previous Sensing/Perception Workshops identified object picking and placing as among key operations in advanced manufacturing environments.  

These operations, especially in a dynamic situation, require significant sensing and perception capabilities. This special session provides an open forum to further explore a central issue: how to evaluate the sensory/perception systems' performance to enable the identified operations. The topics include, but are not limited to:

• What is the current state of practice?
  
• What are the challenges?
  
• What are the performance metrics?
  
• How to evaluate the systems?
  
• How to best specify the systems to be unambiguous and helpful to the users?

A panel is being composed that will include practitioners from the system providers, researchers, and users.

---

SS5 Integrated Performance Assessment Through Experimentation
Organizer: Marshal Childers (ARL)

This special session will focus on achievements in live field experimentation and corresponding simulation of autonomous unmanned ground vehicle technology. Emphasis will be placed on integrated performance in that the systems and technology considered apply hierarchical planning and advanced machine perception and control to enable autonomous UGV mobility and navigation in relevant environments and in relevant scenarios. The fundamental capabilities presented are a cornerstone to the incorporation of unmanned ground vehicles in military applications and society in general.

---

SS6 Performance Evaluation for Mapping & Navigation in Unstructured Environments
Organizers: Rolf Lakaemper (Temple) and Raj Madhavan (UMCP/NIST)

Having robots sense unstructured environments and automatically generate a sufficiently accurate world model is still an unsolved problem, and the solution requires a framework for generating accurate representations of the operational domain. This, in turn, requires scientifically sound and statistically significant metrics, measurement, and evaluation methodologies for quantifying intelligent systems' performance. 

For example, automated guided vehicles are widely used on factory floors and warehouses for transport of goods. Currently they require highly structured environments and reference markers installed throughout plants, which, apart from carrying prohibitively high maintenance and installation costs, are not able to cope with dynamic changes in the environment. This has widespread implications for the applicability of AGVs, and also limits drastically the way modern warehouses and manufacturing floors can be designed. A breakthrough will be achieved if AGVs could cope with unstructured, dynamic environments and adapt to human-centered collaboration. A similar analogy can be extended to various domains. 

This special session will thus address the challenges in achieving this goal, and to open discussions on how to create and experimentally validate a world modeling framework for unstructured environments amidst dynamic objects. The session does not only address researchers, but aims to include end-users, developers, and vendors, to open a channel to actively participate in standards efforts in this field.