Title: Deep reinforcement learning for real-time scheduling in smart manufacturing: A case study from the thermoplastic industry

Speaker: Dr Sharareh Taghipour

Summary:

In real-world settings, manufacturing systems are highly dynamic and encounter uncertainties such as new orders, machine breakdowns, material shortages, and due date changes (or urgent orders). In such systems, real-time scheduling is essential to flexibly adjust production plans and schedules according to the system’s current status. Classical optimization approaches can only acquire optimal (or near-optimal) solutions at a certain point in time and must re-optimize if any changes occur, resulting in significant time complexity. Therefore, the problem beforehand is challenging due to limited decision-making time and complex constraints. By focusing on practical requirements, this research presents a real-time scheduling approach based on deep reinforcement learning (DRL) for the unrelated parallel machines scheduling problem (UPMSP) with batch processing, dedicated machines, sequence-dependent family setups, pre-emptions, and safety stocks. Deep Q-network (DQN) and model-based policy optimization (MBPO) are employed in the presented approach to train two scheduling agents. To address the issue of real-time scheduling, trained agents can accomplish order sequencing and machine assignments in a short amount of time. Furthermore, the trained agents (i.e., the DQN and the MBPO agents) can solve any UPMSP with the addressed constraints without re-training, even when unexpected events and changes occur. In particular, we propose generic state and action representations with dimensions independent of problem scale and requirements while accommodating the considered constraints. Moreover, to enhance the search during the training process, the designed reward function incorporates an upper bound on the optimal solutions, which are obtained using a modified simulated annealing (SA) algorithm. Using data records from an industry partner, test problems are created to assess and evaluate the obtained solutions with the modified SA, selected dispatching rules, and other knowledge-based approaches. The findings demonstrate that the proposed method can achieve comparable results and realize adaptive and real-time production scheduling.

Biography: 

Sharareh Taghipour is Associate Professor and Canada Research Chair in Physical Asset Management at the Department of Mechanical and Industrial Engineering at Toronto Metropolitan University (TMU). She obtained her PhD in Industrial Engineering from the University of Toronto and received her BSc in Mathematics and Computer Science and her MASc in Industrial Engineering, both from Sharif University of Technology, Iran. 

Dr. Taghipour has well-established partnerships and research collaborations with various industry partners from healthcare to energy, mining, transportation, utilities and manufacturing. She is currently serving as the Regional Editor-North America of the Journal of Quality in Maintenance Engineering, and as the Associate Editor of the Proceedings of the Reliability and Maintainability Symposium (RAMS) as well as the Journal of Prognostics and Health Management. 

Dr. Taghipour has received numerous awards; some of her recent awards include PEMAC-Maintenance Management Leadership Award (2022), TMU’s Collaborative Scholarly Research and Creative (SRC) Award (2021), Reliability and Maintainability Symposium – Best Paper Award  (2020), YSGS Outstanding Contribution to Graduate Education Award (2020), Ontario Ministry of Research and Innovation – Early Researcher Award (2019), and TMU Faculty of Engineering and Architectural Science Teaching Award (2018). 

WED 14 APRIL 10:00-11:00 (AEST, Melbourne Time)

ZOOM MEETING ID: 873 1557 5255; PASSWORD: 778635