X-in-the-Loop Simulation for Digital Twins of Traffic Signal Controllers (3)
Performance-Oriented Design and Implementation with TransModeler
"A scalable system is one that can handle growth without requiring fundamental changes to its architecture." - Werner Vogels
TransModeler Threading Model
In an XILS setup involving transportation network simulations, multiple intersections are typically involved. A large network can include tens or even hundreds of intersections, each necessitating a separate external controller process instance alongside the simulator process. This setup presents a scalability challenge.
Figure 1 illustrates the threading model of TransModeler XILS. In this model, TransModeler’s main thread dispatches the SimulationAdvance
event at each simulation step (i.e., every 0.1 seconds). At each step, the simulator sends detector calls to all external controller instances. Subsequently, TransModeler’s main thread must block and wait for all external controller instances to return their signal states before proceeding to the next simulation step.
The blocking wait introduces an inevitable performance penalty due to the sequential processing nature of the microscopic simulation clock, which requires each time tick to occur in order.
Despite this constraint, the process of collecting returned signal states from each controller can be parallelized, presenting a classic fork-and-join scenario for the simulation pipeline.
TransModeler utilizes a high-priority dedicated thread to dispatch DetectorCall
packets to all controllers. Meanwhile, the main thread would block until all controllers send back their load switch states. To avoid inefficient CPU usage, the design must ensure that the main thread’s wait is implemented in a manner that prevents unnecessary CPU spinning and can be promptly waked up. When implemented effectively especially using no-lock atomic operations, this approach can significantly enhance performance.
External Controller Deployment Model
The threading model described applies to the TransModeler simulation process. For external controller process instances, there are two deployment options: they can either reside on the same host computer as the simulation process or be distributed across a network, with each instance assigned a unique IP address.
The two deployment options are only relevant for Software-in-the-Loop Simulation (SILS). In contrast, for Hardware-in-the-Loop Simulation (HILS), the simulator process runs on a single computer, while the external controllers are standalone hardware units (running Linux) that always operate independently.
Preliminary benchmarking has shown that deploying external controller instances on the same host computer together with TransModeler leads to resource contention, resulting in sub-linear scalability. This means that the simulator's speed (real-time factor) decreases disproportionately with the increasing number of intersections under external control.
In contrast, cloud-based deployment offers near-constant scalability, O(1), as there is no resource competition, and TransModeler’s threading model has a well-designed dispatching mechanism. In this setup, the primary factor influencing simulation speed is network latency. Preliminary benchmarking indicates that the overall simulation speed is minimally affected by the number of external controller instances.
Initial tests with cloud-based deployment suggest that the simulator's real-time factor is inversely related to network latency. For instance, if the network latency is 10ms per intersection, the simulation real-time factor could be up to 10 times faster than real-time (100ms / 10ms = 10). Conversely, with a network latency of 1ms per intersection, the real-time factor could be up to 100 times faster than real-time (100ms / 1ms = 100).
Applications
The generic methodologies presented have been successfully applied to integrate two signal controllers, Q-Free’s MAXTIME and SWARCO McCain’s Omni.
This YouTube video illustrates the TransModeler/Omni integration:
Refer to this earlier article for more information about TransModeler/MAXTIME integration: