One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learning

In this work, we propose a method, called GenZ-LTL, for learning to satisfy arbitrary linear temporal logic specifications. During training, we sample reach-avoid subgoals, where each subgoal specifies reach subgoal and a set of avoid subgoals. To reduce complexity, we apply a subgoal-induced observation reduction, which simplifies the state representation when possible. We then train a general subgoal-conditioned policy together with value functions. At test time, given a target LTL task specification, we first construct an equivalent Büchi automaton, and then extract a set of reach-avoid subgoals based on the current state. Each candidate subgoal is evaluated using the learned value functions, and the optimal one is selected and executed.

As shown in the Tables, for finite-horizon tasks, GenZ-LTL consistently outperforms the baselines in both success rate and violation rate. In most tasks, it also learns more efficient policies, requiring fewer steps to satisfy a given specification. For infinite-horizon tasks, GenZ-LTL again achieves lower violation rates and a higher average number of visits to the accepting state, demonstrating improved efficiency over the baselines. We also evaluate the zero-shot generalization performance of GenZ-LTL on increasing complexity of the LTL specifications and environment, the results still show that GenZ-LTL significantly outperforms the baselines.

Show References

We present visualization results of the policy learned by GenZ-LTL in the Zone environment. Our method consistently achieves the desired behavior under both complex finite-horizon and infinite-horizon specifications.