*** buffer overflow detected ***: terminated

[ZiyiLiubird]

When I was running textworld via Python, the programmer threw "*** buffer overflow detected ***: terminated" error after finishing several env tasks. Do you have any suggestions for how to fix this error? Thanks!

[MarcCote]

Hi. thanks for reporting this. Do you have an code example to reproduce this? Is there any stacktrace information other than the error message?

[ZiyiLiubird]

Hi. thanks for reporting this. Do you have an code example to reproduce this? Is there any stacktrace information other than the error message?

Thanks for your reply. Specifically, I created about 100 textworld envs, and then I utilized following codes to manage these envs:

import os
import gym
import textworld.gym

with open("logo.txt", "r") as f:
    logo = f.read()

class EnvWrapper(object):
    def __init__(self, rank:int, comm_size:int, st_task:int, all_task_nums:int, max_episode_steps:int=52):
        self.rank = rank
        self.min_task_id = all_task_nums // comm_size * rank + st_task
        self.max_task_id = all_task_nums // comm_size * (rank + 1) + st_task
        print(f"min task id: {self.min_task_id}")
        print(f"max task id: {self.max_task_id}")
        # if self.max_task_id > all_task_nums:
        #     self.max_task_id = all_task_nums
        self.current_task_id = self.min_task_id
        self.envs_dir = "/mnt/vepfs/devel/ziyiliu/tw_games"
        self.max_episode_steps = max_episode_steps

    def generate_new_task(self):
        env_address = os.path.join(self.envs_dir, f"game{self.current_task_id}", f"custom_game{self.current_task_id}.z8")
        env_id = textworld.gym.register_game(env_address, max_episode_steps=self.max_episode_steps)
        self.env = gym.make(env_id)

    def step(self, action:str):
        obs, reward, done, info = self.env.step(action)
        obs = obs.split('>')[0]
        return obs, reward, done, info

    def reset(self):
        self.generate_new_task()
        obs, infos = self.env.reset()
        obs = obs.split(logo)[-1].split('>')[0]
        return obs
    
    def render(self):
        self.env.render()

    def check_update(self,):
        self.current_task_id += 1
        self.env.close()
        if self.current_task_id == self.max_task_id:
            return True
        return False

If one task is done or exceeds the maximum steps, i.e., 50 (not 52 in the envwrapper), then I will call check_update() function and reset() function to generate a new task env.

Usually, “*** buffer overflow detected ***: terminated” error occurs between running after 4-90 envs.

[MarcCote]

Do you know if it's always the same game that fails? Does it depends on the actions sent to the game or is the error happening when loading the game?

Can you share those 100 games (all .ni, .json and .z8 files)?

Also what OS system are you running TextWorld on?

[MarcCote]

I think I might have found the issue. See microsoft/jericho#74 and the fix (microsoft/jericho#75).

[smallscorpion-code]

When I was running textworld via Python, the programmer threw "*** buffer overflow detected ***: terminated" error after finishing several env tasks. Do you have any suggestions for how to fix this error? Thanks!当时我通过 Python 运行 textworld 时，程序在完成几个环境任务后抛出了 "缓冲区溢出检测到：终止" 错误。您有什么建议来修复这个错误吗？谢谢！

I encountered the same problem as you, how did you solve it in the end?

[MarcCote]

smallscorpion-code I'm assuming this is related to #367 ? I not, can you provide a snippet of code that reproduces this? Also, which version of Jericho do you have installed?