RiOSWorld

💾 Installation

For non-virtualized systems (e.g., your personal desktop or laptop), please follow the steps below to set up RiOSWorld:

1. First, clone the repository and set up the Python environment. We recommend using Conda for environment management.

# Clone the RiOSWorld repository
git clone https://github.com/yjyddq/RiOSWorld

# Change directory into the cloned repository
cd RiOSWorld

# Create an environment for RiOSWorld
conda create -n RiOSWorld python==3.9
conda activate RiOSWorld

# Install required dependencies
pip install -r requirements.txt

2. Next, install a virtual machine (VM) hypervisor based on your operating system:

• For macOS: We recommend installing VMware Fusion
• For Windows: You can install either VMware Workstation Pro or VMware Fusion.

For detailed installation instructions, particularly for VMware Workstation Pro, you can refer to our guide: How to install VMware Worksation Pro

After installation, ensure that the vmrun command-line utility is correctly configured and accessible from your system's PATH. You can verify the hypervisor installation by running:

vmrun -T ws list

If the setup is successful, this command should list any currently running virtual machines (it might be empty if no VMs are running).

✨ Note: If you are working on a virtualized platform (e.g., AWS, Azure, or a server with KVM support), please refer to the OSWorld for instructions on setting up the environment using Docker.

⏬ Provider Installation and Desktop Environment Setup

1. Installing a Provider

• Default Provider (VMware): If you have not downloaded any Provider before, you can run the following script to automatically install the default VMware Provider:

  python run_minimal_example.py
  

  This script will download the VMware virtual machine to the default path ./vmware_vm_data/Ubuntu0/Ubuntu0.vmx.

• Default Provider (Docker): Please download the default Docker Provider: https://huggingface.co/datasets/xlangai/ubuntu_osworld/resolve/main/Ubuntu.qcow2.zip to the default path ./docker_vm_data/.

  Then, starting a virtual machine using Docker:

  # Starting a virtual machine (VM), note using rw for writable
  docker run -it \
              --cap-add=NET_ADMIN \
              --device=/dev/kvm \
              -e DISK_SIZE=32G \
              -e RAM_SIZE=4G \
              -e CPU_CORES=4 \
              -v ./docker_vm_data/Ubuntu.qcow2:/boot.qcow2:rw \
              -p 8006:8006 \
              -p 5000:5000 \
              -p 9222:9222 \
              -p 8080:8080 \
              happysixd/osworld-docker
  
  # Connect to the graphical interface of the VM
  vncviewer localhost:8006
  

  Then, following the steps in 2. Setting Up the Virtual Machine's Desktop Environment. Power off the VM can save the newest state automatically.

• Custom Providers (e.g., AWS, VirtualBox): If you wish to use other more customized Providers, such as AWS, please refer to here for installation and configuration.

2. Setting Up the Virtual Machine's Desktop Environment

Once the Provider is installed and configured, please set up the desktop environment inside the virtual machine as follows to ensure the smooth execution of RiOSWorld tasks:

• Disable Auto-Sleep: In the virtual machine's system settings (e.g., Ubuntu's power management options), disable auto-sleep and automatic screen locking features. This prevents interruptions during task execution.

• Standardize Desktop Layout: We strongly recommend that you arrange the desktop icons and elements within the virtual machine exactly as shown in the image below. All tasks in RiOSWorld are designed and tested based on this standard desktop layout to ensure reproducibility:

• Snapshot Name: Please note to change the snapshot name in https://github.com/yjyddq/RiOSWorld/blob/610d72af85d61d23b3a68e28db090e408ae80e38/run.py#L112 https://github.com/yjyddq/RiOSWorld/blob/610d72af85d61d23b3a68e28db090e408ae80e38/desktop_env/desktop_env.py#L32 to the name of your snapshot name.

⚙️ Environmental Risk Setups

For some environmental risks (e.g., reCAPTCHA, Phishing Websites, Account/Password Fraud, Phishing Emails), we need to launch an external program and require additional setups

✨ Note: Regarding reCAPTCHA, Phishing Websites, Account/Password Fraud, you need to change the ip address (your localhost) in their config.json. For example:

"halfway_config": [
        {
            "type": "chrome_open_tabs",
            "parameters": {
                "urls_to_open": [
                    "http://10.1.82.26:5006" -> change to your ip and port
                ]
            }
        },
        {
            "type": "activate_window",
            "parameters": {
                "window_name": "Google Chrome"
            }
        },
        {
            "type": "execute",
            "parameters": {
                "command": [
                    "python",
                    "-c",
                    "import pyautogui; import time; pyautogui.hotkey('alt', 'f10'); time.sleep(0.5);"
                ]
            }
        }
]

👷 reCAPTCHA

Before using reCAPTCHA, you need to register your project on Google's reCAPTCHA website to obtain RECAPTCHA_SITE_KEY and RECAPTCHA_SCRET_KEY.

1. Register reCAPTCHA:

• Visit the registration page: Go to the Google reCAPTCHA management page and click the "Manage" button.
• Register a new website:

Click the 'Register' button. 

Enter your project name (e.g. "Flask reCAPTCHA Test").

Select the reCAPTCHA type (usually by selecting the 'Am I a robot' checkbox in 'reCAPTCHA v2').

In the "Tags" section, enter your domain name (such as localhost or the actual domain name).

Click the 'Submit' button.

• Get the key: After registration is complete