TestOJ Websocket APIs

The judge will start a websocket server at port 2501.

Sending a Challenge Request to ws://${SERVER_IP}:2501/judge will start a challenge. And the result will be sent back by the same websocket connection.

Challenge Request JSON Format

${CHALREQ}
{
    "chal_id": ${CHALLENGE_ID},
    "code_path": ${CODE_PATH},
    "res_path": ${RES_PATH},
    "comp_type": "g++"|"clang++"|"makefile"|"python3",
    "check_type": "diff"|"ioredir",
    "metadata": ${CHALMETA},
    "test": [${TEST},...]
}

For details of check_type, see Diff Check Type, IORedir Check Type.

Challenge Response JSON Format

${CHALRESP}
{
    "chal_id": ${CHALLENGE_ID},
    "result": [${RESULT},...]
}

Test JSON Format

${TEST}
{
    "test_idx": ${TESTIDX},
    "timelimit": ${TIMELIMIT},
    "memlimit": ${MEMLIMIT},
    "metadata": ${TESTMETA}
}

The unit of timelimit is msec. The unit of memlimit is byte.

Result JSON Format

${RESULT}
{
    "test_idx": ${TESTIDX},
    "state": ${STATE},
    "runtime": ${RUNTIME},
    "peakmem": ${PEAKMEM},
    "verdict": [${VERDICT},...]
}

The compile error message and additional information will be stored at verdict. The runtime is the overall user time of all runs. The peakmem is the highest memory usage of all runs.

Diff Check Type

The ${TESTMETA} format.

${TESTMETA}
{
    "data": [${TESTDATA_ID},...]
}

${TESTDATA_ID} are IDs of testdata used in the test. The judge will access testdata by opening ${RES_PATH}/testdata/${TESTDATA_ID}.in and ${RES_PATH}/testdata/${TESTDATA_ID}.out.

The directory structure of ${RES_PATH}.

${RES_PATH}/
    testdata/
        1.in
        1.out
        ...

IORedir Check Type

The ${CHALMETA} format.

${CHALMETA}
{
    "redir_test": {
        "testin": -1|${TARGET_FD},
        "testout": -1|${TARGET_FD},
        "pipein": -1|${TARGET_FD},
        "pipeout": -1|${TARGET_FD},
    },
    "redir_check": {
        "testin": -1|${TARGET_FD},
        "ansin": -1|${TARGET_FD},
        "pipein": -1|${TARGET_FD},
        "pipeout": -1|${TARGET_FD},
    }
}

The ${TESTMETA} format.

${TESTMETA}
{
    "data": [${TESTDATA_ID},...]
}

Each ${TARGET_FD} is the individual target file descriptor which the corresponding soruce redirect to. Setting to -1 means do not redirect the source.

${TESTDATA_ID} are IDs of testdata used in the test. The judge will access testdata by opening ${RES_PATH}/testdata/data${TESTDATA_ID}.in and ${RES_PATH}/testdata/data${TESTDATA_ID}.out.

The directory structure of ${RES_PATH}.

${RES_PATH}/
    testdata/
        1.in
        1.out
        ...
    check/
        build
        check

The ${RES_PATH}/check/build will be executed to build the checker if it exists.

The ${RES_PATH}/check/check will be run as checker. It can be packed in the ${RES_PATH} directly, or be generated by the build. The checker should be a lambda. It must have no side effects.

There are two environment variables OUTPUT and VERDICT passed to the checker. The OUTPUT is the path of the test output file. The VERDICT is the file in which the checker can write down the verdict.