One of the most critical operations in computer graphics is the projection of a 3D world onto a 2D plane. This operation allows 3D objects to be displayed on conventional 2D screens, enabling us to play 3D video games or run simulations. This transformation is achieved through perspective projections. However, issues can arise when these transformations don't account for different screen sizes or aspect ratios, resulting in skewed and distorted images. In this article, we will delve into the process of correcting these perspective distortions.

Perspective Projections: A Primer

Perspective projection is a type of projection in which the 3D coordinates of an object are transformed into 2D coordinates. It creates an illusion of depth and distance. The further away an object is from the viewer, the smaller it appears on the screen.

However, the proportions of objects can sometimes be distorted due to different screen sizes and aspect ratios. The goal of our investigation was to correct this distortion.

A typical function for creating a perspective projection looks like this:

// Method to set the projection matrix used in OpenGL rendering
private static void setProjectionMatrix(
    float left,      // arg0
    float right,     // arg1
    float bottom,    // arg2
    float top,       // arg3
    float nearClip,  // arg4
    float farClip    // arg5
) {
    float nearClipDouble = nearClip * 2.0F;

    // Set the elements of the projection matrix
    matrix[0] = nearClipDouble / (right - left);
    matrix[1] = 0.0F;
    matrix[2] = 0.0F;
    matrix[3] = 0.0F;
    matrix[4] = 0.0F;
    matrix[5] = nearClipDouble / (top - bottom);
    matrix[6] = 0.0F;
    matrix[7] = 0.0F;
    matrix[8] = (right + left) / (right - left);
    matrix[9] = (top + bottom) / (top - bottom);
    matrix[10] = -(farClip + nearClip) / (farClip - nearClip);
    matrix[11] = -1.0F;
    matrix[12] = 0.0F;
    matrix[13] = 0.0F;
    matrix[14] = -(nearClipDouble * farClip) / (farClip - nearClip);
    matrix[15] = 0.0F;

    // Load the created matrix into OpenGL
    gl.glLoadMatrixf(matrix, 0);

}

Incorporating Field of View (FOV)

Field of view (FOV) plays a critical role in 3D projections. It represents the extent of the observable world at any given moment. FOV essentially determines how zoomed in or zoomed out the final image appears. Horizontal FOV (hFOV) and Vertical FOV (vFOV) represent the FOV in the horizontal and vertical directions, respectively.

To correct the perspective projection, we started by extracting the hFOV and vFOV from the projection matrix. The projection matrix, in computer graphics, is a matrix that transforms the coordinates in the view space to the clip space. We reverse-engineered this matrix and replaced hardcoded FOV values with the extracted hFOV and vFOV. Here is a simplified example of such a function:

The horizontal and vertical FOVs (field of view) are calculated based on the values of the left, right, bottom, and top clipping planes and the near clipping distance. They represent the extent of the observable world that can be seen from the perspective of the camera at any given moment. The calculations use the formula 2 atan((right - left) / (2 nearClip)) and 2 atan((top - bottom) / (2 nearClip)) respectively, which are derived from simple trigonometry.

// Method to set the projection matrix used in OpenGL rendering
private static void setProjectionMatrix(
    float left,      // arg0
    float right,     // arg1
    float bottom,    // arg2
    float top,       // arg3
    float nearClip,  // arg4
    float farClip    // arg5
) {
    float nearClipDouble = nearClip * 2.0F;

    ...

    // Calculate the horizontal and vertical field of view
    double hFOV = 2 * Math.atan((right - left) / nearClipDouble);
    double vFOV = 2 * Math.atan((top - bottom) / nearClipDouble);

    // Convert to degrees
    hFOV = Math.toDegrees(hFOV);
    vFOV = Math.toDegrees(vFOV);

    // Load the created matrix into OpenGL
    gl.glLoadMatrixf(matrix, 0);
}

Implementing the Solution

With the hFOV and vFOV now available, we could adjust the projection of our 3D world onto the 2D screen accordingly.

Our application involves a function that transforms 3D world coordinates (entityX, entityY, entityZ) into 2D screen coordinates. The challenge was to adjust the entity's screen position (spriteDrawX, spriteDrawY) based on the hFOV and vFOV. Here's how the function looked like:

/**
 * Calculates the 2D screen position for a position in the SceneGraph.
 *
 * @param entityX The x-coordinate of the entity in the scene graph.
 * @param entityZ The z-coordinate of the entity in the scene graph.
 * @param yOffset The vertical displacement for positioning the entity.
 * @return An array containing the calculated screen coordinates [x, y] or [-1, -1] if entity is not visible.
 */
public static int[] CalculateSceneGraphScreenPosition(int entityX, int entityZ, int yOffset) {
    final int HALF_FIXED_WIDTH = 256;
    final int HALF_FIXED_HEIGHT = 167;

    int elevation = SceneGraph.getTileHeight(plane, entityX, entityZ) - yOffset;
    entityX -= SceneGraph.cameraX;
    elevation -= SceneGraph.cameraY;
    entityZ -= SceneGraph.cameraZ;

    int sinPitch = MathUtils.sin[Camera.cameraPitch];
    int cosPitch = MathUtils.cos[Camera.cameraPitch];
    int sinYaw = MathUtils.sin[Camera.cameraYaw];
    int cosYaw = MathUtils.cos[Camera.cameraYaw];

    int rotatedX = (entityZ * sinYaw + entityX * cosYaw) >> 16;
    entityZ = (entityZ * cosYaw - entityX * sinYaw) >> 16;
    entityX = rotatedX;

    int rotatedY = (elevation * cosPitch - entityZ * sinPitch) >> 16;
    entityZ = (elevation * sinPitch + entityZ * cosPitch) >> 16;
    elevation = rotatedY;

    int[] screenPos = new int[2]; // X,Y

    if (entityZ >= 50) {
        if(GetWindowMode() == WindowMode.FIXED) {
            screenPos[0] = HALF_FIXED_WIDTH + ((entityX << 9) / entityZ);
            screenPos[1] = HALF_FIXED_HEIGHT + ((elevation << 9) / entityZ);
        } else {
            Dimension canvas = GetWindowDimensions();
            double newViewDistH = (canvas.width / 2) / Math.tan(Math.toRadians(GlRenderer.hFOV) / 2);
            double newViewDistV = (canvas.height / 2) / Math.tan(Math.toRadians(GlRenderer.vFOV) / 2);
            screenPos[0] = canvas.width / 2 + (int)((entityX * newViewDistH) / entityZ);
            screenPos[1] = canvas.height / 2 + (int)((elevation * newViewDistV) / entityZ);
        }
    } else {
        screenPos[0] = -1;
        screenPos[1] = -1;
    }
    return screenPos;
}

We used the tangent function from trigonometry to calculate the new viewing distance based on hFOV and vFOV, and used it to calculate the new screen positions.

Practical Application: 2009Scape Plugin

This theory becomes practical in the realm of game development. For example, in a plugin developed for the game 2009Scape, this corrected perspective projection is used to draw text above items onscreen. This plugin is a quality of life add-on that helps players determine which item drops are valuable and which should be ignored. Accurate world-to-screen projections are crucial in ensuring the text appears at the correct on-screen location, regardless of the player's perspective or window size.

Wrapping Up

This journey into perspective projections and field of view showcases the importance of understanding fundamental computer graphics concepts. By correctly implementing these concepts, we can create a more enjoyable and realistic gaming experience, improve our computational efficiency, and broaden our understanding of 3D graphics in general. It's a testament to the power of computer graphics when well understood and well applied.

Stay tuned for more deep dives into the world of computer graphics and game development. Until next time!