Correcting Perspective Projections in 3D Graphics: 2009Scape Ground Items

One of the most critical operations in computer graphics is the projection of a 3D world onto a 2D plane. This operation allows 3D objects to be displayed on conventional 2D screens, enabling us to play 3D video games or run simulations. This transformation is achieved through perspective projections. However, issues can arise when these transformations don't account for different screen sizes or aspect ratios, resulting in skewed and distorted images. In this article, we will delve into the process of correcting these perspective distortions.

Perspective Projections: A Primer

Perspective projection is a type of projection in which the 3D coordinates of an object are transformed into 2D coordinates. It creates an illusion of depth and distance. The further away an object is from the viewer, the smaller it appears on the screen.

However, the proportions of objects can sometimes be distorted due to different screen sizes and aspect ratios. The goal of our investigation was to correct this distortion.

A typical function for creating a perspective projection looks like this:

// Method to set the projection matrix used in OpenGL rendering
private static void setProjectionMatrix(
    float left,      // arg0
    float right,     // arg1
    float bottom,    // arg2
    float top,       // arg3
    float nearClip,  // arg4
    float farClip    // arg5
) {
    float nearClipDouble = nearClip * 2.0F;

    // Set the elements of the projection matrix
    matrix[0] = nearClipDouble / (right - left);
    matrix[1] = 0.0F;
    matrix[2] = 0.0F;
    matrix[3] = 0.0F;
    matrix[4] = 0.0F;
    matrix[5] = nearClipDouble / (top - bottom);
    matrix[6] = 0.0F;
    matrix[7] = 0.0F;
    matrix[8] = (right + left) / (right - left);
    matrix[9] = (top + bottom) / (top - bottom);
    matrix[10] = -(farClip + nearClip) / (farClip - nearClip);
    matrix[11] = -1.0F;
    matrix[12] = 0.0F;
    matrix[13] = 0.0F;
    matrix[14] = -(nearClipDouble * farClip) / (farClip - nearClip);
    matrix[15] = 0.0F;

    // Load the created matrix into OpenGL
    gl.glLoadMatrixf(matrix, 0);

}

Incorporating Field of View (FOV)

Field of view (FOV) plays a critical role in 3D projections. It represents the extent of the observable world at any given moment. FOV essentially determines how zoomed in or zoomed out the final image appears. Horizontal FOV (hFOV) and Vertical FOV (vFOV) represent the FOV in the horizontal and vertical directions, respectively.

To correct the perspective projection, we started by extracting the hFOV and vFOV from the projection matrix. The projection matrix, in computer graphics, is a matrix that transforms the coordinates in the view space to the clip space. We reverse-engineered this matrix and replaced hardcoded FOV values with the extracted hFOV and vFOV. Here is a simplified example of such a function:

The horizontal and vertical FOVs (field of view) are calculated based on the values of the left, right, bottom, and top clipping planes and the near clipping distance. They represent the extent of the observable world that can be seen from the perspective of the camera at any given moment. The calculations use the formula 2 atan((right - left) / (2 nearClip)) and 2 atan((top - bottom) / (2 nearClip)) respectively, which are derived from simple trigonometry.

// Method to set the projection matrix used in OpenGL rendering
private static void setProjectionMatrix(
    float left,      // arg0
    float right,     // arg1
    float bottom,    // arg2
    float top,       // arg3
    float nearClip,  // arg4
    float farClip    // arg5
) {
    float nearClipDouble = nearClip * 2.0F;

    ...

    // Calculate the horizontal and vertical field of view
    double hFOV = 2 * Math.atan((right - left) / nearClipDouble);
    double vFOV = 2 * Math.atan((top - bottom) / nearClipDouble);

    // Convert to degrees
    hFOV = Math.toDegrees(hFOV);
    vFOV = Math.toDegrees(vFOV);

    // Load the created matrix into OpenGL
    gl.glLoadMatrixf(matrix, 0);
}

Implementing the Solution

With the hFOV and vFOV now available, we could adjust the projection of our 3D world onto the 2D screen accordingly.

Our application involves a function that transforms 3D world coordinates (entityX, entityY, entityZ) into 2D screen coordinates. The challenge was to adjust the entity's screen position (spriteDrawX, spriteDrawY) based on the hFOV and vFOV. Here's how the function looked like:

/**
 * Calculates the 2D screen position for a position in the SceneGraph.
 *
 * @param entityX The x-coordinate of the entity in the scene graph.
 * @param entityZ The z-coordinate of the entity in the scene graph.
 * @param yOffset The vertical displacement for positioning the entity.
 * @return An array containing the calculated screen coordinates [x, y] or [-1, -1] if entity is not visible.
 */
public static int[] CalculateSceneGraphScreenPosition(int entityX, int entityZ, int yOffset) {
    final int HALF_FIXED_WIDTH = 256;
    final int HALF_FIXED_HEIGHT = 167;

    int elevation = SceneGraph.getTileHeight(plane, entityX, entityZ) - yOffset;
    entityX -= SceneGraph.cameraX;
    elevation -= SceneGraph.cameraY;
    entityZ -= SceneGraph.cameraZ;

    int sinPitch = MathUtils.sin[Camera.cameraPitch];
    int cosPitch = MathUtils.cos[Camera.cameraPitch];
    int sinYaw = MathUtils.sin[Camera.cameraYaw];
    int cosYaw = MathUtils.cos[Camera.cameraYaw];

    int rotatedX = (entityZ * sinYaw + entityX * cosYaw) >> 16;
    entityZ = (entityZ * cosYaw - entityX * sinYaw) >> 16;
    entityX = rotatedX;

    int rotatedY = (elevation * cosPitch - entityZ * sinPitch) >> 16;
    entityZ = (elevation * sinPitch + entityZ * cosPitch) >> 16;
    elevation = rotatedY;

    int[] screenPos = new int[2]; // X,Y

    if (entityZ >= 50) {
        if(GetWindowMode() == WindowMode.FIXED) {
            screenPos[0] = HALF_FIXED_WIDTH + ((entityX << 9) / entityZ);
            screenPos[1] = HALF_FIXED_HEIGHT + ((elevation << 9) / entityZ);
        } else {
            Dimension canvas = GetWindowDimensions();
            double newViewDistH = (canvas.width / 2) / Math.tan(Math.toRadians(GlRenderer.hFOV) / 2);
            double newViewDistV = (canvas.height / 2) / Math.tan(Math.toRadians(GlRenderer.vFOV) / 2);
            screenPos[0] = canvas.width / 2 + (int)((entityX * newViewDistH) / entityZ);
            screenPos[1] = canvas.height / 2 + (int)((elevation * newViewDistV) / entityZ);
        }
    } else {
        screenPos[0] = -1;
        screenPos[1] = -1;
    }
    return screenPos;
}

We used the tangent function from trigonometry to calculate the new viewing distance based on hFOV and vFOV, and used it to calculate the new screen positions.

Practical Application: 2009Scape Plugin

This theory becomes practical in the realm of game development. For example, in a plugin developed for the game 2009Scape, this corrected perspective projection is used to draw text above items onscreen. This plugin is a quality of life add-on that helps players determine which item drops are valuable and which should be ignored. Accurate world-to-screen projections are crucial in ensuring the text appears at the correct on-screen location, regardless of the player's perspective or window size.

Wrapping Up

This journey into perspective projections and field of view showcases the importance of understanding fundamental computer graphics concepts. By correctly implementing these concepts, we can create a more enjoyable and realistic gaming experience, improve our computational efficiency, and broaden our understanding of 3D graphics in general. It's a testament to the power of computer graphics when well understood and well applied.

Stay tuned for more deep dives into the world of computer graphics and game development. Until next time!

How To: Reverse Engineer Any Private API (iOS/Android and Desktop)

Have you ever wanted to access data from an application that doesn't provide a Public API? Well I've got great news. That application is getting its data from somewhere. You just need to find out how to plug into it! This process is called Reverse Engineering (Or hacking if you want to pretend you're really smart) a Private API. I will document some tips and useful tools that will help you reverse any Private API from any application on any platform.

Reverse Engineer any Private API - Watch the YouTube video here! https://youtu.be/RchCi6E2hVs

Tools

There are a handful of tools that can be used to complete this task. Windows 10 was my platform of choice for working with the data so I'll be sharing what I used on here.

Fiddler: Fiddler is an HTTP/HTTPS Proxy that can be used to intercept and decrypt SSL/HTTPS traffic. This application is also useful for replaying requests, creating custom request, and exporting a request as cURL to be converted into Python 3. Fiddler is free to use, just sign in with your Google Account! Make sure you install the certificate and enable HTTPS mode so you don't miss any requests. https://www.telerik.com/fiddler

MitM Proxy: Man in the Middle Proxy is a great way to read data from Smart Phone Applications. This is what I used to get all the data I needed for my API reversal. Simply download the executable from https://mitmproxy.org/ to start up a server (disable your firewall or open port 8080) and then enter your PC's IP address into the Proxy Server settings of your Phones WiFi settings. After that navigate to http://mitm.it/ on your Phone and install the provided certificate. Follow the provided instructions on http://mitm.it/ and start sniffing!

Tips

Create a text document to save all your finding and especially any useful URL endpoints you find. Having your information organized will help to ensure that you don't waste time on the same thing twice or need to proxy your device over and over again to find what a request should look like.

For more information and an example of the API reversed you can watch my YouTube tutorial here.