Archive for November, 2012


November 23rd, 2012 No comments

I’ve added a showcase page to demonstrate some of my more interesting work over the years. Go check it out via the menu bar up top or simply click here.

Categories: Me, Site Tags:

Binary Packaging Continued – Now with fancy pictures!

November 22nd, 2012 No comments

Well, they’re just command prompt pictures, but hey it’s better than nothing right?

I’ve continued to develop my binary packaging system, awkwardly titled the ‘Binpressor’. It has now come fairly far into a stable release. Files can be packaged, and the utility can also unpackage files. Lets step through that.

Here I have a folder that contains all the images I use as part of a Uni project. 84 textures, some of which are high resolution hence the folder size of 54.6MB. Let’s give the utility a test drive by packaging the contents of this folder!

Files are recursively added into a list of prepared files. A header file is then generated based on various settings pre-defined in the program (such as buffer sizes for file names) and each file generates its own descriptor that tells the binary program how large a file is, what the file name is and what type of file it is. Now that we have a fancy little package, let’s now unpackage it, made possible by the handy descriptors that we’ve exported.

I can now open up the folder and verify that the image data has been correctly extracted by the fact that the thumbnails are all correctly generated. Upon hovering my mouse over a file, the file details look correct, and opening any of these files results in the correct display of the image. I have actually taken this a step further, and actually packaged the package utility itself, then unpackaged it, then successfully managed to use the unpackaged file to package another file!

One thing you might have noticed is that the utility lists ‘compressing’ and ‘uncompressing’ lines. Yep, that’s a new feature that I’ve added in today. I’ve integrated a C-wrapper around the LZMA SDK, meaning I have access to fast and efficient compression features. You’ll notice some strange scenarios where the compressed file is actually larger than the uncompressed file, but this can be safely ignored due to the very tiny byte differences.

Here I’m compressing a virtual machine settings file down from a small 104 kilobytes (106,496 bytes) down to an even smaller 223 bytes. Mind you that the size information at the bottom of the window does not take into account compression information at the moment.



Categories: Me, Programming Tags: , , , ,

Proprietary Binary Packaging

November 21st, 2012 No comments

With just over a week left at Uni, I’ve been finishing off items left over before I’m completely done (assuming I don’t fail of course). Meanwhile I’ve already began developing some theories based on some projects I can work on. I’ve recently began designing a modularised hybrid 3D game engine, developed by me with the exception of physics and sound libraries. Over the past few months I’ve been developing mainly with the Ogre3D graphics library, and although it’s flexible, matured and powerful, I’d like to start my own process, partially as a great educational experience – a convenient bonus on top of a bachelor degree, no?

As a small project I’ve already began, I’ve been developing my own proprietary binary packaging format that allows me to store files. Rather than using ZIP, RAR, or other common archive files I’ve developed my own, allowing me to use information stored in my own way, which is also a major security boost as no one will be able to simply extract assets out of the packages. Tthen again,¬†modifiability¬†is always a good thing so it would only be for core files. Again, I back my reasoning behind “an educating experience”.

Currently there is no compression being performed on the packages, and I’m contemplating whether or not to bother as file-size is not so much an issue. Uncompressed data loads faster, and compression is only really useful for distribution and such, which is already handled by compression algorithms in installation software or digital distribution platforms.

The packaging tool is currently simply a console program, allowing me to drop files or folders onto it to recursively package all highlighted files into a single package. At a not-too-distant later date I’d like to start writing a reader allowing me to interpret the files in system memory, and eventually finish the binary package format with an extractor so files can be retrieved outside of memory (which is essentially very easy if the reader is already developed). Whether I’ll develop a graphical user interface for a tool is still undecided at the time, although at the moment I do not find it particularly necessary unless I plan on distributing the application for anyone looking for a light and simple binary package format (except then everyone would be able to read the packages!).

The last item I’ll cover is the mixed architectures I’ve used. Rather than writing a system to handle file splitting and large file sizes, I’ve simply created a 32-bit and 64-bit version of the packager. Code-wise there’s very few changes except for the data types. While the packager is not intended for files larger than 2GB, the option is there should I feel the need to waste my hard drive space.

Apologies for the wall of text, but as it’s a binary format I really don’t have any images to show for this post. If you’ve skimmed over it, hopefully it may be of some interest to you.

Categories: Me, Programming Tags: , , , ,

Fast Look-Up Light Shading using HLSL & Ogre

November 7th, 2012 No comments

Over the past 21 weeks at Uni I’ve been in the process of developing a completely functional 3D game, resembling a third-person, stealth-based ninja game. With three weeks of development left until the final hand-over there’s a fair bit left to do. A major issue that arose somewhat midway through the project was performance due to the complex outdoor environments. The main issue was the fixed-function pipeline lighting system which was causing batches in the scene to be redrawn, resulting in both huge batch counts (thousands) and huge triangle counts (over four million).

The way scene optimisation is set up is by performing static geometry batching on the level itself, an integrated feature of the Ogre3D 1.8 rendering library. This combines the thousands of components that the game’s level is built up of into one or more various regions – right now every 100 game units. Fixed function pipeline lights as well as most common shader-based lighting techniques require a a model to be redrawn for each light that affects it. As levels are built out of rather large regions formed by combining level assets, this results in very large redraws, seriously hampering performance when even half a dozen lights are introduced in the level.

Experimenting with the fast lighting algorithm with colour toning – No normal mapping is present yet.


To solve this issue, a rather experimental lighting technique was introduced. A texture is generated that covers the entire level at a very low resolution (in my scenario, 64×64 pixels). Each pixel represents a region that one light can be placed. The RGB value of each pixel represents the exact position of the light (R – X, G – Y, B – Z) in the 3D world. Each update light information is then applied over the entire world in a single pass with no redraws. This results in extremely fast lighting calculation with absolutely no performance hit for additional lights. What this means is that a scene, in theory, could have literally thousands of lights.

Another benefit to the lighting system is the fact that this is not a form of lightmapping but is infact real-time. This means that moving objects within the level have real-time lighting and even better, lights can actually be moved around as well by updating the generated texture each frame.

Addition of enhanced ambient lighting and normal mapping. Incorrect triangle/batch count due to SMAA shader.


The shader was initially provided by an Ogre3D forum moderator, Kojack, providing a very simple version of the shader in what was definitely no more than 20 or 30 lines within the shader. I expanded this to include real-time normal mapping, texture mapping as well as allowing lights to have individual colour and brightness values to create the intended lighting effect within the scene.

Shader code:

sampler2D Texture1: register(s0);
sampler2D Texture2: register(s1);
sampler2D NormalMap: register(s2);

float3 lookup(float3 mappos, float3 offsetpos, float3 norm, float3 bump)
float3 c = float3(0, 0, 0);
float4 streetlightcell = tex2D( Texture1, offsetpos.xz+mappos.xz );
if(streetlightcell.a > 0)
float3 lightvec = streetlightcell.rgb – mappos*1000;
float dist = length(lightvec);
c = * max((dot(lightvec,bump)/50) * (1 /(dist*0.005)),0);

return c;

void gridlight_vp(float4 position : POSITION,
float4 diffusein : COLOR0,
float2 texCoord0 : TEXCOORD0,
float3 normal : NORMAL,
float3 tangent : TANGENT,
float3 binormal : BINORMAL,
out float2 oTexCoord0 : TEXCOORD0,
out float4 oPosition : POSITION,
out float3 posAsColour : TEXCOORD1,
out float4 diffuseout : COLOR0,
out float3 oNormal : TEXCOORD2,
out float3 oTangent : TEXCOORD3,
out float3 oBinormal : TEXCOORD4,
uniform float4x4 worldViewProj,
uniform float4x4 world,
uniform float4x4 WorldInverseTranspose)
oPosition = mul(worldViewProj, position);
posAsColour = (mul(world, position).xyz)*(1.0/1000.0);
diffuseout = diffusein;
oTexCoord0 = texCoord0;

oNormal = normalize(mul(normal, WorldInverseTranspose));
oTangent = normalize(mul(tangent, WorldInverseTranspose));
oBinormal = normalize(mul(binormal, WorldInverseTranspose));

void gridlight_fp(
float2 texCoord0 : TEXCOORD0,
float4 diffuse : COLOR0,
float3 posAsColour : TEXCOORD1,
float3 norm : TEXCOORD2,
float3 tangent : TEXCOORD3,
float3 binormal : TEXCOORD4,
uniform float4 ambient,
out float4 oColour : COLOR)
// Calculate the normal, including the information in the bump map
float3 bump = 1.0 * (tex2D(NormalMap, texCoord0) – (0.5, 0.5, 0.5));
float3 bumpNormal = norm + (bump.x * tangent + bump.y * binormal);
bumpNormal = normalize(bumpNormal);

float gridstep = 1.0 / 64.0;
norm = normalize(norm);
float3 gridlights = (
lookup(posAsColour, float3(gridstep*-1,0,gridstep*-1),norm,bumpNormal)+
lookup(posAsColour, float3(gridstep*0, 0,gridstep*-1),norm,bumpNormal)+
lookup(posAsColour, float3(gridstep*1, 0,gridstep*-1),norm,bumpNormal)+
lookup(posAsColour, float3(gridstep*-1,0,gridstep*0 ),norm,bumpNormal)+
lookup(posAsColour, float3(gridstep*0, 0,gridstep*0 ),norm,bumpNormal)+
lookup(posAsColour, float3(gridstep*1, 0,gridstep*0 ),norm,bumpNormal)+
lookup(posAsColour, float3(gridstep*-1,0,gridstep*1 ),norm,bumpNormal)+
lookup(posAsColour, float3(gridstep*0, 0,gridstep*1 ),norm,bumpNormal)+
lookup(posAsColour, float3(gridstep*1, 0,gridstep*1 ),norm,bumpNormal));
//oColour.rgb = ambient.rgb + /*diffuse.rgb * */ gridlights;// + float3(ogrelight,ogrelight,ogrelight*0.7);
//oColour.a = 1;

float4 texcell = tex2D(Texture2, texCoord0);
gridlights.b = gridlights.b * 0.8;
oColour.rgb = (float3(0.3137254901960784, 0.3137254901960784, 0.3725490196078431) + gridlights) * texcell.rgb;
oColour.a = texcell.a;
//oColour.rgb =;
//oColour.a = 1.0f;

Categories: Me, Programming Tags: , , , ,