Back again, the Mr hollowman!!! It’s been quite sometime “once more” since my previous post, but this is due to many things, mostly getting back late from work, which didn’t give me enough time to work on my personal side projects. Thankfully though i was able to make few things, some related to Mirage and others related to my Unreal Engine deep digging endeavors . While this post was planned to be Vulkan API one, about a problem i got while working on Mirage, and how the API limitations and rules led me to trick the API with it’s own rules to get what was not possible, but i did got the plan changed at last minute. And hold my post into the drafts as i’ll be upgrading it with much more findings (it’s about depth anyways, so when you see a depth related post, you shall know it was that one) and wanted to make this time a post that is more about showing some stuff that i felt happier user after I’ve coded in Mirage. The things mentioned below, and more other quality of life stuff, are not actually related to Vulkan API, but they are part of Mirage and the way i want it to be used. So let’s dive in that tiny features showcase!!!
Bookmarks!
Yet, believe it or not, every time i’m working on an issue or a feature, and there is a perfect angle to show it, i do change the code of Mirage to set the default view matrix to a hardcoded value at the start of the rendering loop. That was naive way to work, specially when i do need multiple views to compare, and I always knew that needs to be changed. But it was a matter of priorities. I need to store and to add bookmarks! so I can jump right away to the view i want.
Yes! that was the way i do it for multiple years for now, just to avoid wasting the time in tooling!!
I did have experience before with such a bookmarking workflow, with Unreal and Unity3d at least, and both usually allow for a fixed amount of bookmarks, usually the amount of the numpad numbers or the FUNCTION keys, so the numpad/F-Keys can be used and utilized as a quick shortcuts. But I’ve always wanted more, specially when working in larger world. And the way to get around that, was usually by adding cameras to the world, which is not that good.
Unity decided to go with the F keys to get up to 12
Where Unreal decided to stick with the num keys from 0 to 9
So, i decided to get the inspiration from those tools, to create bookmarking system, but at the same time, avoid their limitations, and let it open to have endless amount of bookmarks! Not only that, but give them names, so it can be more recognizable when the list gets larger.
Oh, fancy…I finally don’t have to hardcode views anymore!!!
And even better, i can go through all the list via PgUp & PgDn Keys!!! Isn’t that exciting!!!!
FOV ease access
One of the things that i really really love in 3d viewports, is the FOV. They can make thing look much much better, regardless using high or low value, it is all based on the style and realism of the displayed objects, or the final image in general. They can make a character look strogner, more realistic in terms of proportions, and can make it look cartoony. Usually when i deal with Unity3d or Unreal, the FOV of the view is changed from one of the viewport many menus, but that might not be very iterative when you need to keep trying out values from different angles.
There you go. A context menu!
At the same time, I never understood why most 3d engines and applications, would allow the mouse scrolling to zoom. While the mouse movement while holding RMB will do the same function! So i decided to get rid of one, and keep the other. Where the RMB hold-and-move made more sense for me to zoom in/out, the scrolling made perfect sense to me to change the FOV at any time!
Wide angle, narrow angle, all can be served in more intuitive way!
Navigation Speed
As the name implies, this is the speed of movement around the void. And where unreal’s way is very naive, where there is a slider at the top of the view-port. Or it is not??!!
needs multiple steps to change it for once!
At the other hand, I found Unity’s way is very very handy. Which is basically scrolling up or down while holding the RMB, and this will change the movement speed. And with the notification system showing the current viewport navigation speed whenever it is updated (the pop-ups at the middle of the screen), you can say i went all the way as Unity. This feature in Mirage was built exactly the same as Unity3d! When i spoke about Unreal i said “Or it is not”, because i did find that some editors (not the main level editor) can actually low down the navigation speed “a bit” with mouse scrolling, but it can’t go to super slow or deadly fast as Unity3d would do.
Very handy!
All in all, those are few things that i wanted to share, that i find makes the navigation and the overall dealing with the view much more friendly, there are defiantly other quality of life things, but let’s put those into their own episodes to avoid getting a super long article here…I’m not good at writing anyways as you can tell =D
It’s been a while since I did post in here, in fact been even longer since i did post about Mirage. But during that silence, i was still working on…Let’s say some different “Stuff”. And one of the main things I was casually working on during that time was defiantly Mirage.
There are a ton been made in Mirage, really a ton of things, starting from tiny things in loading files, writing files, or re-integration of the entire UI, up to complicated things such as full PBR bones skinned and animated meshes. Today however i want to spot something specific, which is the PBR & IBL rendering itself, regardless what type of mesh data been rendered. And as you might have noticed i put a title for the post as “PFR” to stand for Physically Faked Rendering! And “IFL” as you might have guessed to stand for Image Faked Lighting!
So why I do call it like that?
Well, before adding the full PBR support in Mirage, all what i knew about the PBR approach is the overall idea, what we see at the engine user end of something like Unreal or Unity or even any other engine that support PBR, no more details. Just use specific textures (roughness, metallic,..etc.) to define the surface being rendered. And that’s all of it! And so as my idea about the image based lighting, just put an image to the environment cube, and it is used to light the scene. But when the time came to implement it, i needed to start reading a lot more in depth about the process (At the end of the post, I’ll leave all the links for the articles that helped me to understand and implement that), but at the end of the day, it turns out to be another “emulation” to the feature. Like every thing else about rendering, it is faked emulation, stating from the basic movement of the camera in 3d scene up to things like shadows, caustics & everything else in rendering (Pre-RTX may be) need to be faked in order to give the feeling of it’s existence. Same way we fake that shadow been casted by a mesh and light source, we also fake that there is actual real environmental light and real surface definition for a set of polygons.
Anyways, so how that thing works (IMO) ?
1. SkyBox
We create a cube mesh (or load one), that will surround the entire scene, this will simulate the sky itself. And that’s why it is common to be called the “SkyBox”. I did knew that, but what i didn’t is as follow: The skybox shader itself, is actually the simplest shader here, nothing fancy at all about it! Just a shader to wrap a mesh with texture! One thing to keep in mind here, that this skybox mesh model matrix will need to match the view matrix, this way, when we zoom or move around the scene, the skybox remain far away from reach, and most importantly feels like a skybox!
2. CubeMap
That cube (aka sky) with that basic shader we mentioned, will not use just a simple texture to wrap the surface of the sky. Nope, it’s a 6 faces texture applied to it, this is called “CubeMap” (if u r not familiar with it), this is usually comes with the format of HDR (the type and format will actually depend on many things, my first implementation was actually a normal 6 textures stitched, and later i did support the read of HDR & KTX fiels).
The “Array Layers” is the number of the texture faces, or may be better to say the number of textures per texture!
Now the trick of using this image to light a scene/object is actually very simple, in every inch of the mesh (at pixel level, so better to say at every pixel the mesh is occupying at screen), we check the normal of it (vertex..or face? you guess!) and using this normal to get the color from the SkyBox texture, this will be basically the light color that will be sampled to this area of the mesh, and It’s why i’ve decided to work for a while in adding a new visualizing mode to show surface normals.
4. Irradiance CubeMap
But we now have an issue! This HDR CubeMap image have a ton of pixels that might have a huge range of colors data, This; First, will be very consuming at runtime. Second, won’t be efficient or even give good lighting results (no smooth lighting), and for that we have to take that huge HDR 6 faces CubeMap image, and re-generate a smaller & “blurry” version of it that holds a lot less information (not actually less, but more usable to light a scene), which called Irradiance CubeMap (the name makes perfect sense for the job!) More useful screenshots about the irradiance at the debugging section below.
We go from this, full detailed cubemap. This original cubemap was 2048px
To this blurry cubemap image with more smoothness & blending between colors. The generated imaged usually 64px or so
5. BRDF
Another texture we need to generate, which called BRDF, this one is basically the core reason to call the entire process “fake” as this texture is nothing more than pre-defined values for the surface/material roughness and camera view angles. It is like a table of values, that we look up through it at run-time while calculating the final color of each fragment (that belongs to an object using the PBR shading) on the screen. If we follow the standard common values, we will always get the exact same visual of that texture that is used in most common engines.
The generated BRDF view from NSight, usually looks like this in any PBR renderer The equation from Unreal Engine Siggraph 2013 talk
6. PreFiltered CubeMap
And one more texture to generate, which is usually called “PreFilteredCubeMap”, which is dedicated to the specualr values based on the surface roughness (or roughness texture map if I would say at this moment). This texture idea is very simple, it is basically a series of mip-maps of the CubeMap, the mip 0 which is the most clear one, will represent a surface with roughness of 0.f, which means it is not rough at all. And go up in the chain of mips, until reaching the last mip-map of the cubemap, which is an equal to the roughness of 1.f.
One thing to highlight here, all those textures are generated at the beginning of the rendering process, even before creating any of the rendering pipelines, which means those are sort of baked textures that kept in memory. So what does this mean? Well, it means if the skybox for example will be moving (rotating for example) at runtime, the lighting on the character will be still the same, as the textures already baked based on specific rotation of the skybox, so if the intention is to have static skybox, then it’s okay. But if not, then there need to be resources recreation at runtime to match the skybox rotation, which is not recommended. And i believe this is doable and can be faked with pre-baking some variation of textures based on the skybox rotation (1 texture per 90 degree, 4 would be okay) and then blending between them at runtime (this is still under investigation task)!
Debugging
As you might have noticed, I do LOVE debugging modes (aka viewmodes, aka visualize modes) , so I’ve added a view mode for the cubemap irradiance contribution, which was very useful during the entire process of PBR integration. For example, this is to see how the irradiance contribute to lighting the model in PBR, only without the irradiance generation (or to be more specific, without blurry irradiance!)
And here how it looks when the irradiance correctly generated.
The difference is in the amount of details, where in the first image each pixel of the mesh might get a super different color than it’s neighbor, in the second we relying on the blurry version of the cubemap, which makes each pixel transition smoothly in the color with its neighbors. And if you can’t really tell the difference, then here a closer look with the Comperator tool.
And regarding the final composed image, it might look or even feel the same for the first glimpse. This is the version with the full cubemap details
And here how it looks when the irradiance correctly generated (smooth & blurry).
The difference in the final colored pass, is even more hard to find out, specially in a still image, but if you look closely you will feel some sort of grain or noise, which makes the image feel unnatural, and makes the skin texture feel…Having Scabies! And if you can’t really tell the difference, then here a closer look with the Comperator tool.
Now let’s change the cubemap texture itself, to see how differ the irradiance contribution would be with another lighting conditions. here another screenshot, from exact same view for the exact same model, but with different cubemap texture loaded to the skybox and re-generated the PBR pipeline for the sake of update.
Super different! Fully blue model (Irradiance Contribution viewmode), because this new texture is full of blue and sky colors. And to see all together, final render + Irradiance contribution for two different cubemap textures, just for the sake of understanding how that works, behind the scenes
the compositing part of the shader is as simple as “vec3 diffuse = _irradianceTex * _albedoTex;”
Now, to dig it further! I’ve enabled the skybox rotation, which allows me to rotate it, but as expected, the model lighting never get affected, the lighting still the same! And using the new Irradiance Contribution viewmode can allow us to see that. Which is back to what I did mention before, that generating this type of textures, happening at load time, happening once, as it’s a costly process.
Q: So, is it impossible to get the effect of a correct IBL for PBR pipeline while rotating the skybox? A: Well, the answer for this question is actually depeneds on the case. For example, if we think about a real scenario, the skybox is almost static, and never moves, specially when it is showing some buildings, interiors and whatnot. The point is, as long as the cubemap texture not showing bluesky with clouds only, then it make zero sense to rotate it at runtim (except for testing purposes). The other thing, it depends on the generation time for the irradiance cubemap texture. At my case it take 180.ms to generate, which might be great in terms of a process at load time, but makes not sense to update it at runtime.
At the meantime, you will notice from the video that i do add more of debugging features than actual rendering features. And this is really important for me at this early stage, as it became very helpful to find out things & understanding more behind the scenes using those magical checkboxes!
And of course, the head is from the 3DScanStore, it’s free and great item for testing! I didn’t use all the content that comes with it, and I’ve to paint some textures to fit the PBR pipeline as it is clearly it was not built for PBR!
Next related post, I may show and talk more about the animated/skinned meshes & PBR in Mirage.
Since my last post about “Mirage”, I had the chance to work on different branches in adding several new features to the engine. And while it is a complicated long term project & a side project as well, the progress becomes slow, but still happy with the outcomes regardless the amount.
Screenshop Mirage now have a new tool called Screensho[p], which by that weird name you can easily guess it is the screenshot-ing tool in within the engine. Now no need to capture screens with a third party helper app or any external tools. There is now a full screenshot-ting support that is copying right away from the buffers to a new imageview. In the past years I had the chance to work with several commercial and in-house engines, while those engines either not allowing you to do screenshots without adding your own solutions (Unity as commercial example requires you writing editor extension for that goal), or giving you the opportunity to capture screenshots with many limitations (Unreal for example as an existing limited tool) such as file format, UI capturing and quality.
While writing the Mirage in-engine screenshot tool, I decided to start adding more options, like file format and UI which used to annoy me a lot with in in-house engines more than commercial engines. While in something like Unreal UI will not be captured, in Unity it will be captured, and will look ugly most of the time!
Check out the UI in this Unity based game I made earlier, it is captured using a Unity editor extension I made
Next step will be adding more functions to scale up the image. But yet, satisfied with the current options and generated captures….May be not satisfied with the JPG captures 😀 but what can we do JPG is JPG after all!
Another interesting part (for me at least) regarding this feature, was checking for & creating a user folder. It is something i didn’t think about it yet. The engine will need to store many values and/or files & user settings. So I decided to start handling this now while not much data need to be stored yet. And for the first time in the past ~8 years to handle such a thing in pure cpp. Have always been doing system/files related stuff either in C#, .Net or Py as the most common interfaces with any engine/API I used to use. And coming after those many years of dealing with Windows[C# & Py] to windows[C++]; I only must say, dealing with windows system in c++ got a lot worst over the years! And btw, the engine footage in this article, in addition to the post featured image, are captured with Mirage’s Screenshop tool.
HDR & Cubemaps While working on loading textures to test out models loading & the PBR shaders, I started in a new branch to load textures, as yet all the models were using color based textures. Yes Mirage yet can load textures to sample on meshes, but limited usage, so it was the time to expand that feature and make it BIG. And glad to say that Mirage by now can not only load all textures formats (JPG, PNG, PSD, TGA, TTF,…etc), but also all types. For example apart from the common file format for images, Mirage can read HDR & KTX at the same time as high ranged images, and for each can load as HDR, LDR or Cubemap. All those options can be found in other engines of course, but not all of them at once!
A capture showing couple of PNG, JPG loaded along side to an HDR & RTX. Youll notice that the seleced image format is R8G8B8A8 while being HDR image as you can see, this is because for that example I done conversion from HDR to LDR
While I’ve to stop now, there are many other features been added, such as MSAA, VSync, Editable Wireframes, CollectionAssets, TurnTable Matrix, Editor options…. and much more, will sign off for now, and will be posting again about other things when I get the time. If you’ve any questions, feel free to reach out!
After Vulkan based investigations I made few years ago, I started right away in my self-driven-like-a-lost-fish-in-the-ocean heavy experimentations in order to prepare the path for myself to start in creating a Vulkan (only) based engine. Till this moment I’ve built several 3d apps/engines, two crappy ones (one crapy & one crashy!), and the current one which is in progress and (hopefully) will be the best of the worst! The first two iterations contained a ton of branches at the repository, enough to confuse me, not to mention the thousands of lines commented either because it is not working, or because I was testing with. And crashes all the time for no safety or assertion check!
I learned a lot since that post, especially because I was officially super far far away from graphics coding for near a decade (fixing Unreal bugs or writing shaders not really something to count -_-). Glad to say, a TON of lessons learned from trial & error and happy to say no hardware loss yet =D. But the most important lesson was the must of getting as far away as possible from using short cuts combinations, as I always either forget them, or re-assign several tasks to the same short cut, and sometime it get worst (keep reading about that unique case). And try more to invest in a better solution. Today’s issue that I want to highlight as part of the short cuts, is the debugging, view-modes & visualizations.
Since I started at the first engine iteration, NSight and Renderdock have always been my lovely friends, can’t deny the great help i got from them, not only to detect issues, but also to learn more about the new API. But at the same time can’t ignore the time get wasted sometimes to visualize or dig deep into something related to my passes or shaders. Not to mention that I was assigning the debug views into the numeric keys in my keyboard next to tilde’~’ (0-9) but as soon as I exceed that number, it started to be messy and a big as hell issue from all aspects!
Coming for a huge Unreal Engine based background, I must get some inspiration, I’ve always admired the tools in there, the ideas and the quality is top notch, so I decided to learn from that good user experience, and try to do a similar thing.
I loved this endless list of Unreal Engine viemodes and visualizers, and I found it will be a good milestone for my engine, in order to make it more usable and user friendly, even if it is not something that will be used by millions of users, but still, making it great is a priority in order to push my productivity and experience. So,…I started working on it!
After I finished adding similar solution, and I became very satisfied about it, and freed the memory allocated in my brain to remember near 20 use of keys and combinations. I came to the two major question I always like to ask:
How to make my implementation different from the inspiration source “Unreal Engine”?
How to make it better than any other engine exist?
I run away to check how Unity implemented this user experience, at the end of the day, Unity3d & Unreal Engine is the most common and diverse with features free and accessible engines out there. Unfortunately found it handled through the post-processing stack profile new thing. Which makes it much less usable than Unreal Engine, and makes it far away from my goals.. Making it better. But I appreciate the ability to edit parms foreach mode! Which became an inspiration for something else (will discuss in another post).
This led me to a bit of confusion, but that was good phase, and now I’m really happy to be confused. Because this confusion made me decided to add one more essential question to ask myself, which should help me in defining the how of making the inspiration better. The question was “Why I want it to be better than Unreal Engine ones?” ! The How doesn’t matter, or in another meaning, the How will be easily defined by the Why.
While I used to use Unreal 3 and now 4 for almost a decade, and have always been happy user and satisfied with those type of tools in there, asking myself the “Why” question helped me to see what been a hidden element in my user experience, and putting my hands on the reason that could led to decreasing my usability sometimes. That hidden element was the ability to compare between different views or buffers at the same frame in a decent way. I know Unreal have the “Overview” thing, but this show several buffers, far from each-other and even worst, in tiny squares.
For me, most of the time I was flipping between different views to check for whatever i look for. When I had those modes linked to the keyboard buttons in my code, I was able to toggle quickly between any two different viewmodes or buffers, and check whatever I’m seeking. So, my issue in the mapped keys was the hilarious numbers of shortcuts I need to remember, but my blessing with the mapped keys was the ability to compare in an efficient way. Voilà  !
And that was the outcome of this personal brainstorming. To make it better, I need to fulfill the need of being able to easily compare the modes/buffers.
As soon as I switch to anything, but the final lit mode, a tiny lil slider will show up, to allow me in doing the “Split Visualization” of the modes. Not only that, but also I can compare two different materials/shaders assigned to the same mode (you can see cell shaded material in the video below for example). Later I’ll blog more about my implementation for the viewmodes, as there are some nice stuff going on under the hood, and some optimization tricks I made in order to not fall again into the issues or crashes I got at the first iteration. Not to mention some Vulkan tricks been learned the hard way.
Last time I blogged about Vulkan I said I’m 50/50 sure about it. But after ~couple of years by now since I started with VK, and more than what I ever wrote of lines been written in those crappy engines, I’m glad to say I’m 99% sure about it. And no\ way back to OGL! I’ll try to blog frequently about Mirage engine, and if all goes smooth to the end of my plan, I might put the entire Mirage source online. Not sure yet!
Born in Egypt & worked there, traveled a lot to work in different countries & now working in SE-Asia as a Programmer for PC & Consoles in the AAA industry (Gameplay, AI, Engine/Graphics, UI, Tools). Very passionate about games and like to write codes using any language, SDK, Framework or even engine. Like to have new challenges to learn new thing everyday.
Note:
*All the work here is personal and represent what I'm doing in my free time(Because this site just for my personal & free time activities.), for a list of commercial projects I took part in, please contact me; or simply check my LinkedIn profile.
*More about me in the about section
Follow !
Subscribe for “Sunday A Sunny Day” news !
By subscribing to the weekly news, you will be able to get weekly email with the updates.
[mc4wp_form]
