Jump to content
Devil

How to speed up lightmap build time?

Recommended Posts

Currently it's taking me 1.5 HOURS to build the lightmap in Deflex alpha 10.  The map file is about 1.5MB and lightmap file is about 70MB.  What can I do to speed up lightmap builds in general?  This 1.5 hour waiting per build is killing my productivity.

 

1. Does a lightmap build faster turning off r_bloom, dynamic lights, and other variables? 

 

2. Can those variables be re-enabled after the lightmap builds and have a fully functional lightmap?

 

3. Does the engine use CPU & GPU for lightmap building or primarily GPU? 

 

4.  If I upgrade hard disk from conventional to SSD could I expect a performance increase in lightmap build time or is that tied more to the GPU, CPU, RAM, game engine? 

 

5. What hardware upgrades can I make to help this go faster?  My system specs are in my signature.

Share this post


Link to post
Share on other sites

And here I was thinking 10 minutes on a 1.3 MB .map file was long (lightmap is 15MB).

 

I assume you have covered all brush faces that cannot be seen in darkcaulk?

 

This is the only really proven thing to speed up lighting I have come across, other than getting some Nvidia card that really kicks behind (I have a Radeon HD5850).

 

Something I did note though is to avoid needlessly increasing the total volume the map takes up. E.g. by having some huge tower sticking out of the rest of the map in Z, i.e.the map is 1024u high and the tower is 1200u tall, this ups the volume, and compile time.

 

From your screenshots I am guessing: way too large brushes, way too many light sources, and way too large volume of space taken up by the map.

 

I have an SSD, but not using it for Reflex.

Share this post


Link to post
Share on other sites

Map is geographically huge yes.  I'd take a screenshot but everything is covered in nolight and practically impossible to see.  The world color set to a very dark gray hue to help knock down ambient light.  Back in build 26 we were having a huge problem with ambient light intrusion and that was a way to combat it.  Now with nolight I suppose that isn't as important anymore. 

 

Brush sizes are all within spec.  We don't have a single room larger than 3700 in length or width.  With build 28.4 you can go up to a 8000u brush size.  Brush size is not an issue anymore.  Brush count however might be but the editor works smoothly and the map can be connected to and played fine even at this large size. 

 

All rooms are artificially lit inside.  Yes there are tons of artificial lights.  Last map build I think there were about 100,000 lightcells and 13,000 drawcalls. When I try explaining on the forums just how large my map is I really don't think people comprehend it.  A regular small 1v1 duel map with ambient lighting and no artificial lights would have only a handful of lightcells and maybe 500 drawcalls.  Optimizations aren't just something we do because we want to, they are of paramount importance to the map even loading.  This is why developers have been using Deflex as a proving ground for optimizations.  Still plenty more optimizations needed but for the time being it is playable.  That doesn't really have anything to do with how long it takes to build the lightmap though.

 

One possibility would be to open up some ceilings and let the ambient light do some work instead of artificial lighting.  I haven't experimented with ambient light because it would kill the mood I'm trying to achieve with a high tech theme.  For some of the run courses I guess that's something we could do to open it up a bit but I want to keep the larger rooms artificially lit.

Share this post


Link to post
Share on other sites

It's hard to keep up with all posts on here. :)

 

Okay, the first obvious one is to put the editor_nolight material on all surfaces that don't need to receive lighting, it's not pure black but it's near black (this can be improved in the future). This just means less light-mapping needs to be done, so naturally it will be faster. You can see this used on bdm3 etc.

 

To go further than that, you need to know the light mapper is done entirely on the GPU. So lowering load on the GPU is how you improve the speed of light map compilation. So, basically anything that improves your framerate (lowers the time spent on rendering the game for you) will increase the time that can be spent on the lightmapper. So you can turn off bloom/fxaa/lights/shadows etc, everything that's said here 

 

If you're real hardcore, you can put your game into a really low res too (say 128x128 window) :-)

 

Alternatively buying a top end GPU card will help a lot I believe (although please get people with higher end cards to test for you first!).

 

Also 1.5hours is nuts... bdm3 builds for me in around 2-3 minutes last time I checked (I have a 780ti).

 

There's lots of other things we could do with the tech to reduce the time, but unfortunately I can't spend months messing with the lightmapper code... yet :)

Share this post


Link to post
Share on other sites

I've already plastered every brush with nolight then retextured only visible faces.  Already done every optimization technique I know of.  I'm sure the issue has to do with the hundreds of artificial lights all over the map and not using any outside ambient light.  Every room is completely enclosed.  Geographically huge map doesn't help.  That's all beside the point though.  I know why it's taking so long and it's not a bug it's a legit build.  Wouldn't be the same if my map wasn't pushing new boundaries of some kind. ;P

 

That is exactly what I needed to know thank you.  I'm thinking it would be best for me to create a special maplight build config to switch to when it comes time to build the maplight. 

Share this post


Link to post
Share on other sites

 

To go further than that, you need to know the light mapper is done entirely on the GPU. So lowering load on the GPU is how you improve the speed of light map compilation. So, basically anything that improves your framerate (lowers the time spent on rendering the game for you) will increase the time that can be spent on the lightmapper. So you can turn off bloom/fxaa/lights/shadows etc, everything that's said here 

 

If you're real hardcore, you can put your game into a really low res too (say 128x128 window) :-)

...

Also 1.5hours is nuts... bdm3 builds for me in around 2-3 minutes last time I checked (I have a 780ti).

 

First off, love your map bdm3, it really exemplifies Reflex (I mean design... I don't play much at all).

 

I was just about to suggest something along your lines of though. When you run a light compile, a move out of the map, and look into the void, this sped up the test compiles I did by 2%...

 

I am guessing with the above said... turning off the 2nd desktop screen, and certainly not watching a movie in parallel will also help speed up the compiles?

 

About the tiny window... a compile.cfg could be loaded that sets up such things on the fly, and then returns the settings back to normal... i.e. also turn off/on all the bloom etc... will give that a try and see how much it helps. Will post the config here then.

Share this post


Link to post
Share on other sites

Did some testing. Wanted to find out how fast my Powercolor Radeon HD5850 PCS+ GDDR5 PCIe 2.1 1024MB (slightly overclocked), that is 4.5 years old compares to the Nvidia cards, and also try out the suggested downturning of gfx settings.... and what that yields (for future reference, all this under Reflex 0.28.4).

 

A I used my up until now default settings during the compile:

r_resolution 1680 1050 60 // Back to "full" resolution

r_lm_stats 1              // Turn off light map compile stats again

cl_gibs_maxcount 128      // Return settings to defaults again
r_dynamic_shadows 1
r_bloom 1
r_decals 1
r_fxaa 1
r_dynamic_lights 1

r_lm_build               // Launch the compile

and then compared them against B the suggested settings:

r_resolution 128 128 60 // Switch to minimum low-res for compile

r_lm_stats 1             // Show compile stats, texels/s, compile duration, bounce

cl_gibs_maxcount 1       // 128 default
r_dynamic_shadows 0      // 1 default
r_bloom 0                // 1 default
r_decals 0               // 1 default
r_fxaa 0                 // 1 default
r_dynamic_lights 0       // 1 default

r_lm_build               // Launch the compile

As test map I loaded bdm3, then launched the above configs from the same spawn point, Reflex is in window mode.

  • Compile times for A 278s and B 255s (8% faster)
  • and then I additionally left the level staring into the void, this yields: A 268s and B 253s (5.6% faster).

So staring into the void at a large window resolutions helps. Best compile time to worst (278s to 253s) is 9% faster!

 

Well at least on my rig. I noted that r_lm_stats stops the clock before the compile is at 100%, probably saving the lightmap to disk?

 

 

Test: Putting Reflex on my SSD to check if that makes any difference... best outside small window compile B using an SSD is 255s... so an SSD does not speed up the light compile.

Share this post


Link to post
Share on other sites

Faster Lightmap Compiles for Reflex

Warning: Providing scripts as they are, use at your own risk.

Synopsis: These scripts help turn down the graphics settings of the Reflex level editor
to speed up lightmap compiles.

The files: There are two config files:

    compile_lm.cfg

r_resolution 128 128 60 // Switch to minimum low-res for compile

r_lm_stats 1             // Show compile stats, texels/s, compile duration, bounce

cl_gibs_maxcount 1       // 128 default
r_dynamic_shadows 0      // 1 default
r_bloom 0                // 1 default
r_decals 0               // 1 default
r_fxaa 0                 // 1 default
r_dynamic_lights 0       // 1 default

r_lm_build               // Launch the compile

turns down the setting and performs the lightmap compile, and

    compile_lm_undo.cfg

r_resolution 1680 1050 60 // Back to "full" resolution

r_lm_stats 0              // Turn off light map compile stats again

cl_gibs_maxcount 128      // Return settings to defaults again
r_dynamic_shadows 1
r_bloom 1
r_decals 1
r_fxaa 1
r_dynamic_lights 1

returns the graphics settings back to their defaults, for best quality visuals.

Important: Be sure to check that the settings I am returning to the default
values are the values you actually prefer! Also set the
"r_resolution 1680 1050 60" line to your monitor's settings.

Installation: Place the .cfg files in Reflex's main folder, i.e.

    SteamsteamappscommonReflex

and then to make it more convenient, add these lines to your editor.cfg:

    bind f3 loadconfig compile_lm_undo
    bind f4 loadconfig compile_lm

 

Usage: Then when in Reflex's editor mode, hit F4 to compile the map, this will turn off
various graphics effects and make the window tiny (128x128 px), to gain an
around 9% reduction in lightmap compile time (on my system). Tip: Look into the void while compiling for a small extra boost. After the compile hit F3, to return the edit window to full screen and return the graphics
settings to their defaults.

 

Note: Obviously the scripts can be extended, i.e. to add more graphics settings that might be temporarily turned down for the compile.

Hope that is of some use.

 

AEon (12/2/2014 6:52:26 PM)

 

Download: compile_lm cfgs by AEon.zip

Share this post


Link to post
Share on other sites

On my Powercolor Radeon HD5850 PCS+ GDDR5 PCIe 2.1 1024MB with an Intel i7 920 2668 MHz, I am seeing quite high activity during the lightmap compile, 30-50% on all four cores (for future reference, all this under Reflex 0.28.4).

 

Making me wonder if Reflex is actually compiling the lightmap on my Radeon GPU?

 

Did two back to back compiles of bdm3, outside the map using above compile_lm.cfg and then did the same compile without the 2nd monitor turned on. Both compiles took 255.6 seconds. So no difference I could tell.

 

So at least on my rig:

  • Putting Reflex on an SSD has no advantage.
  • Turning off the 2nd monitor during compile also has no advantage.

Share this post


Link to post
Share on other sites

Yeah none of that helped my build time at all.  That's about all I can think of that might possibly help with build times.  If an SSD doesn't help then I don't know what will.  I've already upgraded my GPU so another upgrade is definitely not in the cards for me anytime soon.  Maybe we can all use the same map to compare build times.  What version of bdm3 are you using the one that came with reflex in build 28.4?  I can use that and perhaps we can get some system comparison data out of the results.

Share this post


Link to post
Share on other sites

It occurred to me to mention what version of the map I used, indeed bdm3 from 0.28.4... would be interesting to see how much faster other hardware is. My scripts should make it easy to do comparable compiles. Note to fly outside the map, for the compile.

Share this post


Link to post
Share on other sites

Yeah I usually do that especially since my map doesn't have any lights requests out in space. :)  I'll have to find bdm3 again because I deleted all maps in my folder since the day I had the issue of someone coming in to callvote another map and screwing over my progress.

Share this post


Link to post
Share on other sites

132s on 28.4 bdm3 - without config optimizations

129s on 28.4 bdm3 - with config optimizations

Hope more people report their buildtimes we'll use bdm3 as a metric since the lightmap is so quick to build. 

Share this post


Link to post
Share on other sites

Stormcrow,

12.5 % faster for you... nice... that really should help your compiles. I no longer have a normal compile key binding, so will be using the 128x128 tiny window for all future compiles.

Share this post


Link to post
Share on other sites

Update for .36 and .37 -> Lightmap compile speedup:

A My light compile script Steam\steamapps\common\Reflex\compile_lm.cfg

r_resolution_windowed 256 128 // Switch to minimum low-res for compile

cl_show_hud 1             // HUD turned on, to see the stats
r_lm_stats 1              // Show compile stats, texels/s, compile duration, bounce

r_flush_constant_buffers_each_frame 1  // no idea if it makes a difference

cl_gibs_maxcount 1
r_effect_quality 0        // 2 is high (default), 1 is med, 0 is low
r_shader_quality 0
r_shadow_quality 0
r_texture_quality 0
r_texture_resolution 0
r_mesh_quality 0

r_fxaa 0                  // Post AA, ALWAYS OFF
r_hbao 0                  // SSAO
r_texture_microdetails 0  // Texture Micro Details
r_dynamic_lights 0        // Dynamic Lights
r_decals 0                // Decals
r_bloom 0                 // Bloom
r_sun 0                   // Sun

r_lm_build                // Launch the lightmap compile

B Undo the low gfx setting after the light compile again via: Steam\steamapps\common\Reflex\compile_lm_undo.cfg

r_resolution_windowed 1680 1028 // Back to "full" resolution

cl_show_hud 1             // keep HUD turned on
r_lm_stats 0              // Turn off light map compile stats again

r_flush_constant_buffers_each_frame 1  // no idea if it makes a difference

cl_gibs_maxcount 128
r_effect_quality 2        // 2 is high, 1 is med, 0 is low
r_shader_quality 2
r_shadow_quality 2
r_texture_quality 2
r_texture_resolution 2
r_mesh_quality 2

r_fxaa 0                  // Post AA, always off
r_hbao 1                  // SSAO
r_texture_microdetails 1  // Texture Micro Details
r_dynamic_lights 1        // Dynamic Lights
r_decals 1                // Decals
r_bloom 1                 // Bloom
r_sun 1                   // Sun

I run these from the editor mode, Reflex running in window mode (not full screen, to let the console commands change the window resolution)... by flying outside of the map, looking into nothingness. Then hit the F4-key to compile the map, then after compile, hit the F3-key to undo the gfx settings to pretty much full quality again.

I use these bindings in my Steam\steamapps\common\Reflex\game.cfg

bind me f3 loadconfig compile_lm_undo
bind me f4 loadconfig compile_lm

I don't have current compile times as hand, but it significantly sped up the compile time on my system.

Edited by AEon

Share this post


Link to post
Share on other sites

I don't have current compile times as hand, but it significantly sped up the compile time on my system.

​Alternatively, you could simply just change com_maxfps to 1 so your PC doesn't have to concentrate on rendering game frames, instead focusing on the lightmap building process instead. Ideally the game should just pause the game and build the lightmap but it doesn't unfortunately, it still tries to do both at the same time.

Edited by Stalast

Share this post


Link to post
Share on other sites

Would be nice to have a "Benchmark" map!

I have to buy an new machine and want to ask: Is an AMD FX-8350 good enough or should i go for an i7-5820K ?

If Intel - do you think the i7-5820K can handle the new Doom? And would fit a 400Euro GFX Card to that system?

Share this post


Link to post
Share on other sites

I don't think lowering FPS would work, and certainly there'd be a minimum resolution at which Reflex will run well? I can imagine that a super low resolution will actually require the GPU to scale things down. Is that not less efficient?

Share this post


Link to post
Share on other sites

From my tests around Xmas last year, lowering the resolution did help. So did not "watching" the compiler work, but to face outside of the map.

As a test map for reference compiles, BDM3 would be an obvious choice, especially since the map is not that large... some of those new CTF maps with the huge rock formation volumes have very high compile times, so testing with those would be tedious. Back then I suggested folks do a few test compiles, but that never happened... thus it never was clear how much influence the GPU actually has on the compile.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×