Wilbur file problem

[Deadshade]

I can still make little sense of the Wilbur file saving and opening formats.
The consequence is that I can't recover the height field that I saved and the (old) manual doesn't help.

So I did a systematic test with the formats that I generally use. It involved an island with a constant altitude of 255 m that I saved in format X and again opened in the same format X. It gave me the results :
- 8 bit BMP Surface : the saved 255 m become 1 m
- BMP Texture : the saved 255 m become 1 m
- PNG Surface : the saved 255 m become 65535 m
- PNG Texture : the saved 255 m become 65535 m
- when I open a file saved in format X specifying in the Open file tab another format Y, I obtain about anything (including an error message).

So when I save a height field, I recover all kind of numbers with the exception of the numbers I saved.
Can somebody (Waldronate ?) explain me how this system works ?
Also what is the difference between saving/opening in PNG Surface and PNG Texture ?

And most importantly how should I save a file so that when I open it later, I get back exactly the same height field I saved ?

[waldronate]

The best format for working with files over time in Wilbur is the MDR file format (all of those options it asks for on save aren't implemented). It saves an uncompressed binary raster of 32-bit floating-point values after its initial header.

In general, an image file type that holds a "surface" has the height field scaled to the image bit depth. The minimum value on the height field maps to 0 and the maximum value on the height field maps to 2**n-1. Thus, an 8-bit format holds values from 0 to 255 and a 16-bit format holds values from 0 to 65535. The goal is to maximize precision at the expense of absolute values.

Wilbur takes the height field and computes an image from it. That image is the texture. A texture is a standard 24-bit color file in the appropriate container format (PNG, BMP, etc.)

[Deadshade]

OK thanks. So what I keep is that if I work with Wilbur, always save in mdr format if I don't want to loose the height field.

I still don't understand why the "surface" formats exist at all as they scale a height map to something that is definitely unwanted, because if I want a height at 1000 then it is because I don't want it at 65 535 or any other number that is not 1000.
I understand now "texture" - it's just an image that should not be used to interpret heights.

While I am at it, I have a few other question.
I was testing the precipiton function and I couldn't find out what the Max Length setting did ; putting it at - 100 or - 1 showed no visible difference and putting it positive didn't seem to show any effect at all.
What is this parameter for ?
Also setting delta at 0.1 or 0.45 didn't make any significant difference - the 0.45 seemed to create flat basins marginally bigger than the 0.1 but I was not really sure of it.
Same for the blend - I couldn't detect any influence varying it from 10% to 100%. What is it for ?

On the other hand, surprisingly, the noise had huge influence - even 2 % applied a few times radically destroyed the whole countryside - mountains were flattened, hills disappeared and everything below hills turned to a pixelised flat plain.
This was not intuitive - I rather expected that noise would keep the rugosity high and the relief rugged yet it didn't - it worked like a giant bulldozer. Actually I thought it was the delta that played the bulldozer function but apparently it doesn't.
What purpose was the noise setting supposed to have and why does it destroy relief so fast ?

[waldronate]

The "surface" formats are for getting data from Wilbur to programs that only support standard image formats. It is a way to get low and high fidelity bump maps into rendering software as well. The reason for scaling the images is that many surfaces don't have a useful representation in just the number of bits available (e.g. a surface from -1000 to +1000 is hard to represent in a 0 to 255 range, while a surface from -1 to +1 is uninteresting (and wasteful of bits) in a 0 to 65535 range).

Precipiton erosion works by dropping an agent onto the surface at a random point and letting it run downhill, moving altitude as it goes. This process repeats for the number of cells that there are on the surface. At each point, the agent locates the lowest adjacent cell, moves delta*(current cell altitude - new cell altitude) from its current cell to the next cell, and then moves to the next cell. Delta is thus (approximately) the hardness of the surface at each point. 0.25 is a good speed vs altitude tradeoff for getting pleasing results. Large values are not terribly helpful here. Tweaking the value doesn't make much difference except in processing speed and uniformity of erosion (smaller delta values take more iterations to get the same result, making it more probable that an agent will get dropped on any particular surface point). A value of 0.5 will make the height difference between the two points zero, while values larger than 0.5 will actually make the moved-from cell lower than the new one; the agent may hop back and forth between the cells, doing no useful work. A value of 0.25 will preserve the original form of the landscape fairly well while still giving a visible erosion result.

An agent will typically push its way downhill until it hits a pit or the edge of the surface (depending on wrap). To keep the agent from getting bound up in a cycle (as can happen with certain delta values) and never finishing, there is an additional lifetime associated with the agent. The maximum length of an agent's movement path is the number of steps specified as Max Length. A negative value is a special flag indicating that the system should compute a useful lifetime (twice the length + height of the surface, or something like that). Setting a small positive value will have a very different effect than a large value. A value of 2, for example, will do something that looks like a lot like a blur (moving energy down just a couple of bins); it is very approximately equivalent to the sort of surface diffusion caused by raindrops on a soft surface. A value of 6 is more like a selective diffusion that leaves ridges, but doesn't form any particular erosional features. Because the agent drops are random, it's hard to hit the exact ridgelines, and so those are more likely to remain. It's only the longer values that can form visible canyons and so on because the agent lifetime needs to be proportional to at least the length of the canyon. The image below shows the original surface and the results of precipiton erosion at max length of 2, 4, 8, 16, 32, 64, 128, and 512.


Blend and noise are only useful for multiple iterations. If a Blend other than "None" is selected, the new surface will be blended with the old one, providing a way to reduce (values less than 100%) or enhance (values more than 100%) the effect of each iteration. Noise adds a little uniform noise after each iteration. It's likely that there is a math error in the code that's causing too much noise to be dropped at each iteration. Note that I haven't use the Blend or Noise operations on this dialog since about two weeks after I wrote them as I much prefer to run several iterations of processing in a block before adding noise, using incise flow, and performing other processing.

[johnvanvliet]

I use BT format as it is 32 bit float
32 bit float IS!!! my working format !!! on most things

and most of the time i have "normalized" the 32 bit float to 0.00000... to 65535.000000...

i use a bunch of other tools ,so i use gdal_translate to move from 32 bit *.tif and ( 8,16,24,32 bit ) isis3 *.cub format to bt
then import the BT file into wilbur
( all bt is is the old /ancient ucf format with the header attached to the file.A binary image with a text header )

then when all is done i use 16 bit unsigned at a FINAL format . wilbur will output a 16 bit (0 to 65535 ) *.png
BUT only a few editors will open it AS a 16 bit png most will convert it to 8 bit


do to a LOT of confusion years ago i got into the HABIT!!! of using a naming convention for images

8 bit - nothing added
16 bit UNSIGNED add "U16" to the name ( imageU16.tiff , imageU16.png imageU16.pgm )
16 bit SIGNED ( -32767 to +32768 ) add "S16 " if a RAW IMAGE
( most formats can not handle this so you have to use a float )
16 bit SIGNED add "S16F" if a floating point tiff

if floating point - nothing added for a tiff ( my default for this format IS 32 bit)

Then there is the LSB and MSB issue
or what is called the IBM ppc VS. Intel x86 CPU war
- this is not much of an issue these days

not all software can handle 32 bit and 16 bit images
the NAME i use dictates the software i use with it

[Deadshade]

Setting a small positive value will have a very different effect than a large value. A value of 2, for example, will do something that looks like a lot like a blur (moving energy down just a couple of bins); it is very approximately equivalent to the sort of surface diffusion caused by raindrops on a soft surface. A value of 6 is more like a selective diffusion that leaves ridges, but doesn't form any particular erosional features. Because the agent drops are random, it's hard to hit the exact ridgelines, and so those are more likely to remain. It's only the longer values that can form visible canyons and so on because the agent lifetime needs to be proportional to at least the length of the canyon. The image below shows the original surface and the results of precipiton erosion at max length of 2, 4, 8, 16, 32, 64, 128, and 512.

Well actually this movie exactly shows what I was wondering about. You see that a visible effect takes place in the 2 first steps (e.g 2 and 4). After that almost nothing happens anymore - the shapes are only marginally changed regardless whether you take 16 or 512. Also there are not more (or longer) canyons with 512 than with 8. So a factor of 100 in lifetime doesn't seem to make much difference.
I can understand your explanation about the life time but it looks like if the agent basically stopped working after a life time of 8 or so. Why is it so ?

Noise adds a little uniform noise after each iteration. It's likely that there is a math error in the code that's causing too much noise to be dropped at each iteration. Note that I haven't use the Blend or Noise operations on this dialog since about two weeks after I wrote them as I much prefer to run several iterations of processing in a block before adding noise, using incise flow, and performing other processing.

Yes I understood that and have been testing these things in a short series of iterations.
Seeing the "nuking" effect of the noise, my question was "What was the original purpose to add this function here ?"
Just make a test of 3 iterations with a random setting and set noise at 5% for each iteration - you will see what I Wonder about.

[waldronate]

Most of the energy in that GIF is spent in filling in the basins. The deep cuts (that is, obvious canyons) are generally where a small basin has filled and the filled area becomes a catchment to run a lot of agents through the single low point. The erosion along the smoothish slopes doesn't have much visible effect at all. Plus, because most basins on that surface are in fact fairly small, the life of an agent is more likely to be terminated by running into a pit than by running out of allowed steps.

The noise thing looks to have a bug where it's more likely to push the surface down. If you respan the surface to 0 to 1000, then you'll see that there is little effect overall, except for the noise. The noise effectively forces the lifetime of an average agent back to one or two steps because that's how long the agent will live until it hits a pit. As I said before, it seemed like a good feature at the time, but precipiton erosion really needs to run a couple of passes between noise additions and the per-pass noise feature just isn't terribly useful.