PDA

View Full Version : Dealing with artifacts in Wilbur



acrsome
04-27-2014, 06:34 PM
My current project is mapping a terraformed Venus- see my WIP in the world/regional forum. I found the MIT topo image- based on Magellan data- for the planet as a grayscale image and loaded it into Wilbur (my first time using Wilbur). I figured using Wilbur might be easier than the hand-drawn stuff I'm transcribing from the USGS topo map of the planet. But the MIT data has some odd artifacts:

63555

This is one small area to give you an idea of what I'm talking about. My question is, as a new Wilbur user, Wilbur seems to be able to do a LOT of cool stuff, but I have no idea even where to begin. Hell, I'm currently working on how to set a sea level...

How can I get rid of those streaks at the left third of the image that Wilbur reads as higher elevations?

How I can extrapolate in a reasonable fashion into the blank spots in the data, like that bit in the upper left? (There are much larger ones elsewhere.)

The grayscale apparently maxed out too low as well- the higher elevations all look like perfect plateaus, as you can see. I'll have to deal with that somehow- probably by hand modifying from the satellite images or something. But that's lower on the priorities I think.

Or would it be better to manipulate the original grayscale image in GIMP or something?

Thanks, All.

waldronate
04-27-2014, 08:43 PM
Without knowing what data you used, I can't offer you any suggestions about the apparent height clipping. If it's in the imported data, then there's nothing that Wilbur can do about it.

The streaks are in the original Magellan data; you'd need to paint that out by hand. Painting in Wilbur is a bit of a pain, especially for the kinds of curve that you'd need to paint along. On the other hand, Wilbur does allow raising and lower of the data, which is probably the sort of thing that you want for that kind of work.

Wilbur does have a (very) limited tool for healing imported SRTM data, but I think that the nature of the gaps will probably prevent it from working well in a lot of cases. The SRTM holes tend to be fairly small and without straight edges; the Magellan data has holes with straight edges that are fairly large.

waldronate
04-27-2014, 08:44 PM
And Sea Level is a property of the shader: Use Texture>>Shader Setup and pick the Altitude tab. Enter your desired altitude in the box labeled "Sea Level".

acrsome
04-28-2014, 11:04 AM
I used the GTDR data set, which was originally complied by MIT, and yes the streaks are in the original data. Essentially, it was a large grayscale JPG file. I changed it to a PNG per your suggestion to get Wilbur to load it. I found it in cylindrical projection on an astrogeology website, so that I can now churn it through G.Projector. Score! (The raw files available from NASA are all Mercator, sinusoidal, or polar projections. Grrr.)

I could certainly send it to you if you like, but I guess you've answered my question already. Answer = yes, it will involve meticulous manipulaiton by hand. Sigh. I guess I have a project for the next six months...

Thanks for the sea level pearl. I'm kind of wading through your neat little program at a very slow rate. I know so little about mapping that I end up going to the (dated) pdf manual or wikipedia a lot. I guess I'll start by playing with the sea level , and then see just how much of those streaks I have to deal with- many may be under water.

Thanks, again!

Redrobes
04-28-2014, 05:08 PM
Post the image - I may be able to help there.

acrsome
04-30-2014, 10:07 AM
I'm not sure if I can. I's 32MB, and isn't there some sort of forum size limit here?

Nonetheless, when I get home I'll see if I can put it in googledocs or some other filesharing thingy or something. (Which I've never done before, so bear with me.)

Redrobes
04-30-2014, 10:17 AM
Whats the URL of the original MIT / astrogeology website file then ?

Can you import these files instead ?
http://pds-geosciences.wustl.edu/mgn/mgn-v-gxdr-v1/mg_3002/gtdr/merc/

acrsome
04-30-2014, 11:59 AM
That's the data, yes. Those are the original GTDR data files I mentioned. (As I'm sure you know, you were in the mercator directory, but sinusoidal and two polar projections are avaialble in a higher directory.)

I suspect that these streak artifacts are in this data too, since my file is just one large stitched-together image of the same data. I stumbled upon a site that would export them as one large jpeg in cylindrical projection for me. That's where I got mine, since cylindrical is more useful, being able to load into G.Projector and all. (I am far too ignorant to manipulate those files into an equirectangular cylindrical projection on my own. And I'm also not sure how to change the CD images into a useful file format.)

USGS Map-A-Planet (http://www.mapaplanet.org/explorer-bin/explorer.cgi?map=Venus&layers=venus_gtdr&west=-180&south=-90&east=180&north=90&center_lat=0&center=0&defaultcenter=on&grid=none&stretch=none&projection=SIMP&advoption=NO&info=NO&resolution=2)

This was the website I used to order the jpeg. You have to dig around a bit to figure out how to order it at the resolution and in the projection that you want. The original GTDS data is about 4.6km/px, so that's what I ordered. It takes them about a day to put the jpeg together for you and email the download link (which is only good for a few days).

I also took a closer look at my stitched-together cylindrical image in Wilbur last night, and there are many more subtle north-south streaks all over the place. I think I have a lot of work ahead of me.



EDIT--- I just re-ordered another version from Map-A-Planet, this time in 16-bit TIFF rather than 8-bit JPG to hopefully get more detail in elevations. (See how much I've learned already?) When I get the link I'll post it here for others to download.



EDIT AGAIN-- Well, you got me thinking about this again, so I was poking around the ISIS forums and found this:

ftp://pdsimage2.wr.usgs.gov/pub/pigpen/venus/Radar_properties/

It looks like there is some sort of topo data in that directory, too, but it is again beyond me. What's frustrating is that I suspect that cleaning up this data and producing a decent image at that resolution would be trivial for some of these astrogeology guys, but I'm struggling. But then, I guess I don't have their resources, eh?

There is also something called the RDRS spherical harmonic topography model available:

http://pds.nasa.gov/ds-view/pds/viewDataset.jsp?dsid=MGN-V-RDRS-5-TOPO-L2-V1.0

http://pds-geosciences.wustl.edu/mgn/mgn-v-rss-5-gravity-l2-v1/mg_5201/

From the description in the first link this might actually be nice data, but I just can't figure it out. And I have no idea what the resolution is, projections, file formats, etc. But the filesizes are quite small, so I suspect that the resolution is poor, and I moved on.

acrsome
04-30-2014, 02:13 PM
Wow, that was fast. Here's the link to download that TIFF I just ordered:

Planetary Data System - Map-a-Planet (http://pdsmaps.wr.usgs.gov/order/dean/venus00n000.html)

Warning: it is 54.1MB, and the link is only good through May 4th. But it looks like it preserves the elevation data that was lost on my last attempt- no plateaus on all the higher mountaintops. Still has the streaks, though. I'll look at it in Wilbur when I get home.

EDIT--- Huh. Wilbur doesn't read TIFFs. I have to convert it into a PNG, and I do that in GIMP, which can only handle 8 bits, so I lose some data anyway. The contours look very pixelated. I'll start hand-correcting what I have, unless Redrobes manages to produce a miracle for me...

acrsome
05-05-2014, 09:45 AM
For anyone who cares- it is a heck of a lot easier to patch blank spots in the data with the clone tool in GIMP than in Wilbur. Once I get this done I'll work on the streaks, which I suspect will be the opposite- easier to manipulate in Wilbur.

Say, what's the difference between simple cylindrical and equirectangular cylindrical? My google-fu is failing me. Near as I can tell they are the same, but my simpe cylindrical images seem to lack the poles. I'm not sure if this is a characteristic of the projection or if the data is simply missing. I've certainly seen maps made using the same data that don't have such large holes, but they may have filled some in with Venera or Pioneer data or something.

Redrobes
05-05-2014, 06:22 PM
Before I do anything I can I check the link. Did you order it using something like this:

Map-a-planet (http://www.mapaplanet.org/explorer-bin/explorer.cgi?map=Venus&layers=venus_gtdr&info=NO&advoption=YES&lines=361&samples=721&sizeSelector=resolution&Resolution=22.76&projection=SIMP&grid=none&stretch=auto&resamp_method=nearest_neighbor&north=90&west=-180&east=180&south=-90&center=0&defaultcenter=on&center_lat=0)

and get the email link through for download ?

acrsome
05-06-2014, 01:58 PM
Yes, that's the website I ordered it from. They then send you a link for download. (But the link I posted above to download the file has expired- we'd have to order a new one.)

Alternately, what's a decent filesharing schema and I'll post it- though I've never done that before. Or I could email it to you. (If you trust me.)

I've been working on patching the blank spots in the data, and I played with the contrast and brightness so that the map uses the full 255 grayscale colors. It's starting to look useable. I still haven't started working on the streaks, though. If you can easily correct them then you're a better man than I. I'll probably start fiddling with them by hand next.

On that note, in Wilbur is there a way to select a region and drop it by a defined amount? For instance, so that every point has it's altitude is lowered by X?

Redrobes
05-06-2014, 02:47 PM
Ok I have requested the image, got the link and downloaded it and then done some filtering on it. I uploaded the unfiltered version Here (http://www.viewing.ltd.uk/Temp/CG/venus/venus_unfiltered.jpg) and the filtered version Here (http://www.viewing.ltd.uk/Temp/CG/venus/venus_filtered.png). Both are 13Mb images so I may remove them from my download areas after a while. I hope they are useful. I cant fill in the blanks but it should provide you with some better data around the stripes bit. I recommend cutting out bits of the filtered version as required instead of using it as the main basis. The filtering does adjust the data somewhat.

acrsome
05-06-2014, 07:55 PM
Wow. Your filter did a great job, and I'm sure that I can easily manipulate parts into another file as you mentioned, using layers in GIMP. But unfortunately you got sent a "bad" file to begin with- the grayscale is maxed out, so all the mountains look like plateaus, again. (Aka height clipping?) For some reason depending upon your settings on the Map-A-Planet site that happens sometimes, I've discovered.

However, I have delved into the arcana of filesharing, if you'd like to see what I'm working on. I got a decent version from Map-A-Planet and re-did the histogram so all 256 grayscale's are used with white (255) defined as the highest point on the planet and others scaled down from there to black (0) being the lowest point on the planet. I've filled in a few of the holes in the data, but not all.

The streaks are still there- I decided that just cloning looks like crap, so I re-started fixing the holes in the data by hand, and I'm far from done.

I just realized that the file I'm working on is probably small enough to post here:

63798

Which is a bit disappointing, actually. I've clearly lost something. The original bitmaps are 13MB, but this is only 3.6MB. How did I lose all of that? And, indeed, if you look at my image it is much more "contoury."

Grr. Well, I'm off to play around with this a bit more...

acrsome
05-06-2014, 10:37 PM
So, here are some more download links from Map-A-Planet, in various formats, and they all look good (no height clipping). Even the JPEGs. Links are good until May 10th.

RAW (http://pdsmaps.wr.usgs.gov/order/dean/venus00n000_4.html)

JPEG (http://pdsmaps.wr.usgs.gov/order/dean/venus00n000_3.html)

GIF (http://pdsmaps.wr.usgs.gov/order/dean/venus00n000_1.html)

TIFF (http://pdsmaps.wr.usgs.gov/order/dean/venus00n000.html)

So, I'm starting over. Again. But, hell, at least I'm learning new stuff about GIMP and Wilbur. If anything these files look too good- the resolution is fine enough that all sorts of streaks and artifacts are visible, and I doubt that I can just extrapolate the blank spaces by hand. I'll have to Clone, which I have yet to get the hang of without it looking obvious. And I'm looking forward to seeing how well the Smooth function works in Wilbur.

Eventually I'm planning on breaking the map down into 5x5-degree-ish (depending upon latitude) regional maps.

Redrobes
05-08-2014, 06:27 AM
I uploaded a new unfiltered version Here (http://viewing.ltd.uk/Temp/CG/Venus/Venus_Unfiltered2.jpg) and the filtered version Here (http://viewing.ltd.uk/Temp/CG/Venus/Venus_Filtered2.png). These are based on the JPG link you provided.

acrsome
05-08-2014, 09:39 AM
Awesome! Thanks! If I could rep you I would. Hell- I'll try, anyway.

If anything, that looks even better than the previous one. Clearly, whatever process you used adds some streaking, too- around the blank spots for instance- but the original ones are gone and as you recommended I can cut/paste around a bit. So, now back to some drudgery... This is going to take a while.